Jul 07 18:46:10.793548 ip-10-0-140-120 systemd[1]: Starting Kubernetes Kubelet... Jul 07 18:46:11.126774 ip-10-0-140-120 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 18:46:11.126774 ip-10-0-140-120 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 18:46:11.126774 ip-10-0-140-120 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 18:46:11.126774 ip-10-0-140-120 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 18:46:11.126774 ip-10-0-140-120 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 18:46:11.128921 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.128664 2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 18:46:11.197111 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133296 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 18:46:11.197111 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133309 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 18:46:11.197111 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133313 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 18:46:11.197111 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133316 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 18:46:11.197111 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133319 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 18:46:11.197111 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133322 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 18:46:11.197111 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133325 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 18:46:11.197111 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133327 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 18:46:11.197111 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133330 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 18:46:11.197111 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133333 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 18:46:11.197111 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133336 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 18:46:11.197111 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133338 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 18:46:11.197111 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133341 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 18:46:11.197111 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133344 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 18:46:11.197111 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133346 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 18:46:11.197111 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133349 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 18:46:11.197111 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133351 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 18:46:11.197111 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133355 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 18:46:11.197111 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133358 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 18:46:11.197111 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133362 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 18:46:11.197755 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133364 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 18:46:11.197755 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133367 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 18:46:11.197755 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133370 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 18:46:11.197755 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133372 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 18:46:11.197755 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133376 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 18:46:11.197755 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133379 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 18:46:11.197755 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133381 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 18:46:11.197755 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133384 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 18:46:11.197755 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133386 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 18:46:11.197755 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133389 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 18:46:11.197755 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133392 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 18:46:11.197755 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133394 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 18:46:11.197755 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133397 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 18:46:11.197755 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133401 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 18:46:11.197755 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133404 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 18:46:11.197755 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133406 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 18:46:11.197755 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133409 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 18:46:11.197755 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133411 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 18:46:11.197755 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133414 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 18:46:11.198423 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133416 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 18:46:11.198423 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133419 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 18:46:11.198423 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133421 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 18:46:11.198423 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133424 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 18:46:11.198423 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133428 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 18:46:11.198423 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133432 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 18:46:11.198423 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133435 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 18:46:11.198423 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133440 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 18:46:11.198423 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133444 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 18:46:11.198423 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133448 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 18:46:11.198423 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133451 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 18:46:11.198423 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133454 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 18:46:11.198423 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133457 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 18:46:11.198423 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133459 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 18:46:11.198423 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133463 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 18:46:11.198423 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133465 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 18:46:11.198423 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133468 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 18:46:11.198423 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133471 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 18:46:11.198423 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133474 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 18:46:11.199097 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133477 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 18:46:11.199097 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133480 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 18:46:11.199097 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133483 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 18:46:11.199097 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133485 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 18:46:11.199097 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133489 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 18:46:11.199097 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133492 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 18:46:11.199097 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133494 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 18:46:11.199097 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133497 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 18:46:11.199097 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133500 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 18:46:11.199097 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133502 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 18:46:11.199097 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133505 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 18:46:11.199097 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133511 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 18:46:11.199097 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133515 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 18:46:11.199097 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133518 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 18:46:11.199097 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133521 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 18:46:11.199097 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133524 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 18:46:11.199097 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133527 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 18:46:11.199097 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133530 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 18:46:11.199097 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133533 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 18:46:11.199097 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133536 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 18:46:11.199770 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133538 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 18:46:11.199770 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133541 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 18:46:11.199770 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133544 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 18:46:11.199770 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133546 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 18:46:11.199770 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133549 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 18:46:11.199770 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133551 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 18:46:11.199770 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133554 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 18:46:11.199770 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133556 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 18:46:11.199770 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133559 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 18:46:11.199770 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133562 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 18:46:11.199770 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133564 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 18:46:11.199770 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133567 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 18:46:11.199770 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133570 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 18:46:11.199770 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133572 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 18:46:11.199770 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133574 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 18:46:11.199770 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133578 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 18:46:11.199770 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133580 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 18:46:11.199770 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133583 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 18:46:11.199770 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133586 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 18:46:11.199770 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133589 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 07 18:46:11.200448 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.133591 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 18:46:11.200448 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134322 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 18:46:11.200448 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134329 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 18:46:11.200448 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134332 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 18:46:11.200448 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134335 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 18:46:11.200448 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134337 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 18:46:11.200448 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134340 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 18:46:11.200448 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134343 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 18:46:11.200448 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134345 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 18:46:11.200448 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134348 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 18:46:11.200448 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134351 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 18:46:11.200448 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134354 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 18:46:11.200448 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134357 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 18:46:11.200448 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134359 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 18:46:11.200448 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134362 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 18:46:11.200448 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134364 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 18:46:11.200448 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134367 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 18:46:11.200448 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134369 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 18:46:11.200448 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134372 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 18:46:11.200448 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134375 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 18:46:11.201113 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134378 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 18:46:11.201113 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134380 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 18:46:11.201113 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134383 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 18:46:11.201113 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134386 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 18:46:11.201113 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134388 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 18:46:11.201113 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134392 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 18:46:11.201113 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134396 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 18:46:11.201113 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134399 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 18:46:11.201113 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134403 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 18:46:11.201113 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134405 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 18:46:11.201113 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134408 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 18:46:11.201113 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134412 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 18:46:11.201113 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134416 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 18:46:11.201113 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134419 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 18:46:11.201113 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134422 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 18:46:11.201113 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134425 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 18:46:11.201113 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134428 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 18:46:11.201113 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134431 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 18:46:11.201113 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134434 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 18:46:11.201763 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134437 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 18:46:11.201763 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134440 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 18:46:11.201763 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134443 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 18:46:11.201763 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134446 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 07 18:46:11.201763 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134450 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 18:46:11.201763 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134452 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 18:46:11.201763 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134455 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 18:46:11.201763 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134457 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 18:46:11.201763 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134460 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 18:46:11.201763 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134463 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 18:46:11.201763 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134466 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 18:46:11.201763 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134468 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 18:46:11.201763 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134471 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 18:46:11.201763 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134473 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 18:46:11.201763 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134476 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 18:46:11.201763 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134478 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 18:46:11.201763 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134481 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 18:46:11.201763 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134484 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 18:46:11.201763 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134487 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 18:46:11.201763 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134490 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 18:46:11.202426 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134493 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 18:46:11.202426 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134496 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 18:46:11.202426 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134498 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 18:46:11.202426 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134501 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 18:46:11.202426 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134504 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 18:46:11.202426 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134512 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 18:46:11.202426 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134515 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 18:46:11.202426 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134519 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 18:46:11.202426 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134521 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 18:46:11.202426 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134524 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 18:46:11.202426 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134526 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 18:46:11.202426 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134529 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 18:46:11.202426 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134532 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 18:46:11.202426 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134534 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 18:46:11.202426 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134539 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 18:46:11.202426 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134542 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 18:46:11.202426 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134545 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 18:46:11.202426 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134549 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 18:46:11.202426 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134552 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 18:46:11.202426 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134554 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 18:46:11.203093 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134557 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 18:46:11.203093 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134560 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 18:46:11.203093 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134562 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 18:46:11.203093 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134565 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 18:46:11.203093 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134567 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 18:46:11.203093 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134570 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 18:46:11.203093 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134572 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 18:46:11.203093 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134575 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 18:46:11.203093 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134577 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 18:46:11.203093 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134580 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 18:46:11.203093 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134583 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 18:46:11.203093 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134585 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 18:46:11.203093 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134588 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 18:46:11.203093 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134590 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 18:46:11.203093 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134593 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 18:46:11.203093 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134595 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 18:46:11.203093 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134598 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 18:46:11.203093 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134602 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 18:46:11.203093 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134604 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 18:46:11.203093 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134607 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 18:46:11.203730 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.134609 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 18:46:11.203730 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135006 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 18:46:11.203730 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135011 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 18:46:11.203730 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135014 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 18:46:11.203730 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135016 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 18:46:11.203730 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135019 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 18:46:11.203730 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135022 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 18:46:11.203730 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135024 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 18:46:11.203730 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135027 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 18:46:11.203730 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135030 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 18:46:11.203730 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135033 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 18:46:11.203730 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135035 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 18:46:11.203730 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135038 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 18:46:11.203730 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135041 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 18:46:11.203730 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135045 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 18:46:11.203730 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135048 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 18:46:11.203730 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135052 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 18:46:11.203730 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135055 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 18:46:11.203730 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135058 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 18:46:11.203730 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135060 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 18:46:11.204603 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135063 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 18:46:11.204603 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135066 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 07 18:46:11.204603 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135068 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 18:46:11.204603 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135071 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 18:46:11.204603 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135073 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 18:46:11.204603 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135076 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 18:46:11.204603 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135078 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 18:46:11.204603 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135081 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 18:46:11.204603 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135084 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 18:46:11.204603 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135092 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 18:46:11.204603 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135096 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 18:46:11.204603 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135098 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 18:46:11.204603 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135101 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 18:46:11.204603 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135104 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 18:46:11.204603 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135106 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 18:46:11.204603 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135109 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 18:46:11.204603 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135111 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 18:46:11.204603 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135114 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 18:46:11.204603 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135117 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 18:46:11.204603 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135119 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 18:46:11.205267 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135122 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 18:46:11.205267 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135124 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 18:46:11.205267 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135127 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 18:46:11.205267 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135130 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 18:46:11.205267 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135132 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 18:46:11.205267 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135135 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 18:46:11.205267 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135137 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 18:46:11.205267 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135141 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 18:46:11.205267 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135145 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 18:46:11.205267 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135148 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 18:46:11.205267 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135151 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 18:46:11.205267 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135154 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 18:46:11.205267 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135156 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 18:46:11.205267 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135159 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 18:46:11.205267 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135162 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 18:46:11.205267 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135165 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 18:46:11.205267 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135167 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 18:46:11.205267 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135170 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 18:46:11.205267 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135172 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 18:46:11.205267 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135175 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 18:46:11.205901 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135178 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 18:46:11.205901 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135183 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 18:46:11.205901 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135186 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 18:46:11.205901 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135188 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 18:46:11.205901 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135191 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 18:46:11.205901 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135194 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 18:46:11.205901 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135196 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 18:46:11.205901 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135199 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 18:46:11.205901 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135201 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 18:46:11.205901 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135204 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 18:46:11.205901 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135206 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 18:46:11.205901 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135209 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 18:46:11.205901 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135211 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 18:46:11.205901 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135213 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 18:46:11.205901 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135216 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 18:46:11.205901 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135219 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 18:46:11.205901 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135222 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 18:46:11.205901 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135224 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 18:46:11.205901 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135227 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 18:46:11.206549 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135229 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 18:46:11.206549 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135233 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 18:46:11.206549 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135236 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 18:46:11.206549 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135238 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 18:46:11.206549 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135241 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 18:46:11.206549 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135243 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 18:46:11.206549 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135246 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 18:46:11.206549 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135248 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 18:46:11.206549 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135266 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 18:46:11.206549 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135269 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 18:46:11.206549 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135271 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 18:46:11.206549 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135274 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 18:46:11.206549 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135276 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 18:46:11.206549 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135279 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 18:46:11.206549 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135286 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 18:46:11.206549 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135290 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 18:46:11.206549 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135293 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 18:46:11.206549 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135296 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 18:46:11.206549 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135299 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 18:46:11.207213 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135302 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 18:46:11.207213 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.135304 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 18:46:11.207213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135379 2583 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 18:46:11.207213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135386 2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 18:46:11.207213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135392 2583 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 18:46:11.207213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135396 2583 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 18:46:11.207213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135401 2583 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 18:46:11.207213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135404 2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 18:46:11.207213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135409 2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 18:46:11.207213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135413 2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 18:46:11.207213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135417 2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 18:46:11.207213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135421 2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 18:46:11.207213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135424 2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 18:46:11.207213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135427 2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 18:46:11.207213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135430 2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 18:46:11.207213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135433 2583 flags.go:64] FLAG: --cgroup-root="" Jul 07 18:46:11.207213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135436 2583 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 18:46:11.207213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135439 2583 flags.go:64] FLAG: --client-ca-file="" Jul 07 18:46:11.207213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135442 2583 flags.go:64] FLAG: --cloud-provider="external" Jul 07 18:46:11.207213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135445 2583 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 18:46:11.207213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135450 2583 flags.go:64] FLAG: --cluster-domain="" Jul 07 18:46:11.207213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135453 2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 18:46:11.207213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135456 2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 18:46:11.207861 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135460 2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 18:46:11.207861 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135463 2583 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 18:46:11.207861 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135467 2583 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 18:46:11.207861 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135470 2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 18:46:11.207861 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135476 2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 18:46:11.207861 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135480 2583 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 18:46:11.207861 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135483 2583 flags.go:64] FLAG: --contention-profiling="false" Jul 07 18:46:11.207861 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135486 2583 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 18:46:11.207861 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135489 2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 18:46:11.207861 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135492 2583 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 18:46:11.207861 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135495 2583 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 18:46:11.207861 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135499 2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 18:46:11.207861 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135502 2583 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 18:46:11.207861 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135504 2583 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 18:46:11.207861 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135507 2583 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 18:46:11.207861 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135510 2583 flags.go:64] FLAG: --enable-server="true" Jul 07 18:46:11.207861 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135513 2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 18:46:11.207861 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135518 2583 flags.go:64] FLAG: --event-burst="100" Jul 07 18:46:11.207861 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135521 2583 flags.go:64] FLAG: --event-qps="50" Jul 07 18:46:11.207861 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135524 2583 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 18:46:11.207861 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135527 2583 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 18:46:11.207861 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135530 2583 flags.go:64] FLAG: --eviction-hard="" Jul 07 18:46:11.207861 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135534 2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 18:46:11.207861 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135538 2583 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 18:46:11.208742 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135540 2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 18:46:11.208742 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135543 2583 flags.go:64] FLAG: --eviction-soft="" Jul 07 18:46:11.208742 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135546 2583 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 18:46:11.208742 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135549 2583 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 18:46:11.208742 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135552 2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 18:46:11.208742 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135555 2583 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 18:46:11.208742 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135557 2583 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 18:46:11.208742 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135560 2583 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 18:46:11.208742 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135563 2583 flags.go:64] FLAG: --feature-gates="" Jul 07 18:46:11.208742 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135567 2583 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 18:46:11.208742 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135570 2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 18:46:11.208742 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135573 2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 18:46:11.208742 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135579 2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 18:46:11.208742 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135583 2583 flags.go:64] FLAG: --healthz-port="10248" Jul 07 18:46:11.208742 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135585 2583 flags.go:64] FLAG: --help="false" Jul 07 18:46:11.208742 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135588 2583 flags.go:64] FLAG: --hostname-override="ip-10-0-140-120.ec2.internal" Jul 07 18:46:11.208742 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135592 2583 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 18:46:11.208742 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135595 2583 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 18:46:11.208742 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135598 2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 18:46:11.208742 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135601 2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 18:46:11.208742 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135604 2583 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 18:46:11.208742 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135607 2583 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 18:46:11.208742 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135610 2583 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 18:46:11.209422 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135613 2583 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 18:46:11.209422 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135616 2583 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 18:46:11.209422 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135619 2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 18:46:11.209422 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135622 2583 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 18:46:11.209422 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135625 2583 flags.go:64] FLAG: --kube-reserved="" Jul 07 18:46:11.209422 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135627 2583 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 18:46:11.209422 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135631 2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 18:46:11.209422 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135634 2583 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 18:46:11.209422 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135637 2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 18:46:11.209422 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135640 2583 flags.go:64] FLAG: --lock-file="" Jul 07 18:46:11.209422 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135642 2583 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 18:46:11.209422 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135645 2583 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 18:46:11.209422 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135648 2583 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 18:46:11.209422 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135653 2583 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 18:46:11.209422 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135656 2583 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 18:46:11.209422 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135659 2583 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 18:46:11.209422 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135662 2583 flags.go:64] FLAG: --logging-format="text" Jul 07 18:46:11.209422 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135665 2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 18:46:11.209422 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135668 2583 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 18:46:11.209422 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135671 2583 flags.go:64] FLAG: --manifest-url="" Jul 07 18:46:11.209422 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135674 2583 flags.go:64] FLAG: --manifest-url-header="" Jul 07 18:46:11.209422 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135681 2583 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 18:46:11.209422 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135684 2583 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 18:46:11.209422 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135689 2583 flags.go:64] FLAG: --max-pods="110" Jul 07 18:46:11.209422 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135691 2583 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 18:46:11.213779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135694 2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 18:46:11.213779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135697 2583 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 18:46:11.213779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135700 2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 18:46:11.213779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135703 2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 18:46:11.213779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135706 2583 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 18:46:11.213779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135708 2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 18:46:11.213779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135716 2583 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 18:46:11.213779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135719 2583 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 18:46:11.213779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135722 2583 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 18:46:11.213779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135726 2583 flags.go:64] FLAG: --pod-cidr="" Jul 07 18:46:11.213779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135729 2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 18:46:11.213779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135734 2583 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 18:46:11.213779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135737 2583 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 18:46:11.213779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135740 2583 flags.go:64] FLAG: --pods-per-core="0" Jul 07 18:46:11.213779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135744 2583 flags.go:64] FLAG: --port="10250" Jul 07 18:46:11.213779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135747 2583 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 18:46:11.213779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135750 2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00436f45c470f4df2" Jul 07 18:46:11.213779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135753 2583 flags.go:64] FLAG: --qos-reserved="" Jul 07 18:46:11.213779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135756 2583 flags.go:64] FLAG: --read-only-port="10255" Jul 07 18:46:11.213779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135758 2583 flags.go:64] FLAG: --register-node="true" Jul 07 18:46:11.213779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135761 2583 flags.go:64] FLAG: --register-with-taints="" Jul 07 18:46:11.213779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135770 2583 flags.go:64] FLAG: --registry-burst="10" Jul 07 18:46:11.213779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135773 2583 flags.go:64] FLAG: --registry-qps="5" Jul 07 18:46:11.213779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135776 2583 flags.go:64] FLAG: --reserved-cpus="" Jul 07 18:46:11.269977 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135779 2583 flags.go:64] FLAG: --reserved-memory="" Jul 07 18:46:11.269977 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135782 2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 18:46:11.269977 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135785 2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 18:46:11.269977 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135788 2583 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 18:46:11.269977 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135793 2583 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 18:46:11.269977 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135797 2583 flags.go:64] FLAG: --runonce="false" Jul 07 18:46:11.269977 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135800 2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 18:46:11.269977 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135803 2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 18:46:11.269977 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135806 2583 flags.go:64] FLAG: --seccomp-default="false" Jul 07 18:46:11.269977 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135809 2583 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 18:46:11.269977 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135812 2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 18:46:11.269977 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135815 2583 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 18:46:11.269977 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135818 2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 18:46:11.269977 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135821 2583 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 18:46:11.269977 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135824 2583 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 18:46:11.269977 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135828 2583 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 18:46:11.269977 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135830 2583 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 18:46:11.269977 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135833 2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 18:46:11.269977 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135836 2583 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 18:46:11.269977 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135840 2583 flags.go:64] FLAG: --system-cgroups="" Jul 07 18:46:11.269977 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135842 2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 18:46:11.269977 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135848 2583 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 18:46:11.269977 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135851 2583 flags.go:64] FLAG: --tls-cert-file="" Jul 07 18:46:11.269977 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135854 2583 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 18:46:11.269977 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135858 2583 flags.go:64] FLAG: --tls-min-version="" Jul 07 18:46:11.236652 ip-10-0-140-120 systemd[1]: Started Kubernetes Kubelet. Jul 07 18:46:11.272331 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135861 2583 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 18:46:11.272331 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135864 2583 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 18:46:11.272331 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135867 2583 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 18:46:11.272331 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135870 2583 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 18:46:11.272331 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135873 2583 flags.go:64] FLAG: --v="2" Jul 07 18:46:11.272331 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135876 2583 flags.go:64] FLAG: --version="false" Jul 07 18:46:11.272331 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135880 2583 flags.go:64] FLAG: --vmodule="" Jul 07 18:46:11.272331 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135884 2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 18:46:11.272331 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.135887 2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 18:46:11.272331 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136000 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 18:46:11.272331 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136003 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 18:46:11.272331 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136007 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 18:46:11.272331 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136011 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 18:46:11.272331 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136013 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 18:46:11.272331 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136016 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 18:46:11.272331 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136019 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 18:46:11.272331 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136022 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 18:46:11.272331 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136024 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 18:46:11.272331 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136027 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 18:46:11.272331 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136030 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 18:46:11.272331 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136032 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 18:46:11.273519 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136036 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 18:46:11.273519 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136040 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 18:46:11.273519 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136043 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 18:46:11.273519 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136046 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 18:46:11.273519 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136049 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 18:46:11.273519 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136052 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 18:46:11.273519 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136054 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 18:46:11.273519 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136057 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 18:46:11.273519 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136061 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 18:46:11.273519 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136064 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 18:46:11.273519 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136070 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 18:46:11.273519 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136073 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 18:46:11.273519 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136076 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 18:46:11.273519 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136079 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 18:46:11.273519 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136081 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 18:46:11.273519 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136084 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 18:46:11.273519 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136086 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 18:46:11.273519 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136089 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 18:46:11.273519 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136091 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 18:46:11.274525 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136094 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 18:46:11.274525 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136097 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 18:46:11.274525 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136099 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 18:46:11.274525 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136103 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 18:46:11.274525 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136107 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 18:46:11.274525 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136109 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 18:46:11.274525 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136112 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 18:46:11.274525 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136114 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 07 18:46:11.274525 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136117 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 18:46:11.274525 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136119 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 18:46:11.274525 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136122 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 18:46:11.274525 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136124 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 18:46:11.274525 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136127 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 18:46:11.274525 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136129 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 18:46:11.274525 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136132 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 18:46:11.274525 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136134 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 18:46:11.274525 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136137 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 18:46:11.274525 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136139 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 18:46:11.274525 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136142 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 18:46:11.274525 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136144 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 18:46:11.275315 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136147 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 18:46:11.275315 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136149 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 18:46:11.275315 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136152 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 18:46:11.275315 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136154 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 18:46:11.275315 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136157 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 18:46:11.275315 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136160 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 18:46:11.275315 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136162 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 18:46:11.275315 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136164 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 18:46:11.275315 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136167 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 18:46:11.275315 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136170 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 18:46:11.275315 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136172 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 18:46:11.275315 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136175 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 18:46:11.275315 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136177 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 18:46:11.275315 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136180 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 18:46:11.275315 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136182 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 18:46:11.275315 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136186 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 18:46:11.275315 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136191 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 18:46:11.275315 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136195 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 18:46:11.275315 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136198 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 18:46:11.275921 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136201 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 18:46:11.275921 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136204 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 18:46:11.275921 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136206 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 18:46:11.275921 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136209 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 18:46:11.275921 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136212 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 18:46:11.275921 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136214 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 18:46:11.275921 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136217 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 18:46:11.275921 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136219 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 18:46:11.275921 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136222 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 18:46:11.275921 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136224 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 18:46:11.275921 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136227 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 18:46:11.275921 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136229 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 18:46:11.275921 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136232 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 18:46:11.275921 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136234 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 18:46:11.275921 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136237 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 18:46:11.275921 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136240 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 18:46:11.275921 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136242 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 18:46:11.275921 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136245 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 18:46:11.275921 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136248 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 18:46:11.275921 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136250 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 18:46:11.276643 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136271 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 18:46:11.276643 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136275 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 18:46:11.276643 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136279 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 18:46:11.276643 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136283 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 18:46:11.276643 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136285 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 18:46:11.276643 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136288 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 18:46:11.276643 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136291 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 18:46:11.276643 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136294 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 18:46:11.276643 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.136298 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 18:46:11.276643 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.136306 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 18:46:11.276643 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.144395 2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 18:46:11.276643 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.144422 2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 18:46:11.276643 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.144449 2583 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 18:46:11.276643 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.144456 2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 18:46:11.276643 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144540 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 18:46:11.277321 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144547 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 18:46:11.277321 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144552 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 18:46:11.277321 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144556 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 18:46:11.277321 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144560 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 18:46:11.277321 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144565 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 18:46:11.277321 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144568 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 18:46:11.277321 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144572 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 18:46:11.277321 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144576 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 18:46:11.277321 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144579 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 18:46:11.277321 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144583 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 18:46:11.277321 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144587 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 18:46:11.277321 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144591 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 18:46:11.277321 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144595 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 18:46:11.277321 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144599 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 18:46:11.277321 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144603 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 18:46:11.277321 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144607 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 18:46:11.277321 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144612 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 18:46:11.277321 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144618 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 18:46:11.277321 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144622 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 18:46:11.278165 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144626 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 18:46:11.278165 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144630 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 18:46:11.278165 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144634 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 18:46:11.278165 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144638 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 07 18:46:11.278165 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144642 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 18:46:11.278165 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144646 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 18:46:11.278165 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144649 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 18:46:11.278165 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144654 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 18:46:11.278165 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144661 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 18:46:11.278165 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144672 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 18:46:11.278165 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144677 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 18:46:11.278165 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144681 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 18:46:11.278165 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144685 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 18:46:11.278165 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144691 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 18:46:11.278165 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144697 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 18:46:11.278165 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144702 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 18:46:11.278165 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144706 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 18:46:11.278165 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144711 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 18:46:11.278165 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144715 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 18:46:11.278165 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144720 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 18:46:11.278960 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144724 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 18:46:11.278960 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144728 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 18:46:11.278960 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144732 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 18:46:11.278960 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144736 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 18:46:11.278960 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144740 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 18:46:11.278960 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144745 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 18:46:11.278960 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144748 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 18:46:11.278960 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144752 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 18:46:11.278960 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144756 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 18:46:11.278960 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144760 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 18:46:11.278960 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144763 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 18:46:11.278960 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144767 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 18:46:11.278960 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144771 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 18:46:11.278960 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144774 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 18:46:11.278960 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144778 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 18:46:11.278960 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144783 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 18:46:11.278960 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144786 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 18:46:11.278960 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144790 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 18:46:11.278960 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144794 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 18:46:11.278960 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144798 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 18:46:11.279866 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144804 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 18:46:11.279866 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144807 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 18:46:11.279866 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144821 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 18:46:11.279866 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144826 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 18:46:11.279866 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144830 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 18:46:11.279866 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144836 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 18:46:11.279866 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144839 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 18:46:11.279866 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144843 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 18:46:11.279866 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144847 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 18:46:11.279866 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144851 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 18:46:11.279866 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144855 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 18:46:11.279866 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144858 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 18:46:11.279866 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144862 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 18:46:11.279866 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144866 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 18:46:11.279866 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144869 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 18:46:11.279866 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144873 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 18:46:11.279866 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144877 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 18:46:11.279866 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144880 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 18:46:11.279866 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144884 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 18:46:11.280539 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144888 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 18:46:11.280539 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144892 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 18:46:11.280539 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144896 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 18:46:11.280539 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144900 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 18:46:11.280539 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144903 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 18:46:11.280539 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144908 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 18:46:11.280539 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144912 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 18:46:11.280539 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144916 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 18:46:11.280539 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144919 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 18:46:11.280539 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144923 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 18:46:11.280539 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144927 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 18:46:11.280539 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144930 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 18:46:11.280539 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144934 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 18:46:11.280539 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144940 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 18:46:11.280539 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144943 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 18:46:11.280539 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144947 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 18:46:11.280539 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144958 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 18:46:11.280539 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144962 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 18:46:11.280539 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144969 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 18:46:11.280539 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.144972 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 18:46:11.281179 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.144980 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 18:46:11.281179 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145187 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 18:46:11.281179 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145195 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 18:46:11.281179 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145199 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 18:46:11.281179 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145203 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 18:46:11.281179 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145207 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 18:46:11.281179 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145212 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 18:46:11.281179 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145216 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 18:46:11.281179 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145220 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 18:46:11.281179 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145223 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 18:46:11.281179 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145227 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 18:46:11.281179 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145231 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 18:46:11.281179 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145234 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 18:46:11.281179 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145238 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 18:46:11.281605 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145242 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 18:46:11.281605 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145246 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 18:46:11.281605 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145249 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 18:46:11.281605 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145269 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 18:46:11.281605 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145273 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 18:46:11.281605 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145290 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 18:46:11.281605 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145293 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 18:46:11.281605 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145297 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 18:46:11.281605 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145301 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 18:46:11.281605 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145305 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 18:46:11.281605 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145308 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 18:46:11.281605 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145313 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 18:46:11.281605 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145321 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 18:46:11.281605 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145325 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 18:46:11.281605 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145329 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 18:46:11.281605 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145341 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 18:46:11.281605 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145350 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 18:46:11.281605 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145356 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 18:46:11.281605 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145360 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 18:46:11.282213 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145365 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 18:46:11.282213 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145368 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 18:46:11.282213 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145373 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 18:46:11.282213 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145377 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 18:46:11.282213 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145381 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 18:46:11.282213 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145385 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 18:46:11.282213 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145389 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 07 18:46:11.282213 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145393 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 18:46:11.282213 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145397 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 18:46:11.282213 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145400 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 18:46:11.282213 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145404 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 18:46:11.282213 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145407 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 18:46:11.282213 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145412 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 18:46:11.282213 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145415 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 18:46:11.282213 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145419 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 18:46:11.282213 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145447 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 18:46:11.282213 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145453 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 18:46:11.282213 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145457 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 18:46:11.282213 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145461 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 18:46:11.282213 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145465 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 18:46:11.282832 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145469 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 18:46:11.282832 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145480 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 18:46:11.282832 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145484 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 18:46:11.282832 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145488 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 18:46:11.282832 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145492 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 18:46:11.282832 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145498 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 18:46:11.282832 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145502 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 18:46:11.282832 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145505 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 18:46:11.282832 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145509 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 18:46:11.282832 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145520 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 18:46:11.282832 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145526 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 18:46:11.282832 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145530 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 18:46:11.282832 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145536 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 18:46:11.282832 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145540 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 18:46:11.282832 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145544 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 18:46:11.282832 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145547 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 18:46:11.282832 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145551 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 18:46:11.282832 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145555 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 18:46:11.282832 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145559 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 18:46:11.282832 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145563 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 18:46:11.283460 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145566 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 18:46:11.283460 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145570 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 18:46:11.283460 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145574 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 18:46:11.283460 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145578 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 18:46:11.283460 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145582 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 18:46:11.283460 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145586 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 18:46:11.283460 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145589 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 18:46:11.283460 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145593 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 18:46:11.283460 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145597 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 18:46:11.283460 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145601 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 18:46:11.283460 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145604 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 18:46:11.283460 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145608 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 18:46:11.283460 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145612 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 18:46:11.283460 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145616 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 18:46:11.283460 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145620 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 18:46:11.283460 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145624 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 18:46:11.283460 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145628 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 18:46:11.283460 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145633 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 18:46:11.283460 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145637 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 18:46:11.283460 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145641 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 18:46:11.284073 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145644 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 18:46:11.284073 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145648 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 18:46:11.284073 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145660 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 18:46:11.284073 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145665 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 18:46:11.284073 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145668 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 18:46:11.284073 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145672 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 18:46:11.284073 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:11.145675 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 18:46:11.284073 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.145682 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 18:46:11.284073 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.145869 2583 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 18:46:11.284073 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.148420 2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 18:46:11.284073 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.150935 2583 server.go:1019] "Starting client certificate rotation" Jul 07 18:46:11.284073 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.151046 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 18:46:11.284073 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.151091 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 18:46:11.284483 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.153856 2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 18:46:11.284483 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.155042 2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 18:46:11.284483 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.168126 2583 log.go:25] "Validated CRI v1 runtime API" Jul 07 18:46:11.284483 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.170158 2583 log.go:25] "Validated CRI v1 image API" Jul 07 18:46:11.284483 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.170855 2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 18:46:11.284483 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.175684 2583 fs.go:135] Filesystem UUIDs: map[23dbcbf2-c871-4a10-a989-8b0d6fdd45f7:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 915cf2cd-aa4b-44be-a78b-f0292d1646c5:/dev/nvme0n1p3] Jul 07 18:46:11.284483 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.175700 2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 18:46:11.405747 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.180362 2583 manager.go:216] Machine: {Timestamp:2026-07-07 18:46:11.179236655 +0000 UTC m=+0.280255221 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099465 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2dce855477c36fcb93ec3eb74d01fa SystemUUID:ec2dce85-5477-c36f-cb93-ec3eb74d01fa BootID:0f7adb97-717a-461c-91ad-2a6ef8010153 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a8:05:33:1f:cb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a8:05:33:1f:cb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c2:ee:64:d7:2d:67 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 18:46:11.405747 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.180471 2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 18:46:11.405747 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.180558 2583 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 18:46:11.405747 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.183271 2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 18:46:11.405747 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.183304 2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-120.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 18:46:11.405747 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.184405 2583 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 18:46:11.405747 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.184420 2583 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 18:46:11.405747 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.184429 2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 18:46:11.405747 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.185027 2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 18:46:11.405747 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.186329 2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 18:46:11.405747 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.190396 2583 state_mem.go:36] "Initialized new in-memory state store" Jul 07 18:46:11.405747 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.190527 2583 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 18:46:11.405747 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.191127 2583 kubelet.go:486] "Attempting to sync node with API server" Jul 07 18:46:11.405747 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.191139 2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 18:46:11.405747 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.191155 2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 18:46:11.405747 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.191164 2583 kubelet.go:398] "Adding apiserver pod source" Jul 07 18:46:11.405747 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.191199 2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 18:46:11.405747 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.191555 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.191573 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.195988 2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.229135 2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.229190 2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.231110 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.231134 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.231144 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.231152 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.231166 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.231174 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.231186 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.231193 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.231203 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.231211 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.231234 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.231242 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.232020 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.232029 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:11.235477 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.235536 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-120.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:11.235658 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-120.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.235900 2583 server.go:1268] "Started kubelet" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.236369 2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.238278 2583 server.go:310] "Adding debug handlers to kubelet server" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.239296 2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.239345 2583 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.241166 2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:11.245960 2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.247778 2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.248538 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:11.251044 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-120.ec2.internal\" not found" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.251574 2583 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.251582 2583 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.251795 2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.252460 2583 reconstruct.go:97] "Volume reconstruction finished" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.252467 2583 reconciler.go:29] "Reconciler: start to sync state" Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.253207 2583 factory.go:55] Registering systemd factory Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.253219 2583 factory.go:223] Registration of the systemd container factory successfully Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.253456 2583 factory.go:153] Registering CRI-O factory Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.253469 2583 factory.go:223] Registration of the crio container factory successfully Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.253527 2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.253547 2583 factory.go:103] Registering Raw factory Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.253562 2583 manager.go:1196] Started watching for new ooms in manager Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.254015 2583 manager.go:349] Starting recovery of all containers Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.262945 2583 manager.go:354] Recovery completed Jul 07 18:46:11.409223 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:11.272900 2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-120.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 18:46:11.410892 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:11.273185 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 18:46:11.410892 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:11.272969 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-120.ec2.internal.18c0164cba31df18 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-120.ec2.internal,UID:ip-10-0-140-120.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-120.ec2.internal,},FirstTimestamp:2026-07-07 18:46:11.235872536 +0000 UTC m=+0.336891103,LastTimestamp:2026-07-07 18:46:11.235872536 +0000 UTC m=+0.336891103,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-120.ec2.internal,}" Jul 07 18:46:11.410892 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.277208 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:46:11.410892 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.286328 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 18:46:11.410892 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.286657 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-120.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:46:11.410892 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.286684 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-120.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:46:11.410892 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.286694 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-120.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:46:11.410892 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.287248 2583 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 18:46:11.410892 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.287284 2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 18:46:11.410892 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.287303 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:46:11.410892 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.287322 2583 state_mem.go:36] "Initialized new in-memory state store" Jul 07 18:46:11.410892 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.287951 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-120.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:46:11.410892 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.287977 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-120.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:46:11.410892 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.287988 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-120.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:46:11.410892 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.289533 2583 policy_none.go:49] "None policy: Start" Jul 07 18:46:11.410892 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.289546 2583 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 18:46:11.410892 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.289557 2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 18:46:11.410892 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.291544 2583 policy_none.go:47] "Start" Jul 07 18:46:11.410892 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:11.295824 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-120.ec2.internal.18c0164cbd38f8b7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-120.ec2.internal,UID:ip-10-0-140-120.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-120.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-120.ec2.internal,},FirstTimestamp:2026-07-07 18:46:11.286669495 +0000 UTC m=+0.387688061,LastTimestamp:2026-07-07 18:46:11.286669495 +0000 UTC m=+0.387688061,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-120.ec2.internal,}" Jul 07 18:46:11.410892 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:11.299659 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-120.ec2.internal.18c0164cbd39436c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-120.ec2.internal,UID:ip-10-0-140-120.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-120.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-120.ec2.internal,},FirstTimestamp:2026-07-07 18:46:11.28668862 +0000 UTC m=+0.387707186,LastTimestamp:2026-07-07 18:46:11.28668862 +0000 UTC m=+0.387707186,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-120.ec2.internal,}" Jul 07 18:46:11.410892 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.302837 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6cwsm" Jul 07 18:46:11.417864 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:11.306965 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-120.ec2.internal.18c0164cbd396886 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-120.ec2.internal,UID:ip-10-0-140-120.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-140-120.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-140-120.ec2.internal,},FirstTimestamp:2026-07-07 18:46:11.286698118 +0000 UTC m=+0.387716685,LastTimestamp:2026-07-07 18:46:11.286698118 +0000 UTC m=+0.387716685,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-120.ec2.internal,}" Jul 07 18:46:11.417864 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:11.309951 2583 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-140-120.ec2.internal.18c0164cbd38f8b7\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-120.ec2.internal.18c0164cbd38f8b7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-120.ec2.internal,UID:ip-10-0-140-120.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-120.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-120.ec2.internal,},FirstTimestamp:2026-07-07 18:46:11.286669495 +0000 UTC m=+0.387688061,LastTimestamp:2026-07-07 18:46:11.287962273 +0000 UTC m=+0.388980845,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-120.ec2.internal,}" Jul 07 18:46:11.417864 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:11.316331 2583 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-140-120.ec2.internal.18c0164cbd39436c\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-120.ec2.internal.18c0164cbd39436c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-120.ec2.internal,UID:ip-10-0-140-120.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-120.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-120.ec2.internal,},FirstTimestamp:2026-07-07 18:46:11.28668862 +0000 UTC m=+0.387707186,LastTimestamp:2026-07-07 18:46:11.287982918 +0000 UTC m=+0.389001484,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-120.ec2.internal,}" Jul 07 18:46:11.417864 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.320724 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6cwsm" Jul 07 18:46:11.417864 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.335115 2583 manager.go:337] "Starting Device Plugin manager" Jul 07 18:46:11.417864 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:11.335159 2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 18:46:11.417864 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.335171 2583 server.go:87] "Starting device plugin registration server" Jul 07 18:46:11.417864 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.335505 2583 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 18:46:11.417864 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.335517 2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 18:46:11.417864 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.335812 2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 18:46:11.417864 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.335882 2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 18:46:11.417864 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.335890 2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 18:46:11.417864 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:11.336437 2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 18:46:11.417864 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:11.336473 2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-120.ec2.internal\" not found" Jul 07 18:46:11.417864 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.406421 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 18:46:11.417864 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.407910 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 18:46:11.417864 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.407948 2583 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 18:46:11.417864 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.407977 2583 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 18:46:11.417864 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:11.408018 2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 18:46:11.502107 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.422096 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 18:46:11.502107 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.436490 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:46:11.502107 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.438685 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-120.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:46:11.502107 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.438709 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-120.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:46:11.502107 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.438721 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-120.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:46:11.502107 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.438745 2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-120.ec2.internal" Jul 07 18:46:11.502107 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.447728 2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-120.ec2.internal" Jul 07 18:46:11.502107 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:11.447748 2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-120.ec2.internal\": node \"ip-10-0-140-120.ec2.internal\" not found" Jul 07 18:46:11.502107 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:11.486411 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-120.ec2.internal\" not found" Jul 07 18:46:11.592907 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.509066 2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-120.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-120.ec2.internal"] Jul 07 18:46:11.592907 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.509172 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:46:11.592907 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.511207 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-120.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:46:11.592907 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.511238 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-120.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:46:11.592907 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.511252 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-120.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:46:11.592907 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.513992 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:46:11.592907 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.514155 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-120.ec2.internal" Jul 07 18:46:11.592907 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.514182 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:46:11.592907 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.514961 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-120.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:46:11.592907 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.514981 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-120.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:46:11.592907 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.514991 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-120.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:46:11.592907 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.515393 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-120.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:46:11.592907 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.515410 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-120.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:46:11.592907 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.515420 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-120.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:46:11.592907 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.517511 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-120.ec2.internal" Jul 07 18:46:11.592907 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.517535 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:46:11.592907 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.519936 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-120.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:46:11.592907 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.519957 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-120.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:46:11.592907 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.519967 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-120.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:46:11.592907 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.569432 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe8c242d7468032595b33cd6dcda8cb2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-120.ec2.internal\" (UID: \"fe8c242d7468032595b33cd6dcda8cb2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-120.ec2.internal" Jul 07 18:46:11.592907 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.569455 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ee9c1d39d9fd4f723b1434f6d29f4c61-config\") pod \"kube-apiserver-proxy-ip-10-0-140-120.ec2.internal\" (UID: \"ee9c1d39d9fd4f723b1434f6d29f4c61\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-120.ec2.internal" Jul 07 18:46:11.592907 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.569485 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fe8c242d7468032595b33cd6dcda8cb2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-120.ec2.internal\" (UID: \"fe8c242d7468032595b33cd6dcda8cb2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-120.ec2.internal" Jul 07 18:46:11.592907 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:11.584326 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-120.ec2.internal\" not found" node="ip-10-0-140-120.ec2.internal" Jul 07 18:46:11.592907 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:11.586574 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-120.ec2.internal\" not found" Jul 07 18:46:11.592907 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:11.588978 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-120.ec2.internal\" not found" node="ip-10-0-140-120.ec2.internal" Jul 07 18:46:11.706962 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.669795 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ee9c1d39d9fd4f723b1434f6d29f4c61-config\") pod \"kube-apiserver-proxy-ip-10-0-140-120.ec2.internal\" (UID: \"ee9c1d39d9fd4f723b1434f6d29f4c61\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-120.ec2.internal" Jul 07 18:46:11.706962 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.669825 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fe8c242d7468032595b33cd6dcda8cb2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-120.ec2.internal\" (UID: \"fe8c242d7468032595b33cd6dcda8cb2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-120.ec2.internal" Jul 07 18:46:11.706962 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.669845 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe8c242d7468032595b33cd6dcda8cb2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-120.ec2.internal\" (UID: \"fe8c242d7468032595b33cd6dcda8cb2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-120.ec2.internal" Jul 07 18:46:11.706962 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.669898 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ee9c1d39d9fd4f723b1434f6d29f4c61-config\") pod \"kube-apiserver-proxy-ip-10-0-140-120.ec2.internal\" (UID: \"ee9c1d39d9fd4f723b1434f6d29f4c61\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-120.ec2.internal" Jul 07 18:46:11.706962 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.669921 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe8c242d7468032595b33cd6dcda8cb2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-120.ec2.internal\" (UID: \"fe8c242d7468032595b33cd6dcda8cb2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-120.ec2.internal" Jul 07 18:46:11.706962 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.669898 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fe8c242d7468032595b33cd6dcda8cb2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-120.ec2.internal\" (UID: \"fe8c242d7468032595b33cd6dcda8cb2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-120.ec2.internal" Jul 07 18:46:11.706962 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:11.686858 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-120.ec2.internal\" not found" Jul 07 18:46:11.842411 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:11.787576 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-120.ec2.internal\" not found" Jul 07 18:46:11.971714 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:11.888841 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-120.ec2.internal\" not found" Jul 07 18:46:11.971714 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.888897 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-120.ec2.internal" Jul 07 18:46:11.971714 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:11.892588 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-120.ec2.internal" Jul 07 18:46:12.086356 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:11.989920 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-120.ec2.internal\" not found" Jul 07 18:46:12.179715 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:12.090852 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-120.ec2.internal\" not found" Jul 07 18:46:12.179715 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:12.153834 2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 18:46:12.179715 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:12.154002 2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 18:46:12.272708 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:12.191320 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-120.ec2.internal\" not found" Jul 07 18:46:12.272708 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:12.257200 2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 18:46:12.352783 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:12.292436 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-120.ec2.internal\" not found" Jul 07 18:46:12.352783 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:12.325043 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 18:41:11 +0000 UTC" deadline="2028-01-14 14:02:02.534901327 +0000 UTC" Jul 07 18:46:12.352783 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:12.325089 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13339h15m50.209816454s" Jul 07 18:46:12.365344 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:12.363793 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:46:12.365344 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:12.365026 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-120.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:46:12.365344 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:12.365070 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-120.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:46:12.365344 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:12.365084 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-120.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:46:12.449211 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:12.393233 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-120.ec2.internal\" not found" Jul 07 18:46:12.565010 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:12.474018 2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 18:46:12.565010 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:12.553303 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-120.ec2.internal" Jul 07 18:46:12.565010 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:12.562242 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 18:46:12.565010 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:12.563067 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-120.ec2.internal" Jul 07 18:46:12.647631 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:12.577210 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 18:46:12.647631 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:12.642821 2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 18:46:12.735773 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:12.652132 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee9c1d39d9fd4f723b1434f6d29f4c61.slice/crio-2bd2821bcb6969da59fbeb65543e83305cba706578b5a286871c39ce3acb9ada WatchSource:0}: Error finding container 2bd2821bcb6969da59fbeb65543e83305cba706578b5a286871c39ce3acb9ada: Status 404 returned error can't find the container with id 2bd2821bcb6969da59fbeb65543e83305cba706578b5a286871c39ce3acb9ada Jul 07 18:46:12.735773 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:12.657223 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 18:46:12.735773 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:12.687974 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe8c242d7468032595b33cd6dcda8cb2.slice/crio-1010e04fc4e348a17ce7e7341cac1d98647a885da4a5169eef6010efe5bb7dda WatchSource:0}: Error finding container 1010e04fc4e348a17ce7e7341cac1d98647a885da4a5169eef6010efe5bb7dda: Status 404 returned error can't find the container with id 1010e04fc4e348a17ce7e7341cac1d98647a885da4a5169eef6010efe5bb7dda Jul 07 18:46:12.985741 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:12.985650 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 18:46:13.194596 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.194555 2583 apiserver.go:52] "Watching apiserver" Jul 07 18:46:13.199792 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.199763 2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 18:46:13.200219 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.200179 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-84cwc","openshift-multus/multus-f9lks","openshift-network-diagnostics/network-check-target-l5d2z","openshift-network-operator/iptables-alerter-2w8tr","kube-system/global-pull-secret-syncer-c6dfn","kube-system/konnectivity-agent-8khjj","kube-system/kube-apiserver-proxy-ip-10-0-140-120.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-120.ec2.internal","openshift-multus/network-metrics-daemon-cpp9l","openshift-ovn-kubernetes/ovnkube-node-9j7xz","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h","openshift-cluster-node-tuning-operator/tuned-xlzmh","openshift-dns/node-resolver-ztg6h","openshift-image-registry/node-ca-rwjnv"] Jul 07 18:46:13.203102 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.203076 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ztg6h" Jul 07 18:46:13.205455 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.205390 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vh4hg\"" Jul 07 18:46:13.205455 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.205402 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 18:46:13.205455 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.205393 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 18:46:13.205675 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.205635 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rwjnv" Jul 07 18:46:13.207578 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.207559 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 18:46:13.207956 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.207814 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 18:46:13.207956 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.207935 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lt9cm\"" Jul 07 18:46:13.208127 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.208098 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2w8tr" Jul 07 18:46:13.208513 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.208243 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 18:46:13.210656 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.210439 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ks2xm\"" Jul 07 18:46:13.210656 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.210485 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 18:46:13.210656 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.210594 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 18:46:13.210833 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.210674 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:13.210833 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.210702 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 18:46:13.210833 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:13.210736 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6dfn" podUID="7cc80f14-06d5-483d-b15b-a44896b686a6" Jul 07 18:46:13.212856 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.212834 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8khjj" Jul 07 18:46:13.215001 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.214910 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 18:46:13.215001 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.214986 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8m7xn\"" Jul 07 18:46:13.215001 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.215003 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 18:46:13.215337 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.215262 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-84cwc" Jul 07 18:46:13.217230 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.217210 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 18:46:13.217403 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.217364 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4sppc\"" Jul 07 18:46:13.217476 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.217465 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 18:46:13.217631 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.217614 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 18:46:13.217701 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.217630 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 18:46:13.217760 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.217743 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 18:46:13.217928 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.217879 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:13.218026 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:13.217965 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cpp9l" podUID="329b4a4f-5489-4021-ab98-bcc7cc5b7dd0" Jul 07 18:46:13.220460 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.220439 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.222541 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.222521 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 18:46:13.222642 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.222522 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 18:46:13.223147 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.222768 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 18:46:13.223147 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.222778 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 18:46:13.223147 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.222839 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 18:46:13.223147 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.222943 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8zbl7\"" Jul 07 18:46:13.223550 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.223531 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 18:46:13.224077 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.223673 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" Jul 07 18:46:13.225665 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.225645 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 18:46:13.226017 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.225962 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 18:46:13.226017 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.225962 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 18:46:13.226542 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.226326 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.227111 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.227092 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zgx4l\"" Jul 07 18:46:13.228416 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.228396 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 18:46:13.228932 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.228813 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 18:46:13.229441 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.229417 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:46:13.230018 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:13.229721 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5d2z" podUID="c7296533-de2d-44a3-8972-5e1b8baeaae2" Jul 07 18:46:13.230018 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.229901 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vc5vl\"" Jul 07 18:46:13.233869 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.233846 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.237073 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.236753 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-x8fnd\"" Jul 07 18:46:13.237073 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.236758 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 18:46:13.251826 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.251796 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 18:46:13.253407 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.253381 2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 18:46:13.270571 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.270543 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 18:46:13.281337 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.281285 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xrcr\" (UniqueName: \"kubernetes.io/projected/5d65ba38-b541-4fe1-8933-4f6f8d366f4d-kube-api-access-4xrcr\") pod \"iptables-alerter-2w8tr\" (UID: \"5d65ba38-b541-4fe1-8933-4f6f8d366f4d\") " pod="openshift-network-operator/iptables-alerter-2w8tr" Jul 07 18:46:13.281461 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.281349 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-tmp\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.281461 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.281379 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-host-run-k8s-cni-cncf-io\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.281461 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.281404 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg6jg\" (UniqueName: \"kubernetes.io/projected/c1e5d07f-f71c-4423-9ef0-882c19222314-kube-api-access-lg6jg\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.281461 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.281430 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/40b947e9-e3e6-43ec-8b88-ff421f627227-host\") pod \"node-ca-rwjnv\" (UID: \"40b947e9-e3e6-43ec-8b88-ff421f627227\") " pod="openshift-image-registry/node-ca-rwjnv" Jul 07 18:46:13.281461 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.281454 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-log-socket\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.281819 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.281481 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgm4p\" (UniqueName: \"kubernetes.io/projected/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-kube-api-access-fgm4p\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.281819 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.281509 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95flf\" (UniqueName: \"kubernetes.io/projected/8b9fb9da-073a-44c8-bf4d-2ee70fa748b9-kube-api-access-95flf\") pod \"node-resolver-ztg6h\" (UID: \"8b9fb9da-073a-44c8-bf4d-2ee70fa748b9\") " pod="openshift-dns/node-resolver-ztg6h" Jul 07 18:46:13.281819 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.281539 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-etc-kubernetes\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.281819 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.281567 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-etc-sysctl-d\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.281819 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.281595 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30caa190-8d65-4825-a681-4af09975ef3e-cnibin\") pod \"multus-additional-cni-plugins-84cwc\" (UID: \"30caa190-8d65-4825-a681-4af09975ef3e\") " pod="openshift-multus/multus-additional-cni-plugins-84cwc" Jul 07 18:46:13.281819 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.281619 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-run-systemd\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.281819 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.281643 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-run-ovn\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.281819 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.281667 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-ovnkube-config\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.281819 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.281691 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-cnibin\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.281819 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.281715 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-node-log\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.281819 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.281742 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4f829b0a-c000-4612-8087-24e9f1f1018c-socket-dir\") pod \"aws-ebs-csi-driver-node-zrv7h\" (UID: \"4f829b0a-c000-4612-8087-24e9f1f1018c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" Jul 07 18:46:13.281819 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.281767 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-sys\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.281819 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.281789 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-hostroot\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.282477 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.281846 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4m28\" (UniqueName: \"kubernetes.io/projected/329b4a4f-5489-4021-ab98-bcc7cc5b7dd0-kube-api-access-v4m28\") pod \"network-metrics-daemon-cpp9l\" (UID: \"329b4a4f-5489-4021-ab98-bcc7cc5b7dd0\") " pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:13.282477 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.281872 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-var-lib-openvswitch\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.282477 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.281965 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-ovn-node-metrics-cert\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.282477 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.281990 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-etc-sysctl-conf\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.282477 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.282014 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-lib-modules\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.282477 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.282037 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-host\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.282477 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.282061 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-multus-conf-dir\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.282477 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.282164 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e48b5a6a-4b69-4e2a-823c-656ce51dc9e8-agent-certs\") pod \"konnectivity-agent-8khjj\" (UID: \"e48b5a6a-4b69-4e2a-823c-656ce51dc9e8\") " pod="kube-system/konnectivity-agent-8khjj" Jul 07 18:46:13.282477 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.282190 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30caa190-8d65-4825-a681-4af09975ef3e-cni-binary-copy\") pod \"multus-additional-cni-plugins-84cwc\" (UID: \"30caa190-8d65-4825-a681-4af09975ef3e\") " pod="openshift-multus/multus-additional-cni-plugins-84cwc" Jul 07 18:46:13.282477 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.282244 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-host-cni-bin\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.282477 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.282288 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nr5d\" (UniqueName: \"kubernetes.io/projected/4f829b0a-c000-4612-8087-24e9f1f1018c-kube-api-access-9nr5d\") pod \"aws-ebs-csi-driver-node-zrv7h\" (UID: \"4f829b0a-c000-4612-8087-24e9f1f1018c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" Jul 07 18:46:13.282477 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.282351 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-host-var-lib-cni-multus\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.283054 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.282525 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-etc-openvswitch\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.283054 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.282578 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-multus-socket-dir-parent\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.283054 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.282607 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-host-var-lib-cni-bin\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.283054 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.282642 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-host-var-lib-kubelet\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.283054 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.282677 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8b9fb9da-073a-44c8-bf4d-2ee70fa748b9-hosts-file\") pod \"node-resolver-ztg6h\" (UID: \"8b9fb9da-073a-44c8-bf4d-2ee70fa748b9\") " pod="openshift-dns/node-resolver-ztg6h" Jul 07 18:46:13.283054 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.282702 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-host-run-ovn-kubernetes\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.283054 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.282727 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4f829b0a-c000-4612-8087-24e9f1f1018c-device-dir\") pod \"aws-ebs-csi-driver-node-zrv7h\" (UID: \"4f829b0a-c000-4612-8087-24e9f1f1018c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" Jul 07 18:46:13.283054 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.282750 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-multus-cni-dir\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.283054 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.282783 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/30caa190-8d65-4825-a681-4af09975ef3e-os-release\") pod \"multus-additional-cni-plugins-84cwc\" (UID: \"30caa190-8d65-4825-a681-4af09975ef3e\") " pod="openshift-multus/multus-additional-cni-plugins-84cwc" Jul 07 18:46:13.283054 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.282812 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/30caa190-8d65-4825-a681-4af09975ef3e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-84cwc\" (UID: \"30caa190-8d65-4825-a681-4af09975ef3e\") " pod="openshift-multus/multus-additional-cni-plugins-84cwc" Jul 07 18:46:13.283054 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.282840 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-host-slash\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.283054 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.282864 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcfn5\" (UniqueName: \"kubernetes.io/projected/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-kube-api-access-fcfn5\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.283054 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.282887 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-host-run-netns\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.283054 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.282942 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b9fb9da-073a-44c8-bf4d-2ee70fa748b9-tmp-dir\") pod \"node-resolver-ztg6h\" (UID: \"8b9fb9da-073a-44c8-bf4d-2ee70fa748b9\") " pod="openshift-dns/node-resolver-ztg6h" Jul 07 18:46:13.283054 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.282966 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/40b947e9-e3e6-43ec-8b88-ff421f627227-serviceca\") pod \"node-ca-rwjnv\" (UID: \"40b947e9-e3e6-43ec-8b88-ff421f627227\") " pod="openshift-image-registry/node-ca-rwjnv" Jul 07 18:46:13.283054 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.282994 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/30caa190-8d65-4825-a681-4af09975ef3e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-84cwc\" (UID: \"30caa190-8d65-4825-a681-4af09975ef3e\") " pod="openshift-multus/multus-additional-cni-plugins-84cwc" Jul 07 18:46:13.283890 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283021 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4f829b0a-c000-4612-8087-24e9f1f1018c-sys-fs\") pod \"aws-ebs-csi-driver-node-zrv7h\" (UID: \"4f829b0a-c000-4612-8087-24e9f1f1018c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" Jul 07 18:46:13.283890 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283044 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7cc80f14-06d5-483d-b15b-a44896b686a6-dbus\") pod \"global-pull-secret-syncer-c6dfn\" (UID: \"7cc80f14-06d5-483d-b15b-a44896b686a6\") " pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:13.283890 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283069 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cc80f14-06d5-483d-b15b-a44896b686a6-original-pull-secret\") pod \"global-pull-secret-syncer-c6dfn\" (UID: \"7cc80f14-06d5-483d-b15b-a44896b686a6\") " pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:13.283890 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283094 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-var-lib-kubelet\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.283890 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283128 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-os-release\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.283890 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283181 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-host-run-multus-certs\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.283890 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283213 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lwxm\" (UniqueName: \"kubernetes.io/projected/40b947e9-e3e6-43ec-8b88-ff421f627227-kube-api-access-7lwxm\") pod \"node-ca-rwjnv\" (UID: \"40b947e9-e3e6-43ec-8b88-ff421f627227\") " pod="openshift-image-registry/node-ca-rwjnv" Jul 07 18:46:13.283890 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283242 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-etc-sysconfig\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.283890 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283290 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-system-cni-dir\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.283890 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283332 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e48b5a6a-4b69-4e2a-823c-656ce51dc9e8-konnectivity-ca\") pod \"konnectivity-agent-8khjj\" (UID: \"e48b5a6a-4b69-4e2a-823c-656ce51dc9e8\") " pod="kube-system/konnectivity-agent-8khjj" Jul 07 18:46:13.283890 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283358 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30caa190-8d65-4825-a681-4af09975ef3e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-84cwc\" (UID: \"30caa190-8d65-4825-a681-4af09975ef3e\") " pod="openshift-multus/multus-additional-cni-plugins-84cwc" Jul 07 18:46:13.283890 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283383 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-run-openvswitch\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.283890 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283412 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4f829b0a-c000-4612-8087-24e9f1f1018c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zrv7h\" (UID: \"4f829b0a-c000-4612-8087-24e9f1f1018c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" Jul 07 18:46:13.283890 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283435 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4f829b0a-c000-4612-8087-24e9f1f1018c-etc-selinux\") pod \"aws-ebs-csi-driver-node-zrv7h\" (UID: \"4f829b0a-c000-4612-8087-24e9f1f1018c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" Jul 07 18:46:13.283890 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283474 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7cc80f14-06d5-483d-b15b-a44896b686a6-kubelet-config\") pod \"global-pull-secret-syncer-c6dfn\" (UID: \"7cc80f14-06d5-483d-b15b-a44896b686a6\") " pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:13.283890 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283519 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-etc-systemd\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.284812 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283545 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30caa190-8d65-4825-a681-4af09975ef3e-system-cni-dir\") pod \"multus-additional-cni-plugins-84cwc\" (UID: \"30caa190-8d65-4825-a681-4af09975ef3e\") " pod="openshift-multus/multus-additional-cni-plugins-84cwc" Jul 07 18:46:13.284812 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283571 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p8b6\" (UniqueName: \"kubernetes.io/projected/30caa190-8d65-4825-a681-4af09975ef3e-kube-api-access-6p8b6\") pod \"multus-additional-cni-plugins-84cwc\" (UID: \"30caa190-8d65-4825-a681-4af09975ef3e\") " pod="openshift-multus/multus-additional-cni-plugins-84cwc" Jul 07 18:46:13.284812 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283597 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-host-run-netns\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.284812 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283624 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.284812 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283654 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-env-overrides\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.284812 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283692 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4f829b0a-c000-4612-8087-24e9f1f1018c-registration-dir\") pod \"aws-ebs-csi-driver-node-zrv7h\" (UID: \"4f829b0a-c000-4612-8087-24e9f1f1018c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" Jul 07 18:46:13.284812 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283748 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-run\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.284812 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283774 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgl6b\" (UniqueName: \"kubernetes.io/projected/c7296533-de2d-44a3-8972-5e1b8baeaae2-kube-api-access-sgl6b\") pod \"network-check-target-l5d2z\" (UID: \"c7296533-de2d-44a3-8972-5e1b8baeaae2\") " pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:46:13.284812 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283805 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c1e5d07f-f71c-4423-9ef0-882c19222314-cni-binary-copy\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.284812 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283830 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-etc-kubernetes\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.284812 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283855 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/329b4a4f-5489-4021-ab98-bcc7cc5b7dd0-metrics-certs\") pod \"network-metrics-daemon-cpp9l\" (UID: \"329b4a4f-5489-4021-ab98-bcc7cc5b7dd0\") " pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:13.284812 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283880 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-systemd-units\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.284812 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283920 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-ovnkube-script-lib\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.284812 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283947 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d65ba38-b541-4fe1-8933-4f6f8d366f4d-host-slash\") pod \"iptables-alerter-2w8tr\" (UID: \"5d65ba38-b541-4fe1-8933-4f6f8d366f4d\") " pod="openshift-network-operator/iptables-alerter-2w8tr" Jul 07 18:46:13.284812 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283971 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-host-kubelet\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.284812 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.283996 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-host-cni-netd\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.285559 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.284030 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5d65ba38-b541-4fe1-8933-4f6f8d366f4d-iptables-alerter-script\") pod \"iptables-alerter-2w8tr\" (UID: \"5d65ba38-b541-4fe1-8933-4f6f8d366f4d\") " pod="openshift-network-operator/iptables-alerter-2w8tr" Jul 07 18:46:13.285559 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.284059 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-etc-modprobe-d\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.285559 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.284083 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-etc-tuned\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.285559 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.284112 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c1e5d07f-f71c-4423-9ef0-882c19222314-multus-daemon-config\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.306983 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.306930 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zkm7v" Jul 07 18:46:13.315375 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.315324 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zkm7v" Jul 07 18:46:13.384743 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.384694 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30caa190-8d65-4825-a681-4af09975ef3e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-84cwc\" (UID: \"30caa190-8d65-4825-a681-4af09975ef3e\") " pod="openshift-multus/multus-additional-cni-plugins-84cwc" Jul 07 18:46:13.384743 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.384738 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-run-openvswitch\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.384996 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.384766 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4f829b0a-c000-4612-8087-24e9f1f1018c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zrv7h\" (UID: \"4f829b0a-c000-4612-8087-24e9f1f1018c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" Jul 07 18:46:13.384996 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.384795 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4f829b0a-c000-4612-8087-24e9f1f1018c-etc-selinux\") pod \"aws-ebs-csi-driver-node-zrv7h\" (UID: \"4f829b0a-c000-4612-8087-24e9f1f1018c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" Jul 07 18:46:13.384996 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.384821 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7cc80f14-06d5-483d-b15b-a44896b686a6-kubelet-config\") pod \"global-pull-secret-syncer-c6dfn\" (UID: \"7cc80f14-06d5-483d-b15b-a44896b686a6\") " pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:13.384996 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.384848 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-etc-systemd\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.384996 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.384873 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30caa190-8d65-4825-a681-4af09975ef3e-system-cni-dir\") pod \"multus-additional-cni-plugins-84cwc\" (UID: \"30caa190-8d65-4825-a681-4af09975ef3e\") " pod="openshift-multus/multus-additional-cni-plugins-84cwc" Jul 07 18:46:13.384996 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.384901 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p8b6\" (UniqueName: \"kubernetes.io/projected/30caa190-8d65-4825-a681-4af09975ef3e-kube-api-access-6p8b6\") pod \"multus-additional-cni-plugins-84cwc\" (UID: \"30caa190-8d65-4825-a681-4af09975ef3e\") " pod="openshift-multus/multus-additional-cni-plugins-84cwc" Jul 07 18:46:13.384996 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.384926 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-host-run-netns\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.384996 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.384952 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.384996 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.384979 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-env-overrides\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.385505 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385019 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4f829b0a-c000-4612-8087-24e9f1f1018c-registration-dir\") pod \"aws-ebs-csi-driver-node-zrv7h\" (UID: \"4f829b0a-c000-4612-8087-24e9f1f1018c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" Jul 07 18:46:13.385505 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385044 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-run\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.385505 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385070 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgl6b\" (UniqueName: \"kubernetes.io/projected/c7296533-de2d-44a3-8972-5e1b8baeaae2-kube-api-access-sgl6b\") pod \"network-check-target-l5d2z\" (UID: \"c7296533-de2d-44a3-8972-5e1b8baeaae2\") " pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:46:13.385505 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385095 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c1e5d07f-f71c-4423-9ef0-882c19222314-cni-binary-copy\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.385505 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385120 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-etc-kubernetes\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.385505 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385126 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-etc-systemd\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.385505 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385178 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/329b4a4f-5489-4021-ab98-bcc7cc5b7dd0-metrics-certs\") pod \"network-metrics-daemon-cpp9l\" (UID: \"329b4a4f-5489-4021-ab98-bcc7cc5b7dd0\") " pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:13.385505 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385206 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-systemd-units\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.385505 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385230 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-ovnkube-script-lib\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.385505 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385280 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d65ba38-b541-4fe1-8933-4f6f8d366f4d-host-slash\") pod \"iptables-alerter-2w8tr\" (UID: \"5d65ba38-b541-4fe1-8933-4f6f8d366f4d\") " pod="openshift-network-operator/iptables-alerter-2w8tr" Jul 07 18:46:13.385505 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385360 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d65ba38-b541-4fe1-8933-4f6f8d366f4d-host-slash\") pod \"iptables-alerter-2w8tr\" (UID: \"5d65ba38-b541-4fe1-8933-4f6f8d366f4d\") " pod="openshift-network-operator/iptables-alerter-2w8tr" Jul 07 18:46:13.385505 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385359 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-run-openvswitch\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.385505 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385388 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-host-kubelet\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.385505 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385415 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30caa190-8d65-4825-a681-4af09975ef3e-system-cni-dir\") pod \"multus-additional-cni-plugins-84cwc\" (UID: \"30caa190-8d65-4825-a681-4af09975ef3e\") " pod="openshift-multus/multus-additional-cni-plugins-84cwc" Jul 07 18:46:13.385505 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385425 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-host-cni-netd\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.385505 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385456 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5d65ba38-b541-4fe1-8933-4f6f8d366f4d-iptables-alerter-script\") pod \"iptables-alerter-2w8tr\" (UID: \"5d65ba38-b541-4fe1-8933-4f6f8d366f4d\") " pod="openshift-network-operator/iptables-alerter-2w8tr" Jul 07 18:46:13.386463 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385536 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-etc-modprobe-d\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.386463 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385534 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-run\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.386463 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385597 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-etc-tuned\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.386463 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385634 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c1e5d07f-f71c-4423-9ef0-882c19222314-multus-daemon-config\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.386463 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385663 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xrcr\" (UniqueName: \"kubernetes.io/projected/5d65ba38-b541-4fe1-8933-4f6f8d366f4d-kube-api-access-4xrcr\") pod \"iptables-alerter-2w8tr\" (UID: \"5d65ba38-b541-4fe1-8933-4f6f8d366f4d\") " pod="openshift-network-operator/iptables-alerter-2w8tr" Jul 07 18:46:13.386463 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385688 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-tmp\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.386463 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385714 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-host-run-k8s-cni-cncf-io\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.386463 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385745 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg6jg\" (UniqueName: \"kubernetes.io/projected/c1e5d07f-f71c-4423-9ef0-882c19222314-kube-api-access-lg6jg\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.386463 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385799 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/40b947e9-e3e6-43ec-8b88-ff421f627227-host\") pod \"node-ca-rwjnv\" (UID: \"40b947e9-e3e6-43ec-8b88-ff421f627227\") " pod="openshift-image-registry/node-ca-rwjnv" Jul 07 18:46:13.386463 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385817 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4f829b0a-c000-4612-8087-24e9f1f1018c-etc-selinux\") pod \"aws-ebs-csi-driver-node-zrv7h\" (UID: \"4f829b0a-c000-4612-8087-24e9f1f1018c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" Jul 07 18:46:13.386463 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385826 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-log-socket\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.386463 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385879 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-log-socket\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.386463 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385889 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-host-kubelet\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.386463 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385388 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30caa190-8d65-4825-a681-4af09975ef3e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-84cwc\" (UID: \"30caa190-8d65-4825-a681-4af09975ef3e\") " pod="openshift-multus/multus-additional-cni-plugins-84cwc" Jul 07 18:46:13.386463 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385939 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-host-cni-netd\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.386463 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385939 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7cc80f14-06d5-483d-b15b-a44896b686a6-kubelet-config\") pod \"global-pull-secret-syncer-c6dfn\" (UID: \"7cc80f14-06d5-483d-b15b-a44896b686a6\") " pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:13.386463 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.385972 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-host-run-netns\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.387213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.386017 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.387213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.386512 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c1e5d07f-f71c-4423-9ef0-882c19222314-cni-binary-copy\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.387213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.386588 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-env-overrides\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.387213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.386597 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5d65ba38-b541-4fe1-8933-4f6f8d366f4d-iptables-alerter-script\") pod \"iptables-alerter-2w8tr\" (UID: \"5d65ba38-b541-4fe1-8933-4f6f8d366f4d\") " pod="openshift-network-operator/iptables-alerter-2w8tr" Jul 07 18:46:13.387213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.386601 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-etc-kubernetes\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.387213 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:13.386699 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:46:13.387213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.386738 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-etc-modprobe-d\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.387213 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:13.386764 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/329b4a4f-5489-4021-ab98-bcc7cc5b7dd0-metrics-certs podName:329b4a4f-5489-4021-ab98-bcc7cc5b7dd0 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:13.886742726 +0000 UTC m=+2.987761286 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/329b4a4f-5489-4021-ab98-bcc7cc5b7dd0-metrics-certs") pod "network-metrics-daemon-cpp9l" (UID: "329b4a4f-5489-4021-ab98-bcc7cc5b7dd0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:46:13.387213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.386813 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4f829b0a-c000-4612-8087-24e9f1f1018c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zrv7h\" (UID: \"4f829b0a-c000-4612-8087-24e9f1f1018c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" Jul 07 18:46:13.387213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.386861 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-systemd-units\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.387213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.387045 2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 18:46:13.387213 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.387149 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-host-run-k8s-cni-cncf-io\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.387872 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.387373 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-ovnkube-script-lib\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.387872 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.387420 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgm4p\" (UniqueName: \"kubernetes.io/projected/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-kube-api-access-fgm4p\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.387872 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.387451 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95flf\" (UniqueName: \"kubernetes.io/projected/8b9fb9da-073a-44c8-bf4d-2ee70fa748b9-kube-api-access-95flf\") pod \"node-resolver-ztg6h\" (UID: \"8b9fb9da-073a-44c8-bf4d-2ee70fa748b9\") " pod="openshift-dns/node-resolver-ztg6h" Jul 07 18:46:13.387872 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.387477 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-etc-kubernetes\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.387872 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.387503 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-etc-sysctl-d\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.387872 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.387517 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4f829b0a-c000-4612-8087-24e9f1f1018c-registration-dir\") pod \"aws-ebs-csi-driver-node-zrv7h\" (UID: \"4f829b0a-c000-4612-8087-24e9f1f1018c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" Jul 07 18:46:13.387872 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.387527 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30caa190-8d65-4825-a681-4af09975ef3e-cnibin\") pod \"multus-additional-cni-plugins-84cwc\" (UID: \"30caa190-8d65-4825-a681-4af09975ef3e\") " pod="openshift-multus/multus-additional-cni-plugins-84cwc" Jul 07 18:46:13.387872 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.387566 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-run-systemd\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.387872 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.387594 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-run-ovn\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.387872 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.387598 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/40b947e9-e3e6-43ec-8b88-ff421f627227-host\") pod \"node-ca-rwjnv\" (UID: \"40b947e9-e3e6-43ec-8b88-ff421f627227\") " pod="openshift-image-registry/node-ca-rwjnv" Jul 07 18:46:13.387872 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.387621 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-ovnkube-config\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.387872 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.387663 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-cnibin\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.387872 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.387688 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-node-log\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.387872 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.387721 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4f829b0a-c000-4612-8087-24e9f1f1018c-socket-dir\") pod \"aws-ebs-csi-driver-node-zrv7h\" (UID: \"4f829b0a-c000-4612-8087-24e9f1f1018c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" Jul 07 18:46:13.387872 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.387747 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-sys\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.387872 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.387761 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c1e5d07f-f71c-4423-9ef0-882c19222314-multus-daemon-config\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.387872 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.387787 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-hostroot\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.387872 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.387822 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4m28\" (UniqueName: \"kubernetes.io/projected/329b4a4f-5489-4021-ab98-bcc7cc5b7dd0-kube-api-access-v4m28\") pod \"network-metrics-daemon-cpp9l\" (UID: \"329b4a4f-5489-4021-ab98-bcc7cc5b7dd0\") " pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:13.388930 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.387833 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-etc-kubernetes\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.388930 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.387849 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-var-lib-openvswitch\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.388930 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.387912 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-ovn-node-metrics-cert\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.388930 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.387928 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-etc-sysctl-d\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.388930 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.387983 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-etc-sysctl-conf\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.388930 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.387566 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30caa190-8d65-4825-a681-4af09975ef3e-cnibin\") pod \"multus-additional-cni-plugins-84cwc\" (UID: \"30caa190-8d65-4825-a681-4af09975ef3e\") " pod="openshift-multus/multus-additional-cni-plugins-84cwc" Jul 07 18:46:13.388930 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388014 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-lib-modules\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.388930 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388045 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-run-ovn\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.388930 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388063 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-host\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.388930 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388101 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-multus-conf-dir\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.388930 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388105 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-cnibin\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.388930 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388138 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e48b5a6a-4b69-4e2a-823c-656ce51dc9e8-agent-certs\") pod \"konnectivity-agent-8khjj\" (UID: \"e48b5a6a-4b69-4e2a-823c-656ce51dc9e8\") " pod="kube-system/konnectivity-agent-8khjj" Jul 07 18:46:13.388930 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388146 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-multus-conf-dir\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.388930 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388164 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30caa190-8d65-4825-a681-4af09975ef3e-cni-binary-copy\") pod \"multus-additional-cni-plugins-84cwc\" (UID: \"30caa190-8d65-4825-a681-4af09975ef3e\") " pod="openshift-multus/multus-additional-cni-plugins-84cwc" Jul 07 18:46:13.388930 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388173 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4f829b0a-c000-4612-8087-24e9f1f1018c-socket-dir\") pod \"aws-ebs-csi-driver-node-zrv7h\" (UID: \"4f829b0a-c000-4612-8087-24e9f1f1018c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" Jul 07 18:46:13.388930 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388192 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-host-cni-bin\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.388930 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388215 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nr5d\" (UniqueName: \"kubernetes.io/projected/4f829b0a-c000-4612-8087-24e9f1f1018c-kube-api-access-9nr5d\") pod \"aws-ebs-csi-driver-node-zrv7h\" (UID: \"4f829b0a-c000-4612-8087-24e9f1f1018c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" Jul 07 18:46:13.388930 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388239 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-host-var-lib-cni-multus\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.389816 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388261 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-etc-openvswitch\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.389816 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388282 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-multus-socket-dir-parent\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.389816 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388303 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-host-var-lib-cni-bin\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.389816 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388324 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-host-var-lib-kubelet\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.389816 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388360 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8b9fb9da-073a-44c8-bf4d-2ee70fa748b9-hosts-file\") pod \"node-resolver-ztg6h\" (UID: \"8b9fb9da-073a-44c8-bf4d-2ee70fa748b9\") " pod="openshift-dns/node-resolver-ztg6h" Jul 07 18:46:13.389816 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388382 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-host-run-ovn-kubernetes\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.389816 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388403 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4f829b0a-c000-4612-8087-24e9f1f1018c-device-dir\") pod \"aws-ebs-csi-driver-node-zrv7h\" (UID: \"4f829b0a-c000-4612-8087-24e9f1f1018c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" Jul 07 18:46:13.389816 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388429 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-multus-cni-dir\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.389816 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388455 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/30caa190-8d65-4825-a681-4af09975ef3e-os-release\") pod \"multus-additional-cni-plugins-84cwc\" (UID: \"30caa190-8d65-4825-a681-4af09975ef3e\") " pod="openshift-multus/multus-additional-cni-plugins-84cwc" Jul 07 18:46:13.389816 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388481 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/30caa190-8d65-4825-a681-4af09975ef3e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-84cwc\" (UID: \"30caa190-8d65-4825-a681-4af09975ef3e\") " pod="openshift-multus/multus-additional-cni-plugins-84cwc" Jul 07 18:46:13.389816 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388510 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-host-slash\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.389816 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388536 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcfn5\" (UniqueName: \"kubernetes.io/projected/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-kube-api-access-fcfn5\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.389816 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388560 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-host-run-netns\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.389816 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388588 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b9fb9da-073a-44c8-bf4d-2ee70fa748b9-tmp-dir\") pod \"node-resolver-ztg6h\" (UID: \"8b9fb9da-073a-44c8-bf4d-2ee70fa748b9\") " pod="openshift-dns/node-resolver-ztg6h" Jul 07 18:46:13.389816 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388612 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/40b947e9-e3e6-43ec-8b88-ff421f627227-serviceca\") pod \"node-ca-rwjnv\" (UID: \"40b947e9-e3e6-43ec-8b88-ff421f627227\") " pod="openshift-image-registry/node-ca-rwjnv" Jul 07 18:46:13.389816 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388640 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/30caa190-8d65-4825-a681-4af09975ef3e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-84cwc\" (UID: \"30caa190-8d65-4825-a681-4af09975ef3e\") " pod="openshift-multus/multus-additional-cni-plugins-84cwc" Jul 07 18:46:13.389816 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388665 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4f829b0a-c000-4612-8087-24e9f1f1018c-sys-fs\") pod \"aws-ebs-csi-driver-node-zrv7h\" (UID: \"4f829b0a-c000-4612-8087-24e9f1f1018c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" Jul 07 18:46:13.390714 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388691 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7cc80f14-06d5-483d-b15b-a44896b686a6-dbus\") pod \"global-pull-secret-syncer-c6dfn\" (UID: \"7cc80f14-06d5-483d-b15b-a44896b686a6\") " pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:13.390714 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388715 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cc80f14-06d5-483d-b15b-a44896b686a6-original-pull-secret\") pod \"global-pull-secret-syncer-c6dfn\" (UID: \"7cc80f14-06d5-483d-b15b-a44896b686a6\") " pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:13.390714 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388741 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-var-lib-kubelet\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.390714 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388762 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-os-release\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.390714 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388786 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-host-run-multus-certs\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.390714 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388796 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-ovnkube-config\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.390714 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388809 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lwxm\" (UniqueName: \"kubernetes.io/projected/40b947e9-e3e6-43ec-8b88-ff421f627227-kube-api-access-7lwxm\") pod \"node-ca-rwjnv\" (UID: \"40b947e9-e3e6-43ec-8b88-ff421f627227\") " pod="openshift-image-registry/node-ca-rwjnv" Jul 07 18:46:13.390714 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388832 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-etc-sysconfig\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.390714 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388855 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-system-cni-dir\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.390714 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388882 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e48b5a6a-4b69-4e2a-823c-656ce51dc9e8-konnectivity-ca\") pod \"konnectivity-agent-8khjj\" (UID: \"e48b5a6a-4b69-4e2a-823c-656ce51dc9e8\") " pod="kube-system/konnectivity-agent-8khjj" Jul 07 18:46:13.390714 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388886 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-sys\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.390714 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.388983 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-var-lib-openvswitch\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.390714 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.389035 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-node-log\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.390714 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.387986 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-run-systemd\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.390714 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.390340 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-lib-modules\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.390714 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.390453 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-etc-sysctl-conf\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.390714 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.390497 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-var-lib-kubelet\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.390714 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.390521 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-host-run-netns\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.391668 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.390817 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b9fb9da-073a-44c8-bf4d-2ee70fa748b9-tmp-dir\") pod \"node-resolver-ztg6h\" (UID: \"8b9fb9da-073a-44c8-bf4d-2ee70fa748b9\") " pod="openshift-dns/node-resolver-ztg6h" Jul 07 18:46:13.391668 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.391062 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-host-cni-bin\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.391668 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.391267 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/40b947e9-e3e6-43ec-8b88-ff421f627227-serviceca\") pod \"node-ca-rwjnv\" (UID: \"40b947e9-e3e6-43ec-8b88-ff421f627227\") " pod="openshift-image-registry/node-ca-rwjnv" Jul 07 18:46:13.391668 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.391313 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-host-var-lib-cni-multus\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.391668 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.391330 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-host-run-ovn-kubernetes\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.391668 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.391385 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4f829b0a-c000-4612-8087-24e9f1f1018c-device-dir\") pod \"aws-ebs-csi-driver-node-zrv7h\" (UID: \"4f829b0a-c000-4612-8087-24e9f1f1018c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" Jul 07 18:46:13.391668 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.391397 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-etc-openvswitch\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.391668 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.391448 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-multus-socket-dir-parent\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.391668 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.391483 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-host-var-lib-cni-bin\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.391668 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.391492 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-multus-cni-dir\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.391668 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.391516 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-host-var-lib-kubelet\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.391668 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.391550 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8b9fb9da-073a-44c8-bf4d-2ee70fa748b9-hosts-file\") pod \"node-resolver-ztg6h\" (UID: \"8b9fb9da-073a-44c8-bf4d-2ee70fa748b9\") " pod="openshift-dns/node-resolver-ztg6h" Jul 07 18:46:13.391668 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.391554 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/30caa190-8d65-4825-a681-4af09975ef3e-os-release\") pod \"multus-additional-cni-plugins-84cwc\" (UID: \"30caa190-8d65-4825-a681-4af09975ef3e\") " pod="openshift-multus/multus-additional-cni-plugins-84cwc" Jul 07 18:46:13.391668 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.391597 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-host-slash\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.391668 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.391649 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-os-release\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.392491 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.391696 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4f829b0a-c000-4612-8087-24e9f1f1018c-sys-fs\") pod \"aws-ebs-csi-driver-node-zrv7h\" (UID: \"4f829b0a-c000-4612-8087-24e9f1f1018c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" Jul 07 18:46:13.392491 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.391743 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7cc80f14-06d5-483d-b15b-a44896b686a6-dbus\") pod \"global-pull-secret-syncer-c6dfn\" (UID: \"7cc80f14-06d5-483d-b15b-a44896b686a6\") " pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:13.392491 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.391755 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/30caa190-8d65-4825-a681-4af09975ef3e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-84cwc\" (UID: \"30caa190-8d65-4825-a681-4af09975ef3e\") " pod="openshift-multus/multus-additional-cni-plugins-84cwc" Jul 07 18:46:13.392491 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:13.391812 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:46:13.392491 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.391815 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-host-run-multus-certs\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.392491 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:13.391891 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc80f14-06d5-483d-b15b-a44896b686a6-original-pull-secret podName:7cc80f14-06d5-483d-b15b-a44896b686a6 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:13.891866624 +0000 UTC m=+2.992885185 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7cc80f14-06d5-483d-b15b-a44896b686a6-original-pull-secret") pod "global-pull-secret-syncer-c6dfn" (UID: "7cc80f14-06d5-483d-b15b-a44896b686a6") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:46:13.392491 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.391944 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-host\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.392491 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.391986 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-etc-sysconfig\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.392491 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.392029 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/30caa190-8d65-4825-a681-4af09975ef3e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-84cwc\" (UID: \"30caa190-8d65-4825-a681-4af09975ef3e\") " pod="openshift-multus/multus-additional-cni-plugins-84cwc" Jul 07 18:46:13.392491 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.392222 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-etc-tuned\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.392491 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.392274 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-hostroot\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.392491 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:13.392226 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:46:13.392491 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.392297 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1e5d07f-f71c-4423-9ef0-882c19222314-system-cni-dir\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.392491 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:13.392312 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:46:13.392491 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:13.392344 2583 projected.go:196] Error preparing data for projected volume kube-api-access-sgl6b for pod openshift-network-diagnostics/network-check-target-l5d2z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:46:13.393272 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.392714 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-ovn-node-metrics-cert\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.393272 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:13.392719 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7296533-de2d-44a3-8972-5e1b8baeaae2-kube-api-access-sgl6b podName:c7296533-de2d-44a3-8972-5e1b8baeaae2 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:13.892685813 +0000 UTC m=+2.993704389 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sgl6b" (UniqueName: "kubernetes.io/projected/c7296533-de2d-44a3-8972-5e1b8baeaae2-kube-api-access-sgl6b") pod "network-check-target-l5d2z" (UID: "c7296533-de2d-44a3-8972-5e1b8baeaae2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:46:13.393465 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.393359 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e48b5a6a-4b69-4e2a-823c-656ce51dc9e8-konnectivity-ca\") pod \"konnectivity-agent-8khjj\" (UID: \"e48b5a6a-4b69-4e2a-823c-656ce51dc9e8\") " pod="kube-system/konnectivity-agent-8khjj" Jul 07 18:46:13.393568 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.393508 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30caa190-8d65-4825-a681-4af09975ef3e-cni-binary-copy\") pod \"multus-additional-cni-plugins-84cwc\" (UID: \"30caa190-8d65-4825-a681-4af09975ef3e\") " pod="openshift-multus/multus-additional-cni-plugins-84cwc" Jul 07 18:46:13.395548 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.395504 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-tmp\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.396870 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.396819 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e48b5a6a-4b69-4e2a-823c-656ce51dc9e8-agent-certs\") pod \"konnectivity-agent-8khjj\" (UID: \"e48b5a6a-4b69-4e2a-823c-656ce51dc9e8\") " pod="kube-system/konnectivity-agent-8khjj" Jul 07 18:46:13.396989 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.396862 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xrcr\" (UniqueName: \"kubernetes.io/projected/5d65ba38-b541-4fe1-8933-4f6f8d366f4d-kube-api-access-4xrcr\") pod \"iptables-alerter-2w8tr\" (UID: \"5d65ba38-b541-4fe1-8933-4f6f8d366f4d\") " pod="openshift-network-operator/iptables-alerter-2w8tr" Jul 07 18:46:13.397474 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.397450 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p8b6\" (UniqueName: \"kubernetes.io/projected/30caa190-8d65-4825-a681-4af09975ef3e-kube-api-access-6p8b6\") pod \"multus-additional-cni-plugins-84cwc\" (UID: \"30caa190-8d65-4825-a681-4af09975ef3e\") " pod="openshift-multus/multus-additional-cni-plugins-84cwc" Jul 07 18:46:13.402971 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.402933 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nr5d\" (UniqueName: \"kubernetes.io/projected/4f829b0a-c000-4612-8087-24e9f1f1018c-kube-api-access-9nr5d\") pod \"aws-ebs-csi-driver-node-zrv7h\" (UID: \"4f829b0a-c000-4612-8087-24e9f1f1018c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" Jul 07 18:46:13.403105 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.402935 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcfn5\" (UniqueName: \"kubernetes.io/projected/bff07fe8-5476-4202-bd92-39b4a2ab6d8c-kube-api-access-fcfn5\") pod \"ovnkube-node-9j7xz\" (UID: \"bff07fe8-5476-4202-bd92-39b4a2ab6d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.403607 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.403438 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgm4p\" (UniqueName: \"kubernetes.io/projected/f379a0f4-5022-4bd6-ad64-ba9c7c7a335d-kube-api-access-fgm4p\") pod \"tuned-xlzmh\" (UID: \"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d\") " pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.403607 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.403469 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95flf\" (UniqueName: \"kubernetes.io/projected/8b9fb9da-073a-44c8-bf4d-2ee70fa748b9-kube-api-access-95flf\") pod \"node-resolver-ztg6h\" (UID: \"8b9fb9da-073a-44c8-bf4d-2ee70fa748b9\") " pod="openshift-dns/node-resolver-ztg6h" Jul 07 18:46:13.405253 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.403922 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg6jg\" (UniqueName: \"kubernetes.io/projected/c1e5d07f-f71c-4423-9ef0-882c19222314-kube-api-access-lg6jg\") pod \"multus-f9lks\" (UID: \"c1e5d07f-f71c-4423-9ef0-882c19222314\") " pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.407203 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.405658 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lwxm\" (UniqueName: \"kubernetes.io/projected/40b947e9-e3e6-43ec-8b88-ff421f627227-kube-api-access-7lwxm\") pod \"node-ca-rwjnv\" (UID: \"40b947e9-e3e6-43ec-8b88-ff421f627227\") " pod="openshift-image-registry/node-ca-rwjnv" Jul 07 18:46:13.407203 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.406766 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4m28\" (UniqueName: \"kubernetes.io/projected/329b4a4f-5489-4021-ab98-bcc7cc5b7dd0-kube-api-access-v4m28\") pod \"network-metrics-daemon-cpp9l\" (UID: \"329b4a4f-5489-4021-ab98-bcc7cc5b7dd0\") " pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:13.422964 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.422883 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-120.ec2.internal" event={"ID":"fe8c242d7468032595b33cd6dcda8cb2","Type":"ContainerStarted","Data":"1010e04fc4e348a17ce7e7341cac1d98647a885da4a5169eef6010efe5bb7dda"} Jul 07 18:46:13.424936 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.424535 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-120.ec2.internal" event={"ID":"ee9c1d39d9fd4f723b1434f6d29f4c61","Type":"ContainerStarted","Data":"2bd2821bcb6969da59fbeb65543e83305cba706578b5a286871c39ce3acb9ada"} Jul 07 18:46:13.517141 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.516985 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ztg6h" Jul 07 18:46:13.525520 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.525487 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rwjnv" Jul 07 18:46:13.535175 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:13.535145 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b9fb9da_073a_44c8_bf4d_2ee70fa748b9.slice/crio-b9aa4ca06f988c8905bf120d302921158f22a476eeb924e70bd0b462fe5ba14e WatchSource:0}: Error finding container b9aa4ca06f988c8905bf120d302921158f22a476eeb924e70bd0b462fe5ba14e: Status 404 returned error can't find the container with id b9aa4ca06f988c8905bf120d302921158f22a476eeb924e70bd0b462fe5ba14e Jul 07 18:46:13.536414 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.536392 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2w8tr" Jul 07 18:46:13.537665 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:13.537636 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40b947e9_e3e6_43ec_8b88_ff421f627227.slice/crio-8256a91c4bfb910e2bd3f684f4de2f7d7884661e7165238a4ba7037d1b815981 WatchSource:0}: Error finding container 8256a91c4bfb910e2bd3f684f4de2f7d7884661e7165238a4ba7037d1b815981: Status 404 returned error can't find the container with id 8256a91c4bfb910e2bd3f684f4de2f7d7884661e7165238a4ba7037d1b815981 Jul 07 18:46:13.544073 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.544049 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8khjj" Jul 07 18:46:13.553432 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.552573 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-84cwc" Jul 07 18:46:13.561073 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:13.561002 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode48b5a6a_4b69_4e2a_823c_656ce51dc9e8.slice/crio-f6b8999e726e275406af779ab339d509f402912cf4016be7954567bbf4dd515b WatchSource:0}: Error finding container f6b8999e726e275406af779ab339d509f402912cf4016be7954567bbf4dd515b: Status 404 returned error can't find the container with id f6b8999e726e275406af779ab339d509f402912cf4016be7954567bbf4dd515b Jul 07 18:46:13.562092 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.562068 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:13.568102 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:13.568076 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30caa190_8d65_4825_a681_4af09975ef3e.slice/crio-c51257d93f40033bd24cd34c90f29a61b926f860eb6d5f6ce3b8b0038c3e6b60 WatchSource:0}: Error finding container c51257d93f40033bd24cd34c90f29a61b926f860eb6d5f6ce3b8b0038c3e6b60: Status 404 returned error can't find the container with id c51257d93f40033bd24cd34c90f29a61b926f860eb6d5f6ce3b8b0038c3e6b60 Jul 07 18:46:13.572196 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.571930 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" Jul 07 18:46:13.574388 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:13.574351 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbff07fe8_5476_4202_bd92_39b4a2ab6d8c.slice/crio-f45609a96f7697ab639edc94cdcf3b1bd71f3aeebb70c7384bcee57f19dedc95 WatchSource:0}: Error finding container f45609a96f7697ab639edc94cdcf3b1bd71f3aeebb70c7384bcee57f19dedc95: Status 404 returned error can't find the container with id f45609a96f7697ab639edc94cdcf3b1bd71f3aeebb70c7384bcee57f19dedc95 Jul 07 18:46:13.580885 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.580861 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" Jul 07 18:46:13.586210 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:13.586156 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f829b0a_c000_4612_8087_24e9f1f1018c.slice/crio-20f04522b1614b3c949569bb66dc475c6b1136b1eadfea961b77567ced0d6d58 WatchSource:0}: Error finding container 20f04522b1614b3c949569bb66dc475c6b1136b1eadfea961b77567ced0d6d58: Status 404 returned error can't find the container with id 20f04522b1614b3c949569bb66dc475c6b1136b1eadfea961b77567ced0d6d58 Jul 07 18:46:13.590471 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.590239 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f9lks" Jul 07 18:46:13.594260 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:13.594232 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf379a0f4_5022_4bd6_ad64_ba9c7c7a335d.slice/crio-2592b6b6e9c2a809c4231a053e95117df07488259dabd507683e56bfc45f7748 WatchSource:0}: Error finding container 2592b6b6e9c2a809c4231a053e95117df07488259dabd507683e56bfc45f7748: Status 404 returned error can't find the container with id 2592b6b6e9c2a809c4231a053e95117df07488259dabd507683e56bfc45f7748 Jul 07 18:46:13.602794 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:13.602768 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1e5d07f_f71c_4423_9ef0_882c19222314.slice/crio-d4b182851961bbf95d2f30244220b258c133384c8123f97ae3101265db18cb97 WatchSource:0}: Error finding container d4b182851961bbf95d2f30244220b258c133384c8123f97ae3101265db18cb97: Status 404 returned error can't find the container with id d4b182851961bbf95d2f30244220b258c133384c8123f97ae3101265db18cb97 Jul 07 18:46:13.893229 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.893123 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cc80f14-06d5-483d-b15b-a44896b686a6-original-pull-secret\") pod \"global-pull-secret-syncer-c6dfn\" (UID: \"7cc80f14-06d5-483d-b15b-a44896b686a6\") " pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:13.893229 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.893187 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgl6b\" (UniqueName: \"kubernetes.io/projected/c7296533-de2d-44a3-8972-5e1b8baeaae2-kube-api-access-sgl6b\") pod \"network-check-target-l5d2z\" (UID: \"c7296533-de2d-44a3-8972-5e1b8baeaae2\") " pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:46:13.893229 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:13.893218 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/329b4a4f-5489-4021-ab98-bcc7cc5b7dd0-metrics-certs\") pod \"network-metrics-daemon-cpp9l\" (UID: \"329b4a4f-5489-4021-ab98-bcc7cc5b7dd0\") " pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:13.893536 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:13.893293 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:46:13.893536 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:13.893362 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:46:13.893536 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:13.893374 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc80f14-06d5-483d-b15b-a44896b686a6-original-pull-secret podName:7cc80f14-06d5-483d-b15b-a44896b686a6 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:14.893353807 +0000 UTC m=+3.994372367 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7cc80f14-06d5-483d-b15b-a44896b686a6-original-pull-secret") pod "global-pull-secret-syncer-c6dfn" (UID: "7cc80f14-06d5-483d-b15b-a44896b686a6") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:46:13.893536 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:13.893373 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:46:13.893536 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:13.893404 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:46:13.893536 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:13.893411 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/329b4a4f-5489-4021-ab98-bcc7cc5b7dd0-metrics-certs podName:329b4a4f-5489-4021-ab98-bcc7cc5b7dd0 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:14.893394818 +0000 UTC m=+3.994413373 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/329b4a4f-5489-4021-ab98-bcc7cc5b7dd0-metrics-certs") pod "network-metrics-daemon-cpp9l" (UID: "329b4a4f-5489-4021-ab98-bcc7cc5b7dd0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:46:13.893536 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:13.893417 2583 projected.go:196] Error preparing data for projected volume kube-api-access-sgl6b for pod openshift-network-diagnostics/network-check-target-l5d2z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:46:13.893536 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:13.893460 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7296533-de2d-44a3-8972-5e1b8baeaae2-kube-api-access-sgl6b podName:c7296533-de2d-44a3-8972-5e1b8baeaae2 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:14.893448565 +0000 UTC m=+3.994467126 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-sgl6b" (UniqueName: "kubernetes.io/projected/c7296533-de2d-44a3-8972-5e1b8baeaae2-kube-api-access-sgl6b") pod "network-check-target-l5d2z" (UID: "c7296533-de2d-44a3-8972-5e1b8baeaae2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:46:14.318996 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:14.318825 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 18:41:13 +0000 UTC" deadline="2028-01-20 17:04:52.922531827 +0000 UTC" Jul 07 18:46:14.318996 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:14.318873 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13486h18m38.603664619s" Jul 07 18:46:14.429101 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:14.428990 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2w8tr" event={"ID":"5d65ba38-b541-4fe1-8933-4f6f8d366f4d","Type":"ContainerStarted","Data":"5e54b7a1689b106ad54d2a9ad82b6ae64c3a666491a1e70bd52999c3d291f97a"} Jul 07 18:46:14.431858 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:14.431817 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rwjnv" event={"ID":"40b947e9-e3e6-43ec-8b88-ff421f627227","Type":"ContainerStarted","Data":"8256a91c4bfb910e2bd3f684f4de2f7d7884661e7165238a4ba7037d1b815981"} Jul 07 18:46:14.433812 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:14.433722 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ztg6h" event={"ID":"8b9fb9da-073a-44c8-bf4d-2ee70fa748b9","Type":"ContainerStarted","Data":"b9aa4ca06f988c8905bf120d302921158f22a476eeb924e70bd0b462fe5ba14e"} Jul 07 18:46:14.436411 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:14.436316 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f9lks" event={"ID":"c1e5d07f-f71c-4423-9ef0-882c19222314","Type":"ContainerStarted","Data":"d4b182851961bbf95d2f30244220b258c133384c8123f97ae3101265db18cb97"} Jul 07 18:46:14.439024 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:14.438890 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" event={"ID":"bff07fe8-5476-4202-bd92-39b4a2ab6d8c","Type":"ContainerStarted","Data":"f45609a96f7697ab639edc94cdcf3b1bd71f3aeebb70c7384bcee57f19dedc95"} Jul 07 18:46:14.443646 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:14.443606 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-84cwc" event={"ID":"30caa190-8d65-4825-a681-4af09975ef3e","Type":"ContainerStarted","Data":"c51257d93f40033bd24cd34c90f29a61b926f860eb6d5f6ce3b8b0038c3e6b60"} Jul 07 18:46:14.446385 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:14.446351 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" event={"ID":"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d","Type":"ContainerStarted","Data":"2592b6b6e9c2a809c4231a053e95117df07488259dabd507683e56bfc45f7748"} Jul 07 18:46:14.448518 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:14.448482 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" event={"ID":"4f829b0a-c000-4612-8087-24e9f1f1018c","Type":"ContainerStarted","Data":"20f04522b1614b3c949569bb66dc475c6b1136b1eadfea961b77567ced0d6d58"} Jul 07 18:46:14.451080 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:14.451051 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8khjj" event={"ID":"e48b5a6a-4b69-4e2a-823c-656ce51dc9e8","Type":"ContainerStarted","Data":"f6b8999e726e275406af779ab339d509f402912cf4016be7954567bbf4dd515b"} Jul 07 18:46:14.901361 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:14.901318 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cc80f14-06d5-483d-b15b-a44896b686a6-original-pull-secret\") pod \"global-pull-secret-syncer-c6dfn\" (UID: \"7cc80f14-06d5-483d-b15b-a44896b686a6\") " pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:14.901562 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:14.901391 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgl6b\" (UniqueName: \"kubernetes.io/projected/c7296533-de2d-44a3-8972-5e1b8baeaae2-kube-api-access-sgl6b\") pod \"network-check-target-l5d2z\" (UID: \"c7296533-de2d-44a3-8972-5e1b8baeaae2\") " pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:46:14.901562 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:14.901420 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/329b4a4f-5489-4021-ab98-bcc7cc5b7dd0-metrics-certs\") pod \"network-metrics-daemon-cpp9l\" (UID: \"329b4a4f-5489-4021-ab98-bcc7cc5b7dd0\") " pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:14.901695 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:14.901583 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:46:14.901695 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:14.901651 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/329b4a4f-5489-4021-ab98-bcc7cc5b7dd0-metrics-certs podName:329b4a4f-5489-4021-ab98-bcc7cc5b7dd0 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:16.901631919 +0000 UTC m=+6.002650480 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/329b4a4f-5489-4021-ab98-bcc7cc5b7dd0-metrics-certs") pod "network-metrics-daemon-cpp9l" (UID: "329b4a4f-5489-4021-ab98-bcc7cc5b7dd0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:46:14.901819 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:14.901719 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:46:14.901819 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:14.901757 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc80f14-06d5-483d-b15b-a44896b686a6-original-pull-secret podName:7cc80f14-06d5-483d-b15b-a44896b686a6 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:16.901745323 +0000 UTC m=+6.002763882 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7cc80f14-06d5-483d-b15b-a44896b686a6-original-pull-secret") pod "global-pull-secret-syncer-c6dfn" (UID: "7cc80f14-06d5-483d-b15b-a44896b686a6") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:46:14.901929 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:14.901847 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:46:14.901929 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:14.901866 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:46:14.901929 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:14.901878 2583 projected.go:196] Error preparing data for projected volume kube-api-access-sgl6b for pod openshift-network-diagnostics/network-check-target-l5d2z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:46:14.901929 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:14.901914 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7296533-de2d-44a3-8972-5e1b8baeaae2-kube-api-access-sgl6b podName:c7296533-de2d-44a3-8972-5e1b8baeaae2 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:16.901902006 +0000 UTC m=+6.002920566 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-sgl6b" (UniqueName: "kubernetes.io/projected/c7296533-de2d-44a3-8972-5e1b8baeaae2-kube-api-access-sgl6b") pod "network-check-target-l5d2z" (UID: "c7296533-de2d-44a3-8972-5e1b8baeaae2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:46:15.319693 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:15.319590 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 18:41:13 +0000 UTC" deadline="2028-04-15 18:05:37.055016506 +0000 UTC" Jul 07 18:46:15.319693 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:15.319641 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15551h19m21.735381518s" Jul 07 18:46:15.410204 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:15.409406 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:15.410204 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:15.409593 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cpp9l" podUID="329b4a4f-5489-4021-ab98-bcc7cc5b7dd0" Jul 07 18:46:15.410204 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:15.410025 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:15.410204 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:15.410127 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6dfn" podUID="7cc80f14-06d5-483d-b15b-a44896b686a6" Jul 07 18:46:15.410877 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:15.410677 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:46:15.410877 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:15.410793 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5d2z" podUID="c7296533-de2d-44a3-8972-5e1b8baeaae2" Jul 07 18:46:16.921600 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:16.920515 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cc80f14-06d5-483d-b15b-a44896b686a6-original-pull-secret\") pod \"global-pull-secret-syncer-c6dfn\" (UID: \"7cc80f14-06d5-483d-b15b-a44896b686a6\") " pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:16.921600 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:16.920595 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgl6b\" (UniqueName: \"kubernetes.io/projected/c7296533-de2d-44a3-8972-5e1b8baeaae2-kube-api-access-sgl6b\") pod \"network-check-target-l5d2z\" (UID: \"c7296533-de2d-44a3-8972-5e1b8baeaae2\") " pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:46:16.921600 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:16.920632 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/329b4a4f-5489-4021-ab98-bcc7cc5b7dd0-metrics-certs\") pod \"network-metrics-daemon-cpp9l\" (UID: \"329b4a4f-5489-4021-ab98-bcc7cc5b7dd0\") " pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:16.921600 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:16.920804 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:46:16.921600 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:16.920872 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/329b4a4f-5489-4021-ab98-bcc7cc5b7dd0-metrics-certs podName:329b4a4f-5489-4021-ab98-bcc7cc5b7dd0 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:20.920851428 +0000 UTC m=+10.021869986 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/329b4a4f-5489-4021-ab98-bcc7cc5b7dd0-metrics-certs") pod "network-metrics-daemon-cpp9l" (UID: "329b4a4f-5489-4021-ab98-bcc7cc5b7dd0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:46:16.921600 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:16.921352 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:46:16.921600 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:16.921408 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc80f14-06d5-483d-b15b-a44896b686a6-original-pull-secret podName:7cc80f14-06d5-483d-b15b-a44896b686a6 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:20.921389903 +0000 UTC m=+10.022408460 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7cc80f14-06d5-483d-b15b-a44896b686a6-original-pull-secret") pod "global-pull-secret-syncer-c6dfn" (UID: "7cc80f14-06d5-483d-b15b-a44896b686a6") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:46:16.921600 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:16.921495 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:46:16.921600 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:16.921512 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:46:16.921600 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:16.921525 2583 projected.go:196] Error preparing data for projected volume kube-api-access-sgl6b for pod openshift-network-diagnostics/network-check-target-l5d2z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:46:16.921600 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:16.921562 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7296533-de2d-44a3-8972-5e1b8baeaae2-kube-api-access-sgl6b podName:c7296533-de2d-44a3-8972-5e1b8baeaae2 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:20.921550093 +0000 UTC m=+10.022568648 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-sgl6b" (UniqueName: "kubernetes.io/projected/c7296533-de2d-44a3-8972-5e1b8baeaae2-kube-api-access-sgl6b") pod "network-check-target-l5d2z" (UID: "c7296533-de2d-44a3-8972-5e1b8baeaae2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:46:17.408815 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:17.408729 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:17.408987 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:17.408887 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cpp9l" podUID="329b4a4f-5489-4021-ab98-bcc7cc5b7dd0" Jul 07 18:46:17.409381 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:17.409357 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:17.409491 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:17.409465 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6dfn" podUID="7cc80f14-06d5-483d-b15b-a44896b686a6" Jul 07 18:46:17.409630 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:17.409613 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:46:17.409727 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:17.409706 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5d2z" podUID="c7296533-de2d-44a3-8972-5e1b8baeaae2" Jul 07 18:46:19.408768 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:19.408718 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:19.409226 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:19.408718 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:19.409226 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:19.408850 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6dfn" podUID="7cc80f14-06d5-483d-b15b-a44896b686a6" Jul 07 18:46:19.409226 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:19.408881 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:46:19.409226 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:19.409003 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cpp9l" podUID="329b4a4f-5489-4021-ab98-bcc7cc5b7dd0" Jul 07 18:46:19.409226 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:19.409089 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5d2z" podUID="c7296533-de2d-44a3-8972-5e1b8baeaae2" Jul 07 18:46:20.952184 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:20.952148 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgl6b\" (UniqueName: \"kubernetes.io/projected/c7296533-de2d-44a3-8972-5e1b8baeaae2-kube-api-access-sgl6b\") pod \"network-check-target-l5d2z\" (UID: \"c7296533-de2d-44a3-8972-5e1b8baeaae2\") " pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:46:20.952671 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:20.952199 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/329b4a4f-5489-4021-ab98-bcc7cc5b7dd0-metrics-certs\") pod \"network-metrics-daemon-cpp9l\" (UID: \"329b4a4f-5489-4021-ab98-bcc7cc5b7dd0\") " pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:20.952671 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:20.952270 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cc80f14-06d5-483d-b15b-a44896b686a6-original-pull-secret\") pod \"global-pull-secret-syncer-c6dfn\" (UID: \"7cc80f14-06d5-483d-b15b-a44896b686a6\") " pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:20.952671 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:20.952359 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:46:20.952671 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:20.952373 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:46:20.952671 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:20.952387 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:46:20.952671 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:20.952401 2583 projected.go:196] Error preparing data for projected volume kube-api-access-sgl6b for pod openshift-network-diagnostics/network-check-target-l5d2z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:46:20.952671 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:20.952428 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc80f14-06d5-483d-b15b-a44896b686a6-original-pull-secret podName:7cc80f14-06d5-483d-b15b-a44896b686a6 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:28.952411938 +0000 UTC m=+18.053430492 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7cc80f14-06d5-483d-b15b-a44896b686a6-original-pull-secret") pod "global-pull-secret-syncer-c6dfn" (UID: "7cc80f14-06d5-483d-b15b-a44896b686a6") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:46:20.952671 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:20.952345 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:46:20.952671 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:20.952454 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7296533-de2d-44a3-8972-5e1b8baeaae2-kube-api-access-sgl6b podName:c7296533-de2d-44a3-8972-5e1b8baeaae2 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:28.952436654 +0000 UTC m=+18.053455224 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-sgl6b" (UniqueName: "kubernetes.io/projected/c7296533-de2d-44a3-8972-5e1b8baeaae2-kube-api-access-sgl6b") pod "network-check-target-l5d2z" (UID: "c7296533-de2d-44a3-8972-5e1b8baeaae2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:46:20.952671 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:20.952528 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/329b4a4f-5489-4021-ab98-bcc7cc5b7dd0-metrics-certs podName:329b4a4f-5489-4021-ab98-bcc7cc5b7dd0 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:28.952510267 +0000 UTC m=+18.053528827 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/329b4a4f-5489-4021-ab98-bcc7cc5b7dd0-metrics-certs") pod "network-metrics-daemon-cpp9l" (UID: "329b4a4f-5489-4021-ab98-bcc7cc5b7dd0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:46:21.408951 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:21.408870 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:21.409111 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:21.408992 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6dfn" podUID="7cc80f14-06d5-483d-b15b-a44896b686a6" Jul 07 18:46:21.409183 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:21.409116 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:46:21.409247 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:21.409222 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5d2z" podUID="c7296533-de2d-44a3-8972-5e1b8baeaae2" Jul 07 18:46:21.410306 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:21.410281 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:21.410427 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:21.410382 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cpp9l" podUID="329b4a4f-5489-4021-ab98-bcc7cc5b7dd0" Jul 07 18:46:23.409064 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:23.409021 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:23.409545 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:23.409161 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cpp9l" podUID="329b4a4f-5489-4021-ab98-bcc7cc5b7dd0" Jul 07 18:46:23.409622 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:23.409604 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:23.409728 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:23.409707 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6dfn" podUID="7cc80f14-06d5-483d-b15b-a44896b686a6" Jul 07 18:46:23.409769 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:23.409760 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:46:23.409850 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:23.409831 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5d2z" podUID="c7296533-de2d-44a3-8972-5e1b8baeaae2" Jul 07 18:46:25.409300 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:25.409044 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:25.409724 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:25.409052 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:25.409724 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:25.409102 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:46:25.409724 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:25.409418 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cpp9l" podUID="329b4a4f-5489-4021-ab98-bcc7cc5b7dd0" Jul 07 18:46:25.409724 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:25.409523 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5d2z" podUID="c7296533-de2d-44a3-8972-5e1b8baeaae2" Jul 07 18:46:25.409724 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:25.409619 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6dfn" podUID="7cc80f14-06d5-483d-b15b-a44896b686a6" Jul 07 18:46:26.479941 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:26.479903 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rwjnv" event={"ID":"40b947e9-e3e6-43ec-8b88-ff421f627227","Type":"ContainerStarted","Data":"09a7b149e84cbc7d48652a9ce60727e312c02ee9b460c73281a919241fae98c2"} Jul 07 18:46:26.481569 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:26.481539 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-120.ec2.internal" event={"ID":"fe8c242d7468032595b33cd6dcda8cb2","Type":"ContainerStarted","Data":"fba85605bc4d1258d33b543dc7ccbd3cf70f0596384786fbb1fc5212ccbee7b2"} Jul 07 18:46:26.495892 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:26.495842 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rwjnv" podStartSLOduration=4.610540967 podStartE2EDuration="15.495816329s" totalImagesPullingTime="10.885275362s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:46:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:26.4953362 +0000 UTC m=+15.596354775" watchObservedRunningTime="2026-07-07 18:46:26.495816329 +0000 UTC m=+15.596834917" Jul 07 18:46:27.408744 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:27.408709 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:27.408909 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:27.408709 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:46:27.408909 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:27.408838 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6dfn" podUID="7cc80f14-06d5-483d-b15b-a44896b686a6" Jul 07 18:46:27.408909 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:27.408715 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:27.409078 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:27.408899 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5d2z" podUID="c7296533-de2d-44a3-8972-5e1b8baeaae2" Jul 07 18:46:27.409078 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:27.409022 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cpp9l" podUID="329b4a4f-5489-4021-ab98-bcc7cc5b7dd0" Jul 07 18:46:29.015633 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:29.015594 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cc80f14-06d5-483d-b15b-a44896b686a6-original-pull-secret\") pod \"global-pull-secret-syncer-c6dfn\" (UID: \"7cc80f14-06d5-483d-b15b-a44896b686a6\") " pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:29.016194 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:29.015643 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgl6b\" (UniqueName: \"kubernetes.io/projected/c7296533-de2d-44a3-8972-5e1b8baeaae2-kube-api-access-sgl6b\") pod \"network-check-target-l5d2z\" (UID: \"c7296533-de2d-44a3-8972-5e1b8baeaae2\") " pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:46:29.016194 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:29.015670 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/329b4a4f-5489-4021-ab98-bcc7cc5b7dd0-metrics-certs\") pod \"network-metrics-daemon-cpp9l\" (UID: \"329b4a4f-5489-4021-ab98-bcc7cc5b7dd0\") " pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:29.016194 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:29.015754 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:46:29.016194 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:29.015812 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/329b4a4f-5489-4021-ab98-bcc7cc5b7dd0-metrics-certs podName:329b4a4f-5489-4021-ab98-bcc7cc5b7dd0 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:45.015797726 +0000 UTC m=+34.116816279 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/329b4a4f-5489-4021-ab98-bcc7cc5b7dd0-metrics-certs") pod "network-metrics-daemon-cpp9l" (UID: "329b4a4f-5489-4021-ab98-bcc7cc5b7dd0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:46:29.016194 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:29.015837 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:46:29.016194 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:29.015867 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:46:29.016194 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:29.015883 2583 projected.go:196] Error preparing data for projected volume kube-api-access-sgl6b for pod openshift-network-diagnostics/network-check-target-l5d2z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:46:29.016194 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:29.015754 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:46:29.016194 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:29.015930 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7296533-de2d-44a3-8972-5e1b8baeaae2-kube-api-access-sgl6b podName:c7296533-de2d-44a3-8972-5e1b8baeaae2 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:45.015917611 +0000 UTC m=+34.116936183 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-sgl6b" (UniqueName: "kubernetes.io/projected/c7296533-de2d-44a3-8972-5e1b8baeaae2-kube-api-access-sgl6b") pod "network-check-target-l5d2z" (UID: "c7296533-de2d-44a3-8972-5e1b8baeaae2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:46:29.016194 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:29.015973 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc80f14-06d5-483d-b15b-a44896b686a6-original-pull-secret podName:7cc80f14-06d5-483d-b15b-a44896b686a6 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:45.015953527 +0000 UTC m=+34.116972084 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7cc80f14-06d5-483d-b15b-a44896b686a6-original-pull-secret") pod "global-pull-secret-syncer-c6dfn" (UID: "7cc80f14-06d5-483d-b15b-a44896b686a6") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:46:29.409286 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:29.409183 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:46:29.409286 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:29.409185 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:29.409506 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:29.409316 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:29.409506 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:29.409329 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5d2z" podUID="c7296533-de2d-44a3-8972-5e1b8baeaae2" Jul 07 18:46:29.409506 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:29.409417 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cpp9l" podUID="329b4a4f-5489-4021-ab98-bcc7cc5b7dd0" Jul 07 18:46:29.409506 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:29.409484 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6dfn" podUID="7cc80f14-06d5-483d-b15b-a44896b686a6" Jul 07 18:46:31.409827 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:31.409768 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:31.410312 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:31.409874 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:46:31.410312 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:31.409890 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6dfn" podUID="7cc80f14-06d5-483d-b15b-a44896b686a6" Jul 07 18:46:31.410312 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:31.409947 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5d2z" podUID="c7296533-de2d-44a3-8972-5e1b8baeaae2" Jul 07 18:46:31.410312 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:31.409983 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:31.410312 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:31.410099 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cpp9l" podUID="329b4a4f-5489-4021-ab98-bcc7cc5b7dd0" Jul 07 18:46:33.408775 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:33.408626 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:33.409507 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:33.408626 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:33.409507 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:33.408870 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cpp9l" podUID="329b4a4f-5489-4021-ab98-bcc7cc5b7dd0" Jul 07 18:46:33.409507 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:33.408643 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:46:33.409507 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:33.408936 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6dfn" podUID="7cc80f14-06d5-483d-b15b-a44896b686a6" Jul 07 18:46:33.409507 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:33.408989 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5d2z" podUID="c7296533-de2d-44a3-8972-5e1b8baeaae2" Jul 07 18:46:33.496605 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:33.496574 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f9lks" event={"ID":"c1e5d07f-f71c-4423-9ef0-882c19222314","Type":"ContainerStarted","Data":"248406e902f3d527f33374d3a155aa8d319b096e3b86dcf8fc623d8d919d6d1d"} Jul 07 18:46:33.499027 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:33.499007 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9j7xz_bff07fe8-5476-4202-bd92-39b4a2ab6d8c/ovn-acl-logging/0.log" Jul 07 18:46:33.499390 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:33.499363 2583 generic.go:356] "Generic (PLEG): container finished" podID="bff07fe8-5476-4202-bd92-39b4a2ab6d8c" containerID="864af1553a8af814794c74b5c50fdd5b744a55ebd2669dbb612323324b632189" exitCode=1 Jul 07 18:46:33.499507 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:33.499440 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" event={"ID":"bff07fe8-5476-4202-bd92-39b4a2ab6d8c","Type":"ContainerStarted","Data":"c7b26a9153f038281511f83cbd2c5df4c2ba00c486726d179f4b730c543d434f"} Jul 07 18:46:33.499567 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:33.499527 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" event={"ID":"bff07fe8-5476-4202-bd92-39b4a2ab6d8c","Type":"ContainerStarted","Data":"b8238672e9e1871f6c831eb0d5d922d0fffd51a12573a82f0c6c1daf15b34c02"} Jul 07 18:46:33.499624 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:33.499580 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" event={"ID":"bff07fe8-5476-4202-bd92-39b4a2ab6d8c","Type":"ContainerDied","Data":"864af1553a8af814794c74b5c50fdd5b744a55ebd2669dbb612323324b632189"} Jul 07 18:46:33.499674 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:33.499631 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" event={"ID":"bff07fe8-5476-4202-bd92-39b4a2ab6d8c","Type":"ContainerStarted","Data":"bd77a01185b71c050b736aa7f6fd43aba9d071dffc9a506568d0cac1047653cb"} Jul 07 18:46:33.505519 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:33.505494 2583 generic.go:356] "Generic (PLEG): container finished" podID="30caa190-8d65-4825-a681-4af09975ef3e" containerID="3b56cb492b6445095e5d31dbd908494a0054b972801fce9db5227df83986aab7" exitCode=0 Jul 07 18:46:33.505676 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:33.505545 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-84cwc" event={"ID":"30caa190-8d65-4825-a681-4af09975ef3e","Type":"ContainerDied","Data":"3b56cb492b6445095e5d31dbd908494a0054b972801fce9db5227df83986aab7"} Jul 07 18:46:33.507301 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:33.507251 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-120.ec2.internal" event={"ID":"ee9c1d39d9fd4f723b1434f6d29f4c61","Type":"ContainerStarted","Data":"9cc32b9d61eec38681368dfd9eaca19f2e85aa02b6a34666e9263d4418cb9098"} Jul 07 18:46:33.508808 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:33.508781 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" event={"ID":"f379a0f4-5022-4bd6-ad64-ba9c7c7a335d","Type":"ContainerStarted","Data":"4c613cc873e84cbe1ab72f1925cee15ada73aac5a1312e72d8ee0e87ec306e41"} Jul 07 18:46:33.510346 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:33.510316 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" event={"ID":"4f829b0a-c000-4612-8087-24e9f1f1018c","Type":"ContainerStarted","Data":"838b129e54ae0e0c64a6922efc0903c3d09e7c11e6508f1f8fcb7b5485b4f435"} Jul 07 18:46:33.511867 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:33.511844 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8khjj" event={"ID":"e48b5a6a-4b69-4e2a-823c-656ce51dc9e8","Type":"ContainerStarted","Data":"6467c7dae2f5908dd01acc7d426fc2908733040b7b6a20210fa81736b05ba13b"} Jul 07 18:46:33.513209 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:33.513180 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ztg6h" event={"ID":"8b9fb9da-073a-44c8-bf4d-2ee70fa748b9","Type":"ContainerStarted","Data":"ccbd9eeb393f854ad4b8e18dcca4c62dc6736b98af9987f262868581802d8bbf"} Jul 07 18:46:33.514769 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:33.514748 2583 generic.go:356] "Generic (PLEG): container finished" podID="fe8c242d7468032595b33cd6dcda8cb2" containerID="fba85605bc4d1258d33b543dc7ccbd3cf70f0596384786fbb1fc5212ccbee7b2" exitCode=0 Jul 07 18:46:33.514845 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:33.514781 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-120.ec2.internal" event={"ID":"fe8c242d7468032595b33cd6dcda8cb2","Type":"ContainerDied","Data":"fba85605bc4d1258d33b543dc7ccbd3cf70f0596384786fbb1fc5212ccbee7b2"} Jul 07 18:46:33.519963 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:33.519908 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-f9lks" podStartSLOduration=1.7695409770000001 podStartE2EDuration="21.519894509s" totalImagesPullingTime="19.750353532s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:46:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:33.519684959 +0000 UTC m=+22.620703543" watchObservedRunningTime="2026-07-07 18:46:33.519894509 +0000 UTC m=+22.620913085" Jul 07 18:46:33.537691 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:33.537634 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xlzmh" podStartSLOduration=2.109188799 podStartE2EDuration="21.53761103s" totalImagesPullingTime="19.428422231s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:46:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:33.537531866 +0000 UTC m=+22.638550441" watchObservedRunningTime="2026-07-07 18:46:33.53761103 +0000 UTC m=+22.638629606" Jul 07 18:46:33.614128 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:33.613781 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-120.ec2.internal" podStartSLOduration=21.61376097 podStartE2EDuration="21.61376097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:46:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:33.583915799 +0000 UTC m=+22.684934374" watchObservedRunningTime="2026-07-07 18:46:33.61376097 +0000 UTC m=+22.714779548" Jul 07 18:46:33.629620 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:33.629577 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ztg6h" podStartSLOduration=11.736942475 podStartE2EDuration="22.629560477s" totalImagesPullingTime="10.892618002s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:46:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:33.629282782 +0000 UTC m=+22.730301349" watchObservedRunningTime="2026-07-07 18:46:33.629560477 +0000 UTC m=+22.730579052" Jul 07 18:46:33.645157 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:33.645113 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8khjj" podStartSLOduration=10.779861923 podStartE2EDuration="21.645096693s" totalImagesPullingTime="10.86523477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:33.644839309 +0000 UTC m=+22.745857886" watchObservedRunningTime="2026-07-07 18:46:33.645096693 +0000 UTC m=+22.746115268" Jul 07 18:46:34.518039 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:34.517998 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2w8tr" event={"ID":"5d65ba38-b541-4fe1-8933-4f6f8d366f4d","Type":"ContainerStarted","Data":"0490d0535dbe2900bc40a5001b257dc906175006e0db83dcdb53a9073d3bc4e1"} Jul 07 18:46:34.519867 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:34.519842 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-120.ec2.internal" event={"ID":"fe8c242d7468032595b33cd6dcda8cb2","Type":"ContainerStarted","Data":"8594fbad00d50e64ab5964662a1c814fcc2ef8815d42aef3e859ddc306243c40"} Jul 07 18:46:34.522853 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:34.522834 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9j7xz_bff07fe8-5476-4202-bd92-39b4a2ab6d8c/ovn-acl-logging/0.log" Jul 07 18:46:34.523325 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:34.523299 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" event={"ID":"bff07fe8-5476-4202-bd92-39b4a2ab6d8c","Type":"ContainerStarted","Data":"a686ad3f09ef732e3dbc1aa23b60c2bad50ac7645c7d1989b4809b4dc329c7a5"} Jul 07 18:46:34.523421 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:34.523375 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" event={"ID":"bff07fe8-5476-4202-bd92-39b4a2ab6d8c","Type":"ContainerStarted","Data":"d143392b0cee34cb60cb4ceb7baa5bd40c7aaede06f26df734a554c0fb1c75e6"} Jul 07 18:46:34.533564 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:34.533515 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2w8tr" podStartSLOduration=11.655425334 podStartE2EDuration="22.533498627s" totalImagesPullingTime="10.878073293s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:46:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:34.533410519 +0000 UTC m=+23.634429094" watchObservedRunningTime="2026-07-07 18:46:34.533498627 +0000 UTC m=+23.634517205" Jul 07 18:46:34.549453 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:34.549398 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-120.ec2.internal" podStartSLOduration=22.549379838 podStartE2EDuration="22.549379838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:46:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:34.54890039 +0000 UTC m=+23.649918964" watchObservedRunningTime="2026-07-07 18:46:34.549379838 +0000 UTC m=+23.650398412" Jul 07 18:46:34.742487 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:34.742457 2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 18:46:35.169419 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:35.169323 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8khjj" Jul 07 18:46:35.170143 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:35.170118 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8khjj" Jul 07 18:46:35.347159 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:35.346975 2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T18:46:34.742480115Z","UUID":"d693a252-6e3d-49b2-b172-4f8f01b15cc8","Handler":null,"Name":"","Endpoint":""} Jul 07 18:46:35.348988 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:35.348968 2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 18:46:35.349104 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:35.349003 2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 18:46:35.408689 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:35.408659 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:35.408689 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:35.408678 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:35.408940 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:35.408661 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:46:35.408940 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:35.408798 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6dfn" podUID="7cc80f14-06d5-483d-b15b-a44896b686a6" Jul 07 18:46:35.408940 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:35.408923 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5d2z" podUID="c7296533-de2d-44a3-8972-5e1b8baeaae2" Jul 07 18:46:35.409105 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:35.409022 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cpp9l" podUID="329b4a4f-5489-4021-ab98-bcc7cc5b7dd0" Jul 07 18:46:35.527373 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:35.527303 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" event={"ID":"4f829b0a-c000-4612-8087-24e9f1f1018c","Type":"ContainerStarted","Data":"15d22ab3fae3fae145e161c86d14faa6ae65ab716343dd1c22679ffb3860749d"} Jul 07 18:46:36.533943 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:36.533741 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9j7xz_bff07fe8-5476-4202-bd92-39b4a2ab6d8c/ovn-acl-logging/0.log" Jul 07 18:46:36.534463 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:36.534354 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" event={"ID":"bff07fe8-5476-4202-bd92-39b4a2ab6d8c","Type":"ContainerStarted","Data":"e3c2a9be3453f21d5bdbb84b86471c8be68421102f7cc5c883af1862702f37d2"} Jul 07 18:46:36.538573 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:36.538544 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" event={"ID":"4f829b0a-c000-4612-8087-24e9f1f1018c","Type":"ContainerStarted","Data":"fb79538d57445a3e6483f815308af5fb92602f491c9884094ef00e98850de83b"} Jul 07 18:46:36.538573 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:36.538570 2583 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 07 18:46:36.556035 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:36.555973 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrv7h" podStartSLOduration=11.45835646 podStartE2EDuration="24.555955002s" totalImagesPullingTime="13.097598542s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:46:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:36.555631153 +0000 UTC m=+25.656649729" watchObservedRunningTime="2026-07-07 18:46:36.555955002 +0000 UTC m=+25.656973577" Jul 07 18:46:37.408653 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:37.408618 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:37.408860 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:37.408618 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:46:37.408860 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:37.408748 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6dfn" podUID="7cc80f14-06d5-483d-b15b-a44896b686a6" Jul 07 18:46:37.408860 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:37.408627 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:37.408860 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:37.408834 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5d2z" podUID="c7296533-de2d-44a3-8972-5e1b8baeaae2" Jul 07 18:46:37.409090 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:37.408928 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cpp9l" podUID="329b4a4f-5489-4021-ab98-bcc7cc5b7dd0" Jul 07 18:46:38.300365 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:38.300285 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8khjj" Jul 07 18:46:38.300833 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:38.300490 2583 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 07 18:46:38.300967 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:38.300952 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8khjj" Jul 07 18:46:38.547066 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:38.546907 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9j7xz_bff07fe8-5476-4202-bd92-39b4a2ab6d8c/ovn-acl-logging/0.log" Jul 07 18:46:38.547445 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:38.547419 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" event={"ID":"bff07fe8-5476-4202-bd92-39b4a2ab6d8c","Type":"ContainerStarted","Data":"1e531a16db66ac50744fb7cfe9bb16216b90b6445fb1cb6d9b3cd7e28ce08c67"} Jul 07 18:46:38.547905 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:38.547886 2583 scope.go:117] "RemoveContainer" containerID="864af1553a8af814794c74b5c50fdd5b744a55ebd2669dbb612323324b632189" Jul 07 18:46:39.408596 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:39.408564 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:46:39.409309 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:39.408570 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:39.409309 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:39.408687 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:39.409309 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:39.408689 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5d2z" podUID="c7296533-de2d-44a3-8972-5e1b8baeaae2" Jul 07 18:46:39.409309 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:39.408758 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6dfn" podUID="7cc80f14-06d5-483d-b15b-a44896b686a6" Jul 07 18:46:39.409309 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:39.408827 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cpp9l" podUID="329b4a4f-5489-4021-ab98-bcc7cc5b7dd0" Jul 07 18:46:39.552714 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:39.552643 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9j7xz_bff07fe8-5476-4202-bd92-39b4a2ab6d8c/ovn-acl-logging/0.log" Jul 07 18:46:39.553057 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:39.553037 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" event={"ID":"bff07fe8-5476-4202-bd92-39b4a2ab6d8c","Type":"ContainerStarted","Data":"d9be16bdca6774dae2848ec655511153127435579d9655cf5c9740b0b81dd27a"} Jul 07 18:46:39.553457 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:39.553434 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:39.553592 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:39.553499 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:39.553592 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:39.553536 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:39.555086 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:39.555057 2583 generic.go:356] "Generic (PLEG): container finished" podID="30caa190-8d65-4825-a681-4af09975ef3e" containerID="719e2b32ba3794e9ae548ca4fec239789df2c8feb73a1c215f663b40f1447c62" exitCode=0 Jul 07 18:46:39.555214 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:39.555117 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-84cwc" event={"ID":"30caa190-8d65-4825-a681-4af09975ef3e","Type":"ContainerDied","Data":"719e2b32ba3794e9ae548ca4fec239789df2c8feb73a1c215f663b40f1447c62"} Jul 07 18:46:39.571311 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:39.571281 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:39.571504 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:39.571432 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:46:39.590664 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:39.590612 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" podStartSLOduration=8.077474987 podStartE2EDuration="27.590591674s" totalImagesPullingTime="19.513116687s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:46:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:39.589540239 +0000 UTC m=+28.690558813" watchObservedRunningTime="2026-07-07 18:46:39.590591674 +0000 UTC m=+28.691610249" Jul 07 18:46:40.350816 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:40.350784 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l5d2z"] Jul 07 18:46:40.351026 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:40.350923 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:46:40.351026 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:40.351004 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5d2z" podUID="c7296533-de2d-44a3-8972-5e1b8baeaae2" Jul 07 18:46:40.353649 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:40.353615 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-c6dfn"] Jul 07 18:46:40.353769 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:40.353719 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:40.353849 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:40.353827 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6dfn" podUID="7cc80f14-06d5-483d-b15b-a44896b686a6" Jul 07 18:46:40.355527 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:40.355506 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cpp9l"] Jul 07 18:46:40.355611 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:40.355589 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:40.355674 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:40.355658 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cpp9l" podUID="329b4a4f-5489-4021-ab98-bcc7cc5b7dd0" Jul 07 18:46:41.561738 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:41.561706 2583 generic.go:356] "Generic (PLEG): container finished" podID="30caa190-8d65-4825-a681-4af09975ef3e" containerID="afd7c63dbf7a186f2d498b3739a6d23f0400502cd897ab2123397785989e4af7" exitCode=0 Jul 07 18:46:41.562204 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:41.561793 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-84cwc" event={"ID":"30caa190-8d65-4825-a681-4af09975ef3e","Type":"ContainerDied","Data":"afd7c63dbf7a186f2d498b3739a6d23f0400502cd897ab2123397785989e4af7"} Jul 07 18:46:42.408710 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:42.408482 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:46:42.408966 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:42.408538 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:42.408966 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:42.408743 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5d2z" podUID="c7296533-de2d-44a3-8972-5e1b8baeaae2" Jul 07 18:46:42.408966 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:42.408839 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cpp9l" podUID="329b4a4f-5489-4021-ab98-bcc7cc5b7dd0" Jul 07 18:46:42.408966 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:42.408546 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:42.409101 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:42.408972 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6dfn" podUID="7cc80f14-06d5-483d-b15b-a44896b686a6" Jul 07 18:46:43.568217 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:43.568182 2583 generic.go:356] "Generic (PLEG): container finished" podID="30caa190-8d65-4825-a681-4af09975ef3e" containerID="8521f5cb4d4bbe89a3cb29e7b93ef06b29b788248233f7d99506f4435437d64c" exitCode=0 Jul 07 18:46:43.568217 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:43.568217 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-84cwc" event={"ID":"30caa190-8d65-4825-a681-4af09975ef3e","Type":"ContainerDied","Data":"8521f5cb4d4bbe89a3cb29e7b93ef06b29b788248233f7d99506f4435437d64c"} Jul 07 18:46:44.408787 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:44.408752 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:46:44.408787 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:44.408780 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:44.409034 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:44.408780 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:44.409034 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:44.408881 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5d2z" podUID="c7296533-de2d-44a3-8972-5e1b8baeaae2" Jul 07 18:46:44.409034 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:44.408973 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cpp9l" podUID="329b4a4f-5489-4021-ab98-bcc7cc5b7dd0" Jul 07 18:46:44.409178 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:44.409070 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6dfn" podUID="7cc80f14-06d5-483d-b15b-a44896b686a6" Jul 07 18:46:45.029514 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:45.029479 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cc80f14-06d5-483d-b15b-a44896b686a6-original-pull-secret\") pod \"global-pull-secret-syncer-c6dfn\" (UID: \"7cc80f14-06d5-483d-b15b-a44896b686a6\") " pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:45.030036 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:45.029527 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgl6b\" (UniqueName: \"kubernetes.io/projected/c7296533-de2d-44a3-8972-5e1b8baeaae2-kube-api-access-sgl6b\") pod \"network-check-target-l5d2z\" (UID: \"c7296533-de2d-44a3-8972-5e1b8baeaae2\") " pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:46:45.030036 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:45.029558 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/329b4a4f-5489-4021-ab98-bcc7cc5b7dd0-metrics-certs\") pod \"network-metrics-daemon-cpp9l\" (UID: \"329b4a4f-5489-4021-ab98-bcc7cc5b7dd0\") " pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:45.030036 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:45.029636 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:46:45.030036 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:45.029660 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:46:45.030036 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:45.029690 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:46:45.030036 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:45.029707 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:46:45.030036 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:45.029713 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/329b4a4f-5489-4021-ab98-bcc7cc5b7dd0-metrics-certs podName:329b4a4f-5489-4021-ab98-bcc7cc5b7dd0 nodeName:}" failed. No retries permitted until 2026-07-07 18:47:17.029694217 +0000 UTC m=+66.130712785 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/329b4a4f-5489-4021-ab98-bcc7cc5b7dd0-metrics-certs") pod "network-metrics-daemon-cpp9l" (UID: "329b4a4f-5489-4021-ab98-bcc7cc5b7dd0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:46:45.030036 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:45.029720 2583 projected.go:196] Error preparing data for projected volume kube-api-access-sgl6b for pod openshift-network-diagnostics/network-check-target-l5d2z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:46:45.030036 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:45.029732 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc80f14-06d5-483d-b15b-a44896b686a6-original-pull-secret podName:7cc80f14-06d5-483d-b15b-a44896b686a6 nodeName:}" failed. No retries permitted until 2026-07-07 18:47:17.029722535 +0000 UTC m=+66.130741096 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7cc80f14-06d5-483d-b15b-a44896b686a6-original-pull-secret") pod "global-pull-secret-syncer-c6dfn" (UID: "7cc80f14-06d5-483d-b15b-a44896b686a6") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:46:45.030036 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:45.029760 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7296533-de2d-44a3-8972-5e1b8baeaae2-kube-api-access-sgl6b podName:c7296533-de2d-44a3-8972-5e1b8baeaae2 nodeName:}" failed. No retries permitted until 2026-07-07 18:47:17.029745038 +0000 UTC m=+66.130763599 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-sgl6b" (UniqueName: "kubernetes.io/projected/c7296533-de2d-44a3-8972-5e1b8baeaae2-kube-api-access-sgl6b") pod "network-check-target-l5d2z" (UID: "c7296533-de2d-44a3-8972-5e1b8baeaae2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:46:46.258621 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.258590 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-120.ec2.internal" event="NodeReady" Jul 07 18:46:46.259135 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.258762 2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 18:46:46.294414 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.294383 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-69d6b476d6-9lkmx"] Jul 07 18:46:46.299168 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.299138 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:46.301503 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.301456 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 07 18:46:46.301503 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.301493 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 07 18:46:46.302028 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.301928 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4gs4q\"" Jul 07 18:46:46.302028 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.301983 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 07 18:46:46.308422 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.308162 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 07 18:46:46.314470 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.313648 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69d6b476d6-9lkmx"] Jul 07 18:46:46.317023 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.317001 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-lk6f2"] Jul 07 18:46:46.320909 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.320887 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5vxxf"] Jul 07 18:46:46.321043 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.321028 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lk6f2" Jul 07 18:46:46.324538 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.324517 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5vxxf" Jul 07 18:46:46.325080 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.325060 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 18:46:46.325375 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.325340 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 18:46:46.326288 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.326249 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-x9x4r\"" Jul 07 18:46:46.326440 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.326419 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 18:46:46.326495 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.326481 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 18:46:46.326819 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.326800 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bkl9p\"" Jul 07 18:46:46.327310 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.327289 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 18:46:46.331705 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.331686 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5vxxf"] Jul 07 18:46:46.335953 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.335919 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lk6f2"] Jul 07 18:46:46.408841 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.408807 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:46:46.408841 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.408832 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:46:46.409080 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.408807 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:46:46.412398 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.412377 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 18:46:46.412825 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.412536 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 18:46:46.412825 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.412644 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8xhzt\"" Jul 07 18:46:46.412825 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.412676 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 18:46:46.412825 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.412710 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 18:46:46.412825 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.412711 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9vz2b\"" Jul 07 18:46:46.440646 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.440619 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-ca-trust-extracted\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:46.440783 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.440650 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-certificates\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:46.440783 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.440672 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-trusted-ca\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:46.440783 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.440740 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnjtt\" (UniqueName: \"kubernetes.io/projected/b8d04c06-cf5f-4fe2-a384-a0420d3e4157-kube-api-access-qnjtt\") pod \"ingress-canary-5vxxf\" (UID: \"b8d04c06-cf5f-4fe2-a384-a0420d3e4157\") " pod="openshift-ingress-canary/ingress-canary-5vxxf" Jul 07 18:46:46.440783 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.440768 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-image-registry-private-configuration\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:46.440940 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.440788 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wg7f2\" (UniqueName: \"kubernetes.io/projected/3368babd-6ffc-4ec0-9dae-bcc00845b37f-kube-api-access-wg7f2\") pod \"dns-default-lk6f2\" (UID: \"3368babd-6ffc-4ec0-9dae-bcc00845b37f\") " pod="openshift-dns/dns-default-lk6f2" Jul 07 18:46:46.440940 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.440812 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7rdv\" (UniqueName: \"kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-kube-api-access-k7rdv\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:46.440940 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.440827 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3368babd-6ffc-4ec0-9dae-bcc00845b37f-metrics-tls\") pod \"dns-default-lk6f2\" (UID: \"3368babd-6ffc-4ec0-9dae-bcc00845b37f\") " pod="openshift-dns/dns-default-lk6f2" Jul 07 18:46:46.440940 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.440880 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-installation-pull-secrets\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:46.440940 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.440904 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-tls\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:46.440940 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.440920 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-bound-sa-token\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:46.441168 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.440991 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8d04c06-cf5f-4fe2-a384-a0420d3e4157-cert\") pod \"ingress-canary-5vxxf\" (UID: \"b8d04c06-cf5f-4fe2-a384-a0420d3e4157\") " pod="openshift-ingress-canary/ingress-canary-5vxxf" Jul 07 18:46:46.441168 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.441010 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3368babd-6ffc-4ec0-9dae-bcc00845b37f-config-volume\") pod \"dns-default-lk6f2\" (UID: \"3368babd-6ffc-4ec0-9dae-bcc00845b37f\") " pod="openshift-dns/dns-default-lk6f2" Jul 07 18:46:46.441168 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.441030 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3368babd-6ffc-4ec0-9dae-bcc00845b37f-tmp-dir\") pod \"dns-default-lk6f2\" (UID: \"3368babd-6ffc-4ec0-9dae-bcc00845b37f\") " pod="openshift-dns/dns-default-lk6f2" Jul 07 18:46:46.465379 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.465342 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:46.542433 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.542342 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wg7f2\" (UniqueName: \"kubernetes.io/projected/3368babd-6ffc-4ec0-9dae-bcc00845b37f-kube-api-access-wg7f2\") pod \"dns-default-lk6f2\" (UID: \"3368babd-6ffc-4ec0-9dae-bcc00845b37f\") " pod="openshift-dns/dns-default-lk6f2" Jul 07 18:46:46.542433 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.542396 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7rdv\" (UniqueName: \"kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-kube-api-access-k7rdv\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:46.542646 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.542530 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3368babd-6ffc-4ec0-9dae-bcc00845b37f-metrics-tls\") pod \"dns-default-lk6f2\" (UID: \"3368babd-6ffc-4ec0-9dae-bcc00845b37f\") " pod="openshift-dns/dns-default-lk6f2" Jul 07 18:46:46.542646 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.542595 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-installation-pull-secrets\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:46.542646 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.542627 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-tls\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:46.542810 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.542651 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-bound-sa-token\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:46.542810 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:46.542663 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 18:46:46.542810 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:46.542736 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3368babd-6ffc-4ec0-9dae-bcc00845b37f-metrics-tls podName:3368babd-6ffc-4ec0-9dae-bcc00845b37f nodeName:}" failed. No retries permitted until 2026-07-07 18:46:47.04271316 +0000 UTC m=+36.143731731 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3368babd-6ffc-4ec0-9dae-bcc00845b37f-metrics-tls") pod "dns-default-lk6f2" (UID: "3368babd-6ffc-4ec0-9dae-bcc00845b37f") : secret "dns-default-metrics-tls" not found Jul 07 18:46:46.542810 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:46.542755 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 18:46:46.542810 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.542758 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8d04c06-cf5f-4fe2-a384-a0420d3e4157-cert\") pod \"ingress-canary-5vxxf\" (UID: \"b8d04c06-cf5f-4fe2-a384-a0420d3e4157\") " pod="openshift-ingress-canary/ingress-canary-5vxxf" Jul 07 18:46:46.542810 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:46.542768 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69d6b476d6-9lkmx: secret "image-registry-tls" not found Jul 07 18:46:46.542810 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.542787 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3368babd-6ffc-4ec0-9dae-bcc00845b37f-config-volume\") pod \"dns-default-lk6f2\" (UID: \"3368babd-6ffc-4ec0-9dae-bcc00845b37f\") " pod="openshift-dns/dns-default-lk6f2" Jul 07 18:46:46.542810 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:46.542816 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-tls podName:37675bf2-2454-4e3c-9c53-9f7ae3517b6a nodeName:}" failed. No retries permitted until 2026-07-07 18:46:47.042804525 +0000 UTC m=+36.143823092 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-tls") pod "image-registry-69d6b476d6-9lkmx" (UID: "37675bf2-2454-4e3c-9c53-9f7ae3517b6a") : secret "image-registry-tls" not found Jul 07 18:46:46.543184 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.542832 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3368babd-6ffc-4ec0-9dae-bcc00845b37f-tmp-dir\") pod \"dns-default-lk6f2\" (UID: \"3368babd-6ffc-4ec0-9dae-bcc00845b37f\") " pod="openshift-dns/dns-default-lk6f2" Jul 07 18:46:46.543184 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.542875 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-ca-trust-extracted\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:46.543184 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.542899 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-certificates\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:46.543184 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.542921 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-trusted-ca\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:46.543184 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.542997 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnjtt\" (UniqueName: \"kubernetes.io/projected/b8d04c06-cf5f-4fe2-a384-a0420d3e4157-kube-api-access-qnjtt\") pod \"ingress-canary-5vxxf\" (UID: \"b8d04c06-cf5f-4fe2-a384-a0420d3e4157\") " pod="openshift-ingress-canary/ingress-canary-5vxxf" Jul 07 18:46:46.543184 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.543042 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-image-registry-private-configuration\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:46.543559 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.543466 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3368babd-6ffc-4ec0-9dae-bcc00845b37f-tmp-dir\") pod \"dns-default-lk6f2\" (UID: \"3368babd-6ffc-4ec0-9dae-bcc00845b37f\") " pod="openshift-dns/dns-default-lk6f2" Jul 07 18:46:46.544026 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.543849 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-ca-trust-extracted\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:46.544026 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.543903 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-trusted-ca\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:46.544026 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.543927 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-certificates\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:46.544208 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.544078 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3368babd-6ffc-4ec0-9dae-bcc00845b37f-config-volume\") pod \"dns-default-lk6f2\" (UID: \"3368babd-6ffc-4ec0-9dae-bcc00845b37f\") " pod="openshift-dns/dns-default-lk6f2" Jul 07 18:46:46.547071 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.547044 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-installation-pull-secrets\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:46.547173 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.547098 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-image-registry-private-configuration\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:46.547289 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.547268 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8d04c06-cf5f-4fe2-a384-a0420d3e4157-cert\") pod \"ingress-canary-5vxxf\" (UID: \"b8d04c06-cf5f-4fe2-a384-a0420d3e4157\") " pod="openshift-ingress-canary/ingress-canary-5vxxf" Jul 07 18:46:46.551602 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.551555 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wg7f2\" (UniqueName: \"kubernetes.io/projected/3368babd-6ffc-4ec0-9dae-bcc00845b37f-kube-api-access-wg7f2\") pod \"dns-default-lk6f2\" (UID: \"3368babd-6ffc-4ec0-9dae-bcc00845b37f\") " pod="openshift-dns/dns-default-lk6f2" Jul 07 18:46:46.551810 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.551792 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7rdv\" (UniqueName: \"kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-kube-api-access-k7rdv\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:46.552415 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.552393 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-bound-sa-token\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:46.552725 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.552704 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnjtt\" (UniqueName: \"kubernetes.io/projected/b8d04c06-cf5f-4fe2-a384-a0420d3e4157-kube-api-access-qnjtt\") pod \"ingress-canary-5vxxf\" (UID: \"b8d04c06-cf5f-4fe2-a384-a0420d3e4157\") " pod="openshift-ingress-canary/ingress-canary-5vxxf" Jul 07 18:46:46.563525 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.563463 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:46.641387 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.641352 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5vxxf" Jul 07 18:46:46.645860 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.645834 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-lqs4g"] Jul 07 18:46:46.649339 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.649315 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-lqs4g" Jul 07 18:46:46.652165 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.652098 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 07 18:46:46.652297 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.652221 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 07 18:46:46.652382 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.652361 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-fn4j4\"" Jul 07 18:46:46.652481 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.652460 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 18:46:46.652752 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.652733 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 18:46:46.659079 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.659057 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 07 18:46:46.662442 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.662416 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-lqs4g"] Jul 07 18:46:46.744609 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.744569 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl"] Jul 07 18:46:46.746399 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.746362 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8f4ce07-b53a-4e57-a9ca-e64da6750c7f-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-lqs4g\" (UID: \"b8f4ce07-b53a-4e57-a9ca-e64da6750c7f\") " pod="openshift-insights/insights-operator-5c847dcfc4-lqs4g" Jul 07 18:46:46.746532 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.746487 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8f4ce07-b53a-4e57-a9ca-e64da6750c7f-tmp\") pod \"insights-operator-5c847dcfc4-lqs4g\" (UID: \"b8f4ce07-b53a-4e57-a9ca-e64da6750c7f\") " pod="openshift-insights/insights-operator-5c847dcfc4-lqs4g" Jul 07 18:46:46.746532 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.746519 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b8f4ce07-b53a-4e57-a9ca-e64da6750c7f-serving-cert\") pod \"insights-operator-5c847dcfc4-lqs4g\" (UID: \"b8f4ce07-b53a-4e57-a9ca-e64da6750c7f\") " pod="openshift-insights/insights-operator-5c847dcfc4-lqs4g" Jul 07 18:46:46.746634 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.746599 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8f4ce07-b53a-4e57-a9ca-e64da6750c7f-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-lqs4g\" (UID: \"b8f4ce07-b53a-4e57-a9ca-e64da6750c7f\") " pod="openshift-insights/insights-operator-5c847dcfc4-lqs4g" Jul 07 18:46:46.746682 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.746633 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b8f4ce07-b53a-4e57-a9ca-e64da6750c7f-snapshots\") pod \"insights-operator-5c847dcfc4-lqs4g\" (UID: \"b8f4ce07-b53a-4e57-a9ca-e64da6750c7f\") " pod="openshift-insights/insights-operator-5c847dcfc4-lqs4g" Jul 07 18:46:46.746742 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.746665 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm2pp\" (UniqueName: \"kubernetes.io/projected/b8f4ce07-b53a-4e57-a9ca-e64da6750c7f-kube-api-access-wm2pp\") pod \"insights-operator-5c847dcfc4-lqs4g\" (UID: \"b8f4ce07-b53a-4e57-a9ca-e64da6750c7f\") " pod="openshift-insights/insights-operator-5c847dcfc4-lqs4g" Jul 07 18:46:46.748927 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.748904 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl" Jul 07 18:46:46.752317 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.752278 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-86c68c69b7-p7g8g"] Jul 07 18:46:46.754215 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.754195 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 07 18:46:46.755528 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.755502 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:46:46.756162 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.756144 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 18:46:46.756657 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.756516 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 18:46:46.756883 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.756697 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-87mj9\"" Jul 07 18:46:46.758357 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:46.758327 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"router-dockercfg-n9c8b\" is forbidden: User \"system:node:ip-10-0-140-120.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-140-120.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"router-dockercfg-n9c8b\"" type="*v1.Secret" Jul 07 18:46:46.759567 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:46.759542 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"router-metrics-certs-default\" is forbidden: User \"system:node:ip-10-0-140-120.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-140-120.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" type="*v1.Secret" Jul 07 18:46:46.760393 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:46.760367 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"router-stats-default\" is forbidden: User \"system:node:ip-10-0-140-120.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-140-120.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" type="*v1.Secret" Jul 07 18:46:46.761138 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:46.761059 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-ingress-cert\" is forbidden: User \"system:node:ip-10-0-140-120.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-140-120.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" type="*v1.Secret" Jul 07 18:46:46.761333 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.761310 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 07 18:46:46.761439 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:46.761352 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"service-ca-bundle\" is forbidden: User \"system:node:ip-10-0-140-120.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-140-120.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" type="*v1.ConfigMap" Jul 07 18:46:46.762350 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:46.762324 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-140-120.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-140-120.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 07 18:46:46.763074 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:46.763054 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-140-120.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-140-120.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 07 18:46:46.792916 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.792838 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl"] Jul 07 18:46:46.805000 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.804968 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-86c68c69b7-p7g8g"] Jul 07 18:46:46.847152 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.847113 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8f4ce07-b53a-4e57-a9ca-e64da6750c7f-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-lqs4g\" (UID: \"b8f4ce07-b53a-4e57-a9ca-e64da6750c7f\") " pod="openshift-insights/insights-operator-5c847dcfc4-lqs4g" Jul 07 18:46:46.847331 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.847174 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/363d866e-b0a1-4351-8f85-b994a18200a7-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-wszbl\" (UID: \"363d866e-b0a1-4351-8f85-b994a18200a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl" Jul 07 18:46:46.847331 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.847238 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/363d866e-b0a1-4351-8f85-b994a18200a7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-wszbl\" (UID: \"363d866e-b0a1-4351-8f85-b994a18200a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl" Jul 07 18:46:46.847331 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.847315 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wpg9\" (UniqueName: \"kubernetes.io/projected/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-kube-api-access-4wpg9\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:46:46.847499 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.847339 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7xt9\" (UniqueName: \"kubernetes.io/projected/363d866e-b0a1-4351-8f85-b994a18200a7-kube-api-access-v7xt9\") pod \"cluster-monitoring-operator-64786d9cbb-wszbl\" (UID: \"363d866e-b0a1-4351-8f85-b994a18200a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl" Jul 07 18:46:46.847499 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.847368 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b8f4ce07-b53a-4e57-a9ca-e64da6750c7f-serving-cert\") pod \"insights-operator-5c847dcfc4-lqs4g\" (UID: \"b8f4ce07-b53a-4e57-a9ca-e64da6750c7f\") " pod="openshift-insights/insights-operator-5c847dcfc4-lqs4g" Jul 07 18:46:46.847499 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.847464 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-metrics-certs\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:46:46.847636 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.847503 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8f4ce07-b53a-4e57-a9ca-e64da6750c7f-tmp\") pod \"insights-operator-5c847dcfc4-lqs4g\" (UID: \"b8f4ce07-b53a-4e57-a9ca-e64da6750c7f\") " pod="openshift-insights/insights-operator-5c847dcfc4-lqs4g" Jul 07 18:46:46.847636 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.847580 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b8f4ce07-b53a-4e57-a9ca-e64da6750c7f-snapshots\") pod \"insights-operator-5c847dcfc4-lqs4g\" (UID: \"b8f4ce07-b53a-4e57-a9ca-e64da6750c7f\") " pod="openshift-insights/insights-operator-5c847dcfc4-lqs4g" Jul 07 18:46:46.847730 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.847668 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8f4ce07-b53a-4e57-a9ca-e64da6750c7f-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-lqs4g\" (UID: \"b8f4ce07-b53a-4e57-a9ca-e64da6750c7f\") " pod="openshift-insights/insights-operator-5c847dcfc4-lqs4g" Jul 07 18:46:46.847797 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.847784 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8f4ce07-b53a-4e57-a9ca-e64da6750c7f-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-lqs4g\" (UID: \"b8f4ce07-b53a-4e57-a9ca-e64da6750c7f\") " pod="openshift-insights/insights-operator-5c847dcfc4-lqs4g" Jul 07 18:46:46.848035 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.848009 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-default-certificate\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:46:46.848164 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.848064 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-stats-auth\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:46:46.848225 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.848199 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm2pp\" (UniqueName: \"kubernetes.io/projected/b8f4ce07-b53a-4e57-a9ca-e64da6750c7f-kube-api-access-wm2pp\") pod \"insights-operator-5c847dcfc4-lqs4g\" (UID: \"b8f4ce07-b53a-4e57-a9ca-e64da6750c7f\") " pod="openshift-insights/insights-operator-5c847dcfc4-lqs4g" Jul 07 18:46:46.848305 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.848228 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-service-ca-bundle\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:46:46.848639 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.848612 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8f4ce07-b53a-4e57-a9ca-e64da6750c7f-tmp\") pod \"insights-operator-5c847dcfc4-lqs4g\" (UID: \"b8f4ce07-b53a-4e57-a9ca-e64da6750c7f\") " pod="openshift-insights/insights-operator-5c847dcfc4-lqs4g" Jul 07 18:46:46.848639 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.848631 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b8f4ce07-b53a-4e57-a9ca-e64da6750c7f-snapshots\") pod \"insights-operator-5c847dcfc4-lqs4g\" (UID: \"b8f4ce07-b53a-4e57-a9ca-e64da6750c7f\") " pod="openshift-insights/insights-operator-5c847dcfc4-lqs4g" Jul 07 18:46:46.849086 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.849067 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8f4ce07-b53a-4e57-a9ca-e64da6750c7f-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-lqs4g\" (UID: \"b8f4ce07-b53a-4e57-a9ca-e64da6750c7f\") " pod="openshift-insights/insights-operator-5c847dcfc4-lqs4g" Jul 07 18:46:46.850128 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.850106 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b8f4ce07-b53a-4e57-a9ca-e64da6750c7f-serving-cert\") pod \"insights-operator-5c847dcfc4-lqs4g\" (UID: \"b8f4ce07-b53a-4e57-a9ca-e64da6750c7f\") " pod="openshift-insights/insights-operator-5c847dcfc4-lqs4g" Jul 07 18:46:46.856645 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.856625 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm2pp\" (UniqueName: \"kubernetes.io/projected/b8f4ce07-b53a-4e57-a9ca-e64da6750c7f-kube-api-access-wm2pp\") pod \"insights-operator-5c847dcfc4-lqs4g\" (UID: \"b8f4ce07-b53a-4e57-a9ca-e64da6750c7f\") " pod="openshift-insights/insights-operator-5c847dcfc4-lqs4g" Jul 07 18:46:46.949191 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.949152 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/363d866e-b0a1-4351-8f85-b994a18200a7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-wszbl\" (UID: \"363d866e-b0a1-4351-8f85-b994a18200a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl" Jul 07 18:46:46.949387 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.949212 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wpg9\" (UniqueName: \"kubernetes.io/projected/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-kube-api-access-4wpg9\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:46:46.949387 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.949236 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7xt9\" (UniqueName: \"kubernetes.io/projected/363d866e-b0a1-4351-8f85-b994a18200a7-kube-api-access-v7xt9\") pod \"cluster-monitoring-operator-64786d9cbb-wszbl\" (UID: \"363d866e-b0a1-4351-8f85-b994a18200a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl" Jul 07 18:46:46.949387 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.949285 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-metrics-certs\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:46:46.949387 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.949343 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-default-certificate\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:46:46.949387 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.949366 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-stats-auth\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:46:46.949632 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:46.949388 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 18:46:46.949632 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.949391 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-service-ca-bundle\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:46:46.949632 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:46.949478 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/363d866e-b0a1-4351-8f85-b994a18200a7-cluster-monitoring-operator-tls podName:363d866e-b0a1-4351-8f85-b994a18200a7 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:47.449456023 +0000 UTC m=+36.550474596 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/363d866e-b0a1-4351-8f85-b994a18200a7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-wszbl" (UID: "363d866e-b0a1-4351-8f85-b994a18200a7") : secret "cluster-monitoring-operator-tls" not found Jul 07 18:46:46.949632 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.949530 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/363d866e-b0a1-4351-8f85-b994a18200a7-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-wszbl\" (UID: \"363d866e-b0a1-4351-8f85-b994a18200a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl" Jul 07 18:46:46.950326 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.950305 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/363d866e-b0a1-4351-8f85-b994a18200a7-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-wszbl\" (UID: \"363d866e-b0a1-4351-8f85-b994a18200a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl" Jul 07 18:46:46.960961 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.960935 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-lqs4g" Jul 07 18:46:46.961756 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.961732 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:46.961968 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:46.961950 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7xt9\" (UniqueName: \"kubernetes.io/projected/363d866e-b0a1-4351-8f85-b994a18200a7-kube-api-access-v7xt9\") pod \"cluster-monitoring-operator-64786d9cbb-wszbl\" (UID: \"363d866e-b0a1-4351-8f85-b994a18200a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl" Jul 07 18:46:47.050249 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.050163 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-tls\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:47.050431 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.050278 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3368babd-6ffc-4ec0-9dae-bcc00845b37f-metrics-tls\") pod \"dns-default-lk6f2\" (UID: \"3368babd-6ffc-4ec0-9dae-bcc00845b37f\") " pod="openshift-dns/dns-default-lk6f2" Jul 07 18:46:47.050431 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:47.050347 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 18:46:47.050431 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:47.050375 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69d6b476d6-9lkmx: secret "image-registry-tls" not found Jul 07 18:46:47.050431 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:47.050425 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 18:46:47.050601 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:47.050449 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-tls podName:37675bf2-2454-4e3c-9c53-9f7ae3517b6a nodeName:}" failed. No retries permitted until 2026-07-07 18:46:48.050428282 +0000 UTC m=+37.151446858 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-tls") pod "image-registry-69d6b476d6-9lkmx" (UID: "37675bf2-2454-4e3c-9c53-9f7ae3517b6a") : secret "image-registry-tls" not found Jul 07 18:46:47.050601 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:47.050495 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3368babd-6ffc-4ec0-9dae-bcc00845b37f-metrics-tls podName:3368babd-6ffc-4ec0-9dae-bcc00845b37f nodeName:}" failed. No retries permitted until 2026-07-07 18:46:48.050475719 +0000 UTC m=+37.151494280 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3368babd-6ffc-4ec0-9dae-bcc00845b37f-metrics-tls") pod "dns-default-lk6f2" (UID: "3368babd-6ffc-4ec0-9dae-bcc00845b37f") : secret "dns-default-metrics-tls" not found Jul 07 18:46:47.061399 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.061370 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:47.453452 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.453369 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/363d866e-b0a1-4351-8f85-b994a18200a7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-wszbl\" (UID: \"363d866e-b0a1-4351-8f85-b994a18200a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl" Jul 07 18:46:47.453833 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:47.453528 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 18:46:47.453833 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:47.453600 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/363d866e-b0a1-4351-8f85-b994a18200a7-cluster-monitoring-operator-tls podName:363d866e-b0a1-4351-8f85-b994a18200a7 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:48.453585363 +0000 UTC m=+37.554603916 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/363d866e-b0a1-4351-8f85-b994a18200a7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-wszbl" (UID: "363d866e-b0a1-4351-8f85-b994a18200a7") : secret "cluster-monitoring-operator-tls" not found Jul 07 18:46:47.619219 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.619187 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zg9v"] Jul 07 18:46:47.619528 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.619487 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 18:46:47.622194 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.622170 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zg9v" Jul 07 18:46:47.624463 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.624437 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-26cqm\"" Jul 07 18:46:47.624808 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.624791 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 07 18:46:47.624919 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.624857 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 07 18:46:47.624919 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.624892 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 07 18:46:47.631149 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.631129 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zg9v"] Jul 07 18:46:47.719101 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.719087 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cbv87"] Jul 07 18:46:47.722345 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.722323 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cbv87" Jul 07 18:46:47.726857 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.725829 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-jlwwh\"" Jul 07 18:46:47.726857 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.726187 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 07 18:46:47.726857 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.726435 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 07 18:46:47.735372 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.733470 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cbv87"] Jul 07 18:46:47.755602 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.755378 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dccf34c-d46e-4122-81a2-2794860331a8-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-7zg9v\" (UID: \"3dccf34c-d46e-4122-81a2-2794860331a8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zg9v" Jul 07 18:46:47.755602 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.755452 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72pn2\" (UniqueName: \"kubernetes.io/projected/3dccf34c-d46e-4122-81a2-2794860331a8-kube-api-access-72pn2\") pod \"cluster-samples-operator-56fcf86bf-7zg9v\" (UID: \"3dccf34c-d46e-4122-81a2-2794860331a8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zg9v" Jul 07 18:46:47.765220 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.765111 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 07 18:46:47.774766 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.774735 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-default-certificate\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:46:47.811118 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.811090 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5vxxf"] Jul 07 18:46:47.813395 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.813367 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-lqs4g"] Jul 07 18:46:47.813521 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.813502 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 18:46:47.817447 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:47.817418 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8f4ce07_b53a_4e57_a9ca_e64da6750c7f.slice/crio-895baeadc552310194efa8b5de6facad4b6df49b96a31597d6c97477886ef2aa WatchSource:0}: Error finding container 895baeadc552310194efa8b5de6facad4b6df49b96a31597d6c97477886ef2aa: Status 404 returned error can't find the container with id 895baeadc552310194efa8b5de6facad4b6df49b96a31597d6c97477886ef2aa Jul 07 18:46:47.818496 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:47.818470 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8d04c06_cf5f_4fe2_a384_a0420d3e4157.slice/crio-459be6fce6679621438df1cc6a0aac823a197e955a7edab99b456ef6830af503 WatchSource:0}: Error finding container 459be6fce6679621438df1cc6a0aac823a197e955a7edab99b456ef6830af503: Status 404 returned error can't find the container with id 459be6fce6679621438df1cc6a0aac823a197e955a7edab99b456ef6830af503 Jul 07 18:46:47.824456 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.824437 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wpg9\" (UniqueName: \"kubernetes.io/projected/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-kube-api-access-4wpg9\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:46:47.856816 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.856784 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72pn2\" (UniqueName: \"kubernetes.io/projected/3dccf34c-d46e-4122-81a2-2794860331a8-kube-api-access-72pn2\") pod \"cluster-samples-operator-56fcf86bf-7zg9v\" (UID: \"3dccf34c-d46e-4122-81a2-2794860331a8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zg9v" Jul 07 18:46:47.856954 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.856825 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxkg5\" (UniqueName: \"kubernetes.io/projected/fe9ddaa2-77d8-4b20-866d-caa5fce1427c-kube-api-access-vxkg5\") pod \"volume-data-source-validator-5d456fdbcd-cbv87\" (UID: \"fe9ddaa2-77d8-4b20-866d-caa5fce1427c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cbv87" Jul 07 18:46:47.857059 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.857041 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dccf34c-d46e-4122-81a2-2794860331a8-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-7zg9v\" (UID: \"3dccf34c-d46e-4122-81a2-2794860331a8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zg9v" Jul 07 18:46:47.857174 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:47.857159 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 18:46:47.857240 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:47.857229 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dccf34c-d46e-4122-81a2-2794860331a8-samples-operator-tls podName:3dccf34c-d46e-4122-81a2-2794860331a8 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:48.357210745 +0000 UTC m=+37.458229315 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3dccf34c-d46e-4122-81a2-2794860331a8-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-7zg9v" (UID: "3dccf34c-d46e-4122-81a2-2794860331a8") : secret "samples-operator-tls" not found Jul 07 18:46:47.865913 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.865892 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72pn2\" (UniqueName: \"kubernetes.io/projected/3dccf34c-d46e-4122-81a2-2794860331a8-kube-api-access-72pn2\") pod \"cluster-samples-operator-56fcf86bf-7zg9v\" (UID: \"3dccf34c-d46e-4122-81a2-2794860331a8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zg9v" Jul 07 18:46:47.950389 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:47.950364 2583 configmap.go:193] Couldn't get configMap openshift-ingress/service-ca-bundle: failed to sync configmap cache: timed out waiting for the condition Jul 07 18:46:47.950389 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:47.950377 2583 secret.go:189] Couldn't get secret openshift-ingress/router-stats-default: failed to sync secret cache: timed out waiting for the condition Jul 07 18:46:47.950529 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:47.950376 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: failed to sync secret cache: timed out waiting for the condition Jul 07 18:46:47.950529 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:47.950447 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-service-ca-bundle podName:7ea1005f-1048-4ec3-8fb7-e5ed794095a7 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:48.450431302 +0000 UTC m=+37.551449855 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-service-ca-bundle") pod "router-default-86c68c69b7-p7g8g" (UID: "7ea1005f-1048-4ec3-8fb7-e5ed794095a7") : failed to sync configmap cache: timed out waiting for the condition Jul 07 18:46:47.950529 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:47.950468 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-metrics-certs podName:7ea1005f-1048-4ec3-8fb7-e5ed794095a7 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:48.450454861 +0000 UTC m=+37.551473416 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-metrics-certs") pod "router-default-86c68c69b7-p7g8g" (UID: "7ea1005f-1048-4ec3-8fb7-e5ed794095a7") : failed to sync secret cache: timed out waiting for the condition Jul 07 18:46:47.950529 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:47.950481 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-stats-auth podName:7ea1005f-1048-4ec3-8fb7-e5ed794095a7 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:48.450476028 +0000 UTC m=+37.551494581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "stats-auth" (UniqueName: "kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-stats-auth") pod "router-default-86c68c69b7-p7g8g" (UID: "7ea1005f-1048-4ec3-8fb7-e5ed794095a7") : failed to sync secret cache: timed out waiting for the condition Jul 07 18:46:47.957969 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.957914 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxkg5\" (UniqueName: \"kubernetes.io/projected/fe9ddaa2-77d8-4b20-866d-caa5fce1427c-kube-api-access-vxkg5\") pod \"volume-data-source-validator-5d456fdbcd-cbv87\" (UID: \"fe9ddaa2-77d8-4b20-866d-caa5fce1427c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cbv87" Jul 07 18:46:47.966080 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:47.966061 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxkg5\" (UniqueName: \"kubernetes.io/projected/fe9ddaa2-77d8-4b20-866d-caa5fce1427c-kube-api-access-vxkg5\") pod \"volume-data-source-validator-5d456fdbcd-cbv87\" (UID: \"fe9ddaa2-77d8-4b20-866d-caa5fce1427c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cbv87" Jul 07 18:46:48.038767 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.038729 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cbv87" Jul 07 18:46:48.042928 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.042906 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 07 18:46:48.059065 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.058855 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-tls\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:48.059184 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:48.059000 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 18:46:48.059184 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:48.059142 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69d6b476d6-9lkmx: secret "image-registry-tls" not found Jul 07 18:46:48.059294 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.059187 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3368babd-6ffc-4ec0-9dae-bcc00845b37f-metrics-tls\") pod \"dns-default-lk6f2\" (UID: \"3368babd-6ffc-4ec0-9dae-bcc00845b37f\") " pod="openshift-dns/dns-default-lk6f2" Jul 07 18:46:48.059294 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:48.059201 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-tls podName:37675bf2-2454-4e3c-9c53-9f7ae3517b6a nodeName:}" failed. No retries permitted until 2026-07-07 18:46:50.059185855 +0000 UTC m=+39.160204408 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-tls") pod "image-registry-69d6b476d6-9lkmx" (UID: "37675bf2-2454-4e3c-9c53-9f7ae3517b6a") : secret "image-registry-tls" not found Jul 07 18:46:48.059381 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:48.059310 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 18:46:48.059381 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:48.059365 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3368babd-6ffc-4ec0-9dae-bcc00845b37f-metrics-tls podName:3368babd-6ffc-4ec0-9dae-bcc00845b37f nodeName:}" failed. No retries permitted until 2026-07-07 18:46:50.059348899 +0000 UTC m=+39.160367454 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3368babd-6ffc-4ec0-9dae-bcc00845b37f-metrics-tls") pod "dns-default-lk6f2" (UID: "3368babd-6ffc-4ec0-9dae-bcc00845b37f") : secret "dns-default-metrics-tls" not found Jul 07 18:46:48.083079 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.083052 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-n9c8b\"" Jul 07 18:46:48.170327 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.170296 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cbv87"] Jul 07 18:46:48.173191 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:48.173168 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe9ddaa2_77d8_4b20_866d_caa5fce1427c.slice/crio-48d9bac87a90a4a4b1856cc8df83997e552712dcad550c62c89d1dd91f919bda WatchSource:0}: Error finding container 48d9bac87a90a4a4b1856cc8df83997e552712dcad550c62c89d1dd91f919bda: Status 404 returned error can't find the container with id 48d9bac87a90a4a4b1856cc8df83997e552712dcad550c62c89d1dd91f919bda Jul 07 18:46:48.184472 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.184444 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 07 18:46:48.361092 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.360994 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dccf34c-d46e-4122-81a2-2794860331a8-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-7zg9v\" (UID: \"3dccf34c-d46e-4122-81a2-2794860331a8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zg9v" Jul 07 18:46:48.361239 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:48.361228 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 18:46:48.361387 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:48.361373 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dccf34c-d46e-4122-81a2-2794860331a8-samples-operator-tls podName:3dccf34c-d46e-4122-81a2-2794860331a8 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:49.361347217 +0000 UTC m=+38.462365790 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3dccf34c-d46e-4122-81a2-2794860331a8-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-7zg9v" (UID: "3dccf34c-d46e-4122-81a2-2794860331a8") : secret "samples-operator-tls" not found Jul 07 18:46:48.362918 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.362723 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 07 18:46:48.462170 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.462127 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-service-ca-bundle\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:46:48.462775 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.462215 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/363d866e-b0a1-4351-8f85-b994a18200a7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-wszbl\" (UID: \"363d866e-b0a1-4351-8f85-b994a18200a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl" Jul 07 18:46:48.462775 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.462304 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-metrics-certs\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:46:48.462775 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:48.462345 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-service-ca-bundle podName:7ea1005f-1048-4ec3-8fb7-e5ed794095a7 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:49.462319363 +0000 UTC m=+38.563337923 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-service-ca-bundle") pod "router-default-86c68c69b7-p7g8g" (UID: "7ea1005f-1048-4ec3-8fb7-e5ed794095a7") : configmap references non-existent config key: service-ca.crt Jul 07 18:46:48.462775 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:48.462396 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 18:46:48.462775 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:48.462404 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 18:46:48.462775 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:48.462449 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/363d866e-b0a1-4351-8f85-b994a18200a7-cluster-monitoring-operator-tls podName:363d866e-b0a1-4351-8f85-b994a18200a7 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:50.462438255 +0000 UTC m=+39.563456811 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/363d866e-b0a1-4351-8f85-b994a18200a7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-wszbl" (UID: "363d866e-b0a1-4351-8f85-b994a18200a7") : secret "cluster-monitoring-operator-tls" not found Jul 07 18:46:48.462775 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:48.462470 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-metrics-certs podName:7ea1005f-1048-4ec3-8fb7-e5ed794095a7 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:49.462460372 +0000 UTC m=+38.563478928 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-metrics-certs") pod "router-default-86c68c69b7-p7g8g" (UID: "7ea1005f-1048-4ec3-8fb7-e5ed794095a7") : secret "router-metrics-certs-default" not found Jul 07 18:46:48.462775 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.462528 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-stats-auth\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:46:48.465677 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.465651 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-stats-auth\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:46:48.582409 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.582320 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cbv87" event={"ID":"fe9ddaa2-77d8-4b20-866d-caa5fce1427c","Type":"ContainerStarted","Data":"48d9bac87a90a4a4b1856cc8df83997e552712dcad550c62c89d1dd91f919bda"} Jul 07 18:46:48.587683 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.587651 2583 generic.go:356] "Generic (PLEG): container finished" podID="30caa190-8d65-4825-a681-4af09975ef3e" containerID="bcf6564086f7d1833b703b320f01e01e0a16fb1ef111742249d97f7d5af2ee07" exitCode=0 Jul 07 18:46:48.587819 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.587728 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-84cwc" event={"ID":"30caa190-8d65-4825-a681-4af09975ef3e","Type":"ContainerDied","Data":"bcf6564086f7d1833b703b320f01e01e0a16fb1ef111742249d97f7d5af2ee07"} Jul 07 18:46:48.589750 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.589692 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-lqs4g" event={"ID":"b8f4ce07-b53a-4e57-a9ca-e64da6750c7f","Type":"ContainerStarted","Data":"895baeadc552310194efa8b5de6facad4b6df49b96a31597d6c97477886ef2aa"} Jul 07 18:46:48.593159 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.592999 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5vxxf" event={"ID":"b8d04c06-cf5f-4fe2-a384-a0420d3e4157","Type":"ContainerStarted","Data":"459be6fce6679621438df1cc6a0aac823a197e955a7edab99b456ef6830af503"} Jul 07 18:46:48.610115 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.610086 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-5x9w8"] Jul 07 18:46:48.617643 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.617580 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-5x9w8" Jul 07 18:46:48.628282 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.627841 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 07 18:46:48.628282 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.628153 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-7mzs5\"" Jul 07 18:46:48.631395 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.628589 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 07 18:46:48.631395 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.628826 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 07 18:46:48.631395 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.629079 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 07 18:46:48.631395 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.630824 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-5x9w8"] Jul 07 18:46:48.635478 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.635456 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 07 18:46:48.765418 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.765334 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctds2\" (UniqueName: \"kubernetes.io/projected/f390fdec-e425-4d23-a559-9681817515ec-kube-api-access-ctds2\") pod \"console-operator-68b757fb67-5x9w8\" (UID: \"f390fdec-e425-4d23-a559-9681817515ec\") " pod="openshift-console-operator/console-operator-68b757fb67-5x9w8" Jul 07 18:46:48.765555 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.765482 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f390fdec-e425-4d23-a559-9681817515ec-trusted-ca\") pod \"console-operator-68b757fb67-5x9w8\" (UID: \"f390fdec-e425-4d23-a559-9681817515ec\") " pod="openshift-console-operator/console-operator-68b757fb67-5x9w8" Jul 07 18:46:48.765555 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.765526 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f390fdec-e425-4d23-a559-9681817515ec-config\") pod \"console-operator-68b757fb67-5x9w8\" (UID: \"f390fdec-e425-4d23-a559-9681817515ec\") " pod="openshift-console-operator/console-operator-68b757fb67-5x9w8" Jul 07 18:46:48.765653 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.765627 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f390fdec-e425-4d23-a559-9681817515ec-serving-cert\") pod \"console-operator-68b757fb67-5x9w8\" (UID: \"f390fdec-e425-4d23-a559-9681817515ec\") " pod="openshift-console-operator/console-operator-68b757fb67-5x9w8" Jul 07 18:46:48.866886 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.866852 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctds2\" (UniqueName: \"kubernetes.io/projected/f390fdec-e425-4d23-a559-9681817515ec-kube-api-access-ctds2\") pod \"console-operator-68b757fb67-5x9w8\" (UID: \"f390fdec-e425-4d23-a559-9681817515ec\") " pod="openshift-console-operator/console-operator-68b757fb67-5x9w8" Jul 07 18:46:48.867050 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.866984 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f390fdec-e425-4d23-a559-9681817515ec-trusted-ca\") pod \"console-operator-68b757fb67-5x9w8\" (UID: \"f390fdec-e425-4d23-a559-9681817515ec\") " pod="openshift-console-operator/console-operator-68b757fb67-5x9w8" Jul 07 18:46:48.867050 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.867022 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f390fdec-e425-4d23-a559-9681817515ec-config\") pod \"console-operator-68b757fb67-5x9w8\" (UID: \"f390fdec-e425-4d23-a559-9681817515ec\") " pod="openshift-console-operator/console-operator-68b757fb67-5x9w8" Jul 07 18:46:48.867187 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.867080 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f390fdec-e425-4d23-a559-9681817515ec-serving-cert\") pod \"console-operator-68b757fb67-5x9w8\" (UID: \"f390fdec-e425-4d23-a559-9681817515ec\") " pod="openshift-console-operator/console-operator-68b757fb67-5x9w8" Jul 07 18:46:48.868308 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.868222 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f390fdec-e425-4d23-a559-9681817515ec-config\") pod \"console-operator-68b757fb67-5x9w8\" (UID: \"f390fdec-e425-4d23-a559-9681817515ec\") " pod="openshift-console-operator/console-operator-68b757fb67-5x9w8" Jul 07 18:46:48.868415 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.868402 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f390fdec-e425-4d23-a559-9681817515ec-trusted-ca\") pod \"console-operator-68b757fb67-5x9w8\" (UID: \"f390fdec-e425-4d23-a559-9681817515ec\") " pod="openshift-console-operator/console-operator-68b757fb67-5x9w8" Jul 07 18:46:48.870916 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.870891 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f390fdec-e425-4d23-a559-9681817515ec-serving-cert\") pod \"console-operator-68b757fb67-5x9w8\" (UID: \"f390fdec-e425-4d23-a559-9681817515ec\") " pod="openshift-console-operator/console-operator-68b757fb67-5x9w8" Jul 07 18:46:48.878473 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.878447 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctds2\" (UniqueName: \"kubernetes.io/projected/f390fdec-e425-4d23-a559-9681817515ec-kube-api-access-ctds2\") pod \"console-operator-68b757fb67-5x9w8\" (UID: \"f390fdec-e425-4d23-a559-9681817515ec\") " pod="openshift-console-operator/console-operator-68b757fb67-5x9w8" Jul 07 18:46:48.941473 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:48.941434 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-5x9w8" Jul 07 18:46:49.251251 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:49.251219 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:49.371913 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:49.371685 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dccf34c-d46e-4122-81a2-2794860331a8-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-7zg9v\" (UID: \"3dccf34c-d46e-4122-81a2-2794860331a8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zg9v" Jul 07 18:46:49.372079 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:49.371829 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 18:46:49.372079 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:49.372076 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dccf34c-d46e-4122-81a2-2794860331a8-samples-operator-tls podName:3dccf34c-d46e-4122-81a2-2794860331a8 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:51.372049849 +0000 UTC m=+40.473068416 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3dccf34c-d46e-4122-81a2-2794860331a8-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-7zg9v" (UID: "3dccf34c-d46e-4122-81a2-2794860331a8") : secret "samples-operator-tls" not found Jul 07 18:46:49.377400 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:49.377367 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:49.387815 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:49.387771 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-5x9w8"] Jul 07 18:46:49.390413 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:46:49.390374 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf390fdec_e425_4d23_a559_9681817515ec.slice/crio-b2b887c97447b5ba8ad36dbd9d669fe5377ab372cc129f165c5298064088879e WatchSource:0}: Error finding container b2b887c97447b5ba8ad36dbd9d669fe5377ab372cc129f165c5298064088879e: Status 404 returned error can't find the container with id b2b887c97447b5ba8ad36dbd9d669fe5377ab372cc129f165c5298064088879e Jul 07 18:46:49.472459 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:49.472421 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-metrics-certs\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:46:49.472982 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:49.472497 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-service-ca-bundle\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:46:49.472982 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:49.472588 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 18:46:49.472982 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:49.472632 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-service-ca-bundle podName:7ea1005f-1048-4ec3-8fb7-e5ed794095a7 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:51.472612275 +0000 UTC m=+40.573630849 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-service-ca-bundle") pod "router-default-86c68c69b7-p7g8g" (UID: "7ea1005f-1048-4ec3-8fb7-e5ed794095a7") : configmap references non-existent config key: service-ca.crt Jul 07 18:46:49.472982 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:49.472651 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-metrics-certs podName:7ea1005f-1048-4ec3-8fb7-e5ed794095a7 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:51.472642181 +0000 UTC m=+40.573660740 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-metrics-certs") pod "router-default-86c68c69b7-p7g8g" (UID: "7ea1005f-1048-4ec3-8fb7-e5ed794095a7") : secret "router-metrics-certs-default" not found Jul 07 18:46:49.484310 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:49.484281 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:49.585524 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:49.585427 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:49.597774 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:49.597744 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cbv87" event={"ID":"fe9ddaa2-77d8-4b20-866d-caa5fce1427c","Type":"ContainerStarted","Data":"a6e132e6ccf7d64c561dd4056283cdc16db653fac864451291ca766100eca6b3"} Jul 07 18:46:49.601060 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:49.601030 2583 generic.go:356] "Generic (PLEG): container finished" podID="30caa190-8d65-4825-a681-4af09975ef3e" containerID="a9c044680d0723777f5d4bf9445409f515284b15b681dbb4f0dbab3b71ea67cf" exitCode=0 Jul 07 18:46:49.601182 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:49.601118 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-84cwc" event={"ID":"30caa190-8d65-4825-a681-4af09975ef3e","Type":"ContainerDied","Data":"a9c044680d0723777f5d4bf9445409f515284b15b681dbb4f0dbab3b71ea67cf"} Jul 07 18:46:49.602890 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:49.602867 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-5x9w8" event={"ID":"f390fdec-e425-4d23-a559-9681817515ec","Type":"ContainerStarted","Data":"b2b887c97447b5ba8ad36dbd9d669fe5377ab372cc129f165c5298064088879e"} Jul 07 18:46:49.614871 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:49.614816 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cbv87" podStartSLOduration=1.504106255 podStartE2EDuration="2.614800305s" totalImagesPullingTime="1.11069405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:49.613928456 +0000 UTC m=+38.714947031" watchObservedRunningTime="2026-07-07 18:46:49.614800305 +0000 UTC m=+38.715818880" Jul 07 18:46:49.659850 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:49.659813 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:49.731915 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:49.731850 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:50.077620 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:50.077585 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3368babd-6ffc-4ec0-9dae-bcc00845b37f-metrics-tls\") pod \"dns-default-lk6f2\" (UID: \"3368babd-6ffc-4ec0-9dae-bcc00845b37f\") " pod="openshift-dns/dns-default-lk6f2" Jul 07 18:46:50.077776 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:50.077706 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-tls\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:50.077776 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:50.077750 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 18:46:50.077882 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:50.077818 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3368babd-6ffc-4ec0-9dae-bcc00845b37f-metrics-tls podName:3368babd-6ffc-4ec0-9dae-bcc00845b37f nodeName:}" failed. No retries permitted until 2026-07-07 18:46:54.077795813 +0000 UTC m=+43.178814368 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3368babd-6ffc-4ec0-9dae-bcc00845b37f-metrics-tls") pod "dns-default-lk6f2" (UID: "3368babd-6ffc-4ec0-9dae-bcc00845b37f") : secret "dns-default-metrics-tls" not found Jul 07 18:46:50.077882 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:50.077854 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 18:46:50.077882 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:50.077870 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69d6b476d6-9lkmx: secret "image-registry-tls" not found Jul 07 18:46:50.078017 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:50.077936 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-tls podName:37675bf2-2454-4e3c-9c53-9f7ae3517b6a nodeName:}" failed. No retries permitted until 2026-07-07 18:46:54.077921204 +0000 UTC m=+43.178939756 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-tls") pod "image-registry-69d6b476d6-9lkmx" (UID: "37675bf2-2454-4e3c-9c53-9f7ae3517b6a") : secret "image-registry-tls" not found Jul 07 18:46:50.446656 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:50.446617 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:50.480731 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:50.480685 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/363d866e-b0a1-4351-8f85-b994a18200a7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-wszbl\" (UID: \"363d866e-b0a1-4351-8f85-b994a18200a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl" Jul 07 18:46:50.481196 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:50.480854 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 18:46:50.481196 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:50.480943 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/363d866e-b0a1-4351-8f85-b994a18200a7-cluster-monitoring-operator-tls podName:363d866e-b0a1-4351-8f85-b994a18200a7 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:54.4809224 +0000 UTC m=+43.581940974 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/363d866e-b0a1-4351-8f85-b994a18200a7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-wszbl" (UID: "363d866e-b0a1-4351-8f85-b994a18200a7") : secret "cluster-monitoring-operator-tls" not found Jul 07 18:46:50.552937 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:50.552870 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:50.652786 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:50.652755 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:51.389434 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:51.388706 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dccf34c-d46e-4122-81a2-2794860331a8-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-7zg9v\" (UID: \"3dccf34c-d46e-4122-81a2-2794860331a8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zg9v" Jul 07 18:46:51.389434 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:51.388959 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 18:46:51.389434 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:51.389022 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dccf34c-d46e-4122-81a2-2794860331a8-samples-operator-tls podName:3dccf34c-d46e-4122-81a2-2794860331a8 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:55.389004148 +0000 UTC m=+44.490022718 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3dccf34c-d46e-4122-81a2-2794860331a8-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-7zg9v" (UID: "3dccf34c-d46e-4122-81a2-2794860331a8") : secret "samples-operator-tls" not found Jul 07 18:46:51.489651 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:51.489559 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-metrics-certs\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:46:51.490101 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:51.489671 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-service-ca-bundle\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:46:51.490101 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:51.489745 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 18:46:51.490101 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:51.489823 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-metrics-certs podName:7ea1005f-1048-4ec3-8fb7-e5ed794095a7 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:55.489801971 +0000 UTC m=+44.590820543 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-metrics-certs") pod "router-default-86c68c69b7-p7g8g" (UID: "7ea1005f-1048-4ec3-8fb7-e5ed794095a7") : secret "router-metrics-certs-default" not found Jul 07 18:46:51.490101 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:51.489866 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-service-ca-bundle podName:7ea1005f-1048-4ec3-8fb7-e5ed794095a7 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:55.489849983 +0000 UTC m=+44.590868540 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-service-ca-bundle") pod "router-default-86c68c69b7-p7g8g" (UID: "7ea1005f-1048-4ec3-8fb7-e5ed794095a7") : configmap references non-existent config key: service-ca.crt Jul 07 18:46:51.609583 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:51.609134 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-lqs4g" event={"ID":"b8f4ce07-b53a-4e57-a9ca-e64da6750c7f","Type":"ContainerStarted","Data":"540ff09500b92833ea4d9d0d2a9228f110ec2651df611b077f6db3b21bba1109"} Jul 07 18:46:51.613168 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:51.613140 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-84cwc" event={"ID":"30caa190-8d65-4825-a681-4af09975ef3e","Type":"ContainerStarted","Data":"a40d6e0cd12ea990e86ee1a9281bfac7f1fd300382c8a0cb49a15547842429ae"} Jul 07 18:46:51.614787 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:51.614756 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5vxxf" event={"ID":"b8d04c06-cf5f-4fe2-a384-a0420d3e4157","Type":"ContainerStarted","Data":"6dbfb511f970f95faa1cea6eed1c6b04f5c9f21b7081f7d722332a64cf6e5307"} Jul 07 18:46:51.626654 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:51.626598 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-lqs4g" podStartSLOduration=2.245404946 podStartE2EDuration="5.626582211s" totalImagesPullingTime="3.381177265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:51.626154653 +0000 UTC m=+40.727173228" watchObservedRunningTime="2026-07-07 18:46:51.626582211 +0000 UTC m=+40.727600786" Jul 07 18:46:51.650193 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:51.650132 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-84cwc" podStartSLOduration=17.320866007 podStartE2EDuration="39.650114127s" totalImagesPullingTime="22.32924812s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:46:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:51.649382313 +0000 UTC m=+40.750400891" watchObservedRunningTime="2026-07-07 18:46:51.650114127 +0000 UTC m=+40.751132703" Jul 07 18:46:51.664933 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:51.664891 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5vxxf" podStartSLOduration=2.276650673 podStartE2EDuration="5.664875198s" totalImagesPullingTime="3.388224525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:51.664359651 +0000 UTC m=+40.765378227" watchObservedRunningTime="2026-07-07 18:46:51.664875198 +0000 UTC m=+40.765893750" Jul 07 18:46:51.881288 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:51.881187 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:51.978841 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:51.978806 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:52.698341 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:52.698305 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-t9wzd"] Jul 07 18:46:52.714868 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:52.714841 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:46:52.717377 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:52.717348 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8tq5g\"" Jul 07 18:46:52.717521 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:52.717463 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 18:46:52.717870 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:52.717852 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 18:46:52.729382 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:52.729356 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t9wzd"] Jul 07 18:46:52.780844 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:52.780762 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:52.863423 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:52.863392 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:52.901990 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:52.901954 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/67c94c15-f630-41c3-9d1d-33332df68075-data-volume\") pod \"insights-runtime-extractor-t9wzd\" (UID: \"67c94c15-f630-41c3-9d1d-33332df68075\") " pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:46:52.902154 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:52.901993 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9v9s\" (UniqueName: \"kubernetes.io/projected/67c94c15-f630-41c3-9d1d-33332df68075-kube-api-access-x9v9s\") pod \"insights-runtime-extractor-t9wzd\" (UID: \"67c94c15-f630-41c3-9d1d-33332df68075\") " pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:46:52.902154 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:52.902072 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/67c94c15-f630-41c3-9d1d-33332df68075-crio-socket\") pod \"insights-runtime-extractor-t9wzd\" (UID: \"67c94c15-f630-41c3-9d1d-33332df68075\") " pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:46:52.902154 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:52.902100 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67c94c15-f630-41c3-9d1d-33332df68075-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t9wzd\" (UID: \"67c94c15-f630-41c3-9d1d-33332df68075\") " pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:46:52.902331 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:52.902192 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67c94c15-f630-41c3-9d1d-33332df68075-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t9wzd\" (UID: \"67c94c15-f630-41c3-9d1d-33332df68075\") " pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:46:53.002718 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:53.002628 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67c94c15-f630-41c3-9d1d-33332df68075-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t9wzd\" (UID: \"67c94c15-f630-41c3-9d1d-33332df68075\") " pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:46:53.002847 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:53.002752 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/67c94c15-f630-41c3-9d1d-33332df68075-data-volume\") pod \"insights-runtime-extractor-t9wzd\" (UID: \"67c94c15-f630-41c3-9d1d-33332df68075\") " pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:46:53.002847 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:53.002754 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 18:46:53.002847 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:53.002771 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9v9s\" (UniqueName: \"kubernetes.io/projected/67c94c15-f630-41c3-9d1d-33332df68075-kube-api-access-x9v9s\") pod \"insights-runtime-extractor-t9wzd\" (UID: \"67c94c15-f630-41c3-9d1d-33332df68075\") " pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:46:53.002847 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:53.002800 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/67c94c15-f630-41c3-9d1d-33332df68075-crio-socket\") pod \"insights-runtime-extractor-t9wzd\" (UID: \"67c94c15-f630-41c3-9d1d-33332df68075\") " pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:46:53.002847 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:53.002832 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67c94c15-f630-41c3-9d1d-33332df68075-kube-rbac-proxy-cm podName:67c94c15-f630-41c3-9d1d-33332df68075 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:53.502810159 +0000 UTC m=+42.603828725 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67c94c15-f630-41c3-9d1d-33332df68075-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t9wzd" (UID: "67c94c15-f630-41c3-9d1d-33332df68075") : configmap "kube-rbac-proxy" not found Jul 07 18:46:53.003024 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:53.002856 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67c94c15-f630-41c3-9d1d-33332df68075-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t9wzd\" (UID: \"67c94c15-f630-41c3-9d1d-33332df68075\") " pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:46:53.003024 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:53.002911 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/67c94c15-f630-41c3-9d1d-33332df68075-crio-socket\") pod \"insights-runtime-extractor-t9wzd\" (UID: \"67c94c15-f630-41c3-9d1d-33332df68075\") " pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:46:53.003024 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:53.002976 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 18:46:53.003119 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:53.003027 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67c94c15-f630-41c3-9d1d-33332df68075-insights-runtime-extractor-tls podName:67c94c15-f630-41c3-9d1d-33332df68075 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:53.503013412 +0000 UTC m=+42.604031985 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67c94c15-f630-41c3-9d1d-33332df68075-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t9wzd" (UID: "67c94c15-f630-41c3-9d1d-33332df68075") : secret "insights-runtime-extractor-tls" not found Jul 07 18:46:53.003119 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:53.003104 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/67c94c15-f630-41c3-9d1d-33332df68075-data-volume\") pod \"insights-runtime-extractor-t9wzd\" (UID: \"67c94c15-f630-41c3-9d1d-33332df68075\") " pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:46:53.015172 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:53.015144 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9v9s\" (UniqueName: \"kubernetes.io/projected/67c94c15-f630-41c3-9d1d-33332df68075-kube-api-access-x9v9s\") pod \"insights-runtime-extractor-t9wzd\" (UID: \"67c94c15-f630-41c3-9d1d-33332df68075\") " pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:46:53.025732 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:53.025708 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:53.128581 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:53.128530 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:53.506638 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:53.506607 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67c94c15-f630-41c3-9d1d-33332df68075-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t9wzd\" (UID: \"67c94c15-f630-41c3-9d1d-33332df68075\") " pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:46:53.506779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:53.506662 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67c94c15-f630-41c3-9d1d-33332df68075-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t9wzd\" (UID: \"67c94c15-f630-41c3-9d1d-33332df68075\") " pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:46:53.506779 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:53.506743 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 18:46:53.506842 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:53.506774 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 18:46:53.506842 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:53.506792 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67c94c15-f630-41c3-9d1d-33332df68075-kube-rbac-proxy-cm podName:67c94c15-f630-41c3-9d1d-33332df68075 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:54.506778876 +0000 UTC m=+43.607797443 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67c94c15-f630-41c3-9d1d-33332df68075-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t9wzd" (UID: "67c94c15-f630-41c3-9d1d-33332df68075") : configmap "kube-rbac-proxy" not found Jul 07 18:46:53.506923 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:53.506903 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67c94c15-f630-41c3-9d1d-33332df68075-insights-runtime-extractor-tls podName:67c94c15-f630-41c3-9d1d-33332df68075 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:54.506887199 +0000 UTC m=+43.607905763 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67c94c15-f630-41c3-9d1d-33332df68075-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t9wzd" (UID: "67c94c15-f630-41c3-9d1d-33332df68075") : secret "insights-runtime-extractor-tls" not found Jul 07 18:46:53.621629 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:53.621557 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5x9w8_f390fdec-e425-4d23-a559-9681817515ec/console-operator/0.log" Jul 07 18:46:53.621629 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:53.621594 2583 generic.go:356] "Generic (PLEG): container finished" podID="f390fdec-e425-4d23-a559-9681817515ec" containerID="c0f8968132e9a684ef3fb27d0fd814757f656e2ddefd544ac4f2f7938707f659" exitCode=255 Jul 07 18:46:53.621777 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:53.621634 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-5x9w8" event={"ID":"f390fdec-e425-4d23-a559-9681817515ec","Type":"ContainerDied","Data":"c0f8968132e9a684ef3fb27d0fd814757f656e2ddefd544ac4f2f7938707f659"} Jul 07 18:46:53.621876 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:53.621862 2583 scope.go:117] "RemoveContainer" containerID="c0f8968132e9a684ef3fb27d0fd814757f656e2ddefd544ac4f2f7938707f659" Jul 07 18:46:54.113773 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:54.113743 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-tls\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:46:54.114191 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:54.113814 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3368babd-6ffc-4ec0-9dae-bcc00845b37f-metrics-tls\") pod \"dns-default-lk6f2\" (UID: \"3368babd-6ffc-4ec0-9dae-bcc00845b37f\") " pod="openshift-dns/dns-default-lk6f2" Jul 07 18:46:54.114191 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:54.113904 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 18:46:54.114191 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:54.113925 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69d6b476d6-9lkmx: secret "image-registry-tls" not found Jul 07 18:46:54.114191 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:54.113929 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 18:46:54.114191 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:54.113979 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3368babd-6ffc-4ec0-9dae-bcc00845b37f-metrics-tls podName:3368babd-6ffc-4ec0-9dae-bcc00845b37f nodeName:}" failed. No retries permitted until 2026-07-07 18:47:02.113965996 +0000 UTC m=+51.214984549 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3368babd-6ffc-4ec0-9dae-bcc00845b37f-metrics-tls") pod "dns-default-lk6f2" (UID: "3368babd-6ffc-4ec0-9dae-bcc00845b37f") : secret "dns-default-metrics-tls" not found Jul 07 18:46:54.114191 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:54.113992 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-tls podName:37675bf2-2454-4e3c-9c53-9f7ae3517b6a nodeName:}" failed. No retries permitted until 2026-07-07 18:47:02.11398633 +0000 UTC m=+51.215004883 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-tls") pod "image-registry-69d6b476d6-9lkmx" (UID: "37675bf2-2454-4e3c-9c53-9f7ae3517b6a") : secret "image-registry-tls" not found Jul 07 18:46:54.283004 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:54.282977 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:54.384873 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:54.384787 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:54.490219 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:54.490194 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ztg6h_8b9fb9da-073a-44c8-bf4d-2ee70fa748b9/dns-node-resolver/0.log" Jul 07 18:46:54.518604 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:54.518565 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67c94c15-f630-41c3-9d1d-33332df68075-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t9wzd\" (UID: \"67c94c15-f630-41c3-9d1d-33332df68075\") " pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:46:54.518836 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:54.518687 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 18:46:54.518836 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:54.518712 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/363d866e-b0a1-4351-8f85-b994a18200a7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-wszbl\" (UID: \"363d866e-b0a1-4351-8f85-b994a18200a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl" Jul 07 18:46:54.518836 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:54.518762 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67c94c15-f630-41c3-9d1d-33332df68075-kube-rbac-proxy-cm podName:67c94c15-f630-41c3-9d1d-33332df68075 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:56.518744587 +0000 UTC m=+45.619763148 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67c94c15-f630-41c3-9d1d-33332df68075-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t9wzd" (UID: "67c94c15-f630-41c3-9d1d-33332df68075") : configmap "kube-rbac-proxy" not found Jul 07 18:46:54.518836 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:54.518797 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67c94c15-f630-41c3-9d1d-33332df68075-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t9wzd\" (UID: \"67c94c15-f630-41c3-9d1d-33332df68075\") " pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:46:54.518979 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:54.518857 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 18:46:54.518979 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:54.518907 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 18:46:54.518979 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:54.518916 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/363d866e-b0a1-4351-8f85-b994a18200a7-cluster-monitoring-operator-tls podName:363d866e-b0a1-4351-8f85-b994a18200a7 nodeName:}" failed. No retries permitted until 2026-07-07 18:47:02.518901351 +0000 UTC m=+51.619919923 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/363d866e-b0a1-4351-8f85-b994a18200a7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-wszbl" (UID: "363d866e-b0a1-4351-8f85-b994a18200a7") : secret "cluster-monitoring-operator-tls" not found Jul 07 18:46:54.518979 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:54.518945 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67c94c15-f630-41c3-9d1d-33332df68075-insights-runtime-extractor-tls podName:67c94c15-f630-41c3-9d1d-33332df68075 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:56.518932657 +0000 UTC m=+45.619951210 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67c94c15-f630-41c3-9d1d-33332df68075-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t9wzd" (UID: "67c94c15-f630-41c3-9d1d-33332df68075") : secret "insights-runtime-extractor-tls" not found Jul 07 18:46:54.626507 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:54.626481 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5x9w8_f390fdec-e425-4d23-a559-9681817515ec/console-operator/1.log" Jul 07 18:46:54.626925 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:54.626907 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5x9w8_f390fdec-e425-4d23-a559-9681817515ec/console-operator/0.log" Jul 07 18:46:54.627027 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:54.626949 2583 generic.go:356] "Generic (PLEG): container finished" podID="f390fdec-e425-4d23-a559-9681817515ec" containerID="f48496f84e8787c551104f5dbcbbec779e8473f5646eb7615361427e7e9a79fa" exitCode=255 Jul 07 18:46:54.627027 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:54.626988 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-5x9w8" event={"ID":"f390fdec-e425-4d23-a559-9681817515ec","Type":"ContainerDied","Data":"f48496f84e8787c551104f5dbcbbec779e8473f5646eb7615361427e7e9a79fa"} Jul 07 18:46:54.627132 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:54.627090 2583 scope.go:117] "RemoveContainer" containerID="c0f8968132e9a684ef3fb27d0fd814757f656e2ddefd544ac4f2f7938707f659" Jul 07 18:46:54.627298 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:54.627283 2583 scope.go:117] "RemoveContainer" containerID="f48496f84e8787c551104f5dbcbbec779e8473f5646eb7615361427e7e9a79fa" Jul 07 18:46:54.627509 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:54.627485 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-5x9w8_openshift-console-operator(f390fdec-e425-4d23-a559-9681817515ec)\"" pod="openshift-console-operator/console-operator-68b757fb67-5x9w8" podUID="f390fdec-e425-4d23-a559-9681817515ec" Jul 07 18:46:55.091567 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:55.091539 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rwjnv_40b947e9-e3e6-43ec-8b88-ff421f627227/node-ca/0.log" Jul 07 18:46:55.427605 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:55.427522 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dccf34c-d46e-4122-81a2-2794860331a8-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-7zg9v\" (UID: \"3dccf34c-d46e-4122-81a2-2794860331a8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zg9v" Jul 07 18:46:55.428038 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:55.427689 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 18:46:55.428038 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:55.427785 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dccf34c-d46e-4122-81a2-2794860331a8-samples-operator-tls podName:3dccf34c-d46e-4122-81a2-2794860331a8 nodeName:}" failed. No retries permitted until 2026-07-07 18:47:03.42776261 +0000 UTC m=+52.528781163 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3dccf34c-d46e-4122-81a2-2794860331a8-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-7zg9v" (UID: "3dccf34c-d46e-4122-81a2-2794860331a8") : secret "samples-operator-tls" not found Jul 07 18:46:55.528953 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:55.528912 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-metrics-certs\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:46:55.529113 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:55.529061 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 18:46:55.529113 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:55.529087 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-service-ca-bundle\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:46:55.529184 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:55.529121 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-metrics-certs podName:7ea1005f-1048-4ec3-8fb7-e5ed794095a7 nodeName:}" failed. No retries permitted until 2026-07-07 18:47:03.529104402 +0000 UTC m=+52.630122980 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-metrics-certs") pod "router-default-86c68c69b7-p7g8g" (UID: "7ea1005f-1048-4ec3-8fb7-e5ed794095a7") : secret "router-metrics-certs-default" not found Jul 07 18:46:55.529222 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:55.529183 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-service-ca-bundle podName:7ea1005f-1048-4ec3-8fb7-e5ed794095a7 nodeName:}" failed. No retries permitted until 2026-07-07 18:47:03.529168716 +0000 UTC m=+52.630187280 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-service-ca-bundle") pod "router-default-86c68c69b7-p7g8g" (UID: "7ea1005f-1048-4ec3-8fb7-e5ed794095a7") : configmap references non-existent config key: service-ca.crt Jul 07 18:46:55.630553 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:55.630527 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5x9w8_f390fdec-e425-4d23-a559-9681817515ec/console-operator/1.log" Jul 07 18:46:55.630906 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:55.630885 2583 scope.go:117] "RemoveContainer" containerID="f48496f84e8787c551104f5dbcbbec779e8473f5646eb7615361427e7e9a79fa" Jul 07 18:46:55.631060 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:55.631041 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-5x9w8_openshift-console-operator(f390fdec-e425-4d23-a559-9681817515ec)\"" pod="openshift-console-operator/console-operator-68b757fb67-5x9w8" podUID="f390fdec-e425-4d23-a559-9681817515ec" Jul 07 18:46:55.690338 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:55.690270 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5vxxf_b8d04c06-cf5f-4fe2-a384-a0420d3e4157/serve-healthcheck-canary/0.log" Jul 07 18:46:55.868306 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:55.868242 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:56.536514 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:56.536469 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67c94c15-f630-41c3-9d1d-33332df68075-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t9wzd\" (UID: \"67c94c15-f630-41c3-9d1d-33332df68075\") " pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:46:56.537002 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:56.536533 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67c94c15-f630-41c3-9d1d-33332df68075-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t9wzd\" (UID: \"67c94c15-f630-41c3-9d1d-33332df68075\") " pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:46:56.537002 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:56.536606 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 18:46:56.537002 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:56.536670 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67c94c15-f630-41c3-9d1d-33332df68075-insights-runtime-extractor-tls podName:67c94c15-f630-41c3-9d1d-33332df68075 nodeName:}" failed. No retries permitted until 2026-07-07 18:47:00.536655106 +0000 UTC m=+49.637673673 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67c94c15-f630-41c3-9d1d-33332df68075-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t9wzd" (UID: "67c94c15-f630-41c3-9d1d-33332df68075") : secret "insights-runtime-extractor-tls" not found Jul 07 18:46:56.537002 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:56.536674 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 18:46:56.537002 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:56.536730 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67c94c15-f630-41c3-9d1d-33332df68075-kube-rbac-proxy-cm podName:67c94c15-f630-41c3-9d1d-33332df68075 nodeName:}" failed. No retries permitted until 2026-07-07 18:47:00.536714232 +0000 UTC m=+49.637732787 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67c94c15-f630-41c3-9d1d-33332df68075-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t9wzd" (UID: "67c94c15-f630-41c3-9d1d-33332df68075") : configmap "kube-rbac-proxy" not found Jul 07 18:46:56.908817 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:56.908783 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:56.982609 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:56.982549 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:57.900371 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:57.900336 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:57.999766 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:57.999735 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:58.099951 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:58.099920 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:58.942285 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:58.942239 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-5x9w8" Jul 07 18:46:58.942752 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:58.942314 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-5x9w8" Jul 07 18:46:58.942752 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:58.942740 2583 scope.go:117] "RemoveContainer" containerID="f48496f84e8787c551104f5dbcbbec779e8473f5646eb7615361427e7e9a79fa" Jul 07 18:46:58.942945 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:46:58.942923 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-5x9w8_openshift-console-operator(f390fdec-e425-4d23-a559-9681817515ec)\"" pod="openshift-console-operator/console-operator-68b757fb67-5x9w8" podUID="f390fdec-e425-4d23-a559-9681817515ec" Jul 07 18:46:59.820437 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:59.820406 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:46:59.876694 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:46:59.876660 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:00.257220 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:00.257133 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:00.362627 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:00.362590 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:00.529234 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:00.529156 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:00.570905 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:00.570867 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67c94c15-f630-41c3-9d1d-33332df68075-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t9wzd\" (UID: \"67c94c15-f630-41c3-9d1d-33332df68075\") " pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:47:00.571079 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:00.570938 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67c94c15-f630-41c3-9d1d-33332df68075-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t9wzd\" (UID: \"67c94c15-f630-41c3-9d1d-33332df68075\") " pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:47:00.571079 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:47:00.571017 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 18:47:00.571079 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:47:00.571025 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 18:47:00.571208 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:47:00.571084 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67c94c15-f630-41c3-9d1d-33332df68075-insights-runtime-extractor-tls podName:67c94c15-f630-41c3-9d1d-33332df68075 nodeName:}" failed. No retries permitted until 2026-07-07 18:47:08.571067377 +0000 UTC m=+57.672085933 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67c94c15-f630-41c3-9d1d-33332df68075-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t9wzd" (UID: "67c94c15-f630-41c3-9d1d-33332df68075") : secret "insights-runtime-extractor-tls" not found Jul 07 18:47:00.571208 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:47:00.571100 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67c94c15-f630-41c3-9d1d-33332df68075-kube-rbac-proxy-cm podName:67c94c15-f630-41c3-9d1d-33332df68075 nodeName:}" failed. No retries permitted until 2026-07-07 18:47:08.571093083 +0000 UTC m=+57.672111636 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67c94c15-f630-41c3-9d1d-33332df68075-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t9wzd" (UID: "67c94c15-f630-41c3-9d1d-33332df68075") : configmap "kube-rbac-proxy" not found Jul 07 18:47:00.629882 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:00.629851 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:02.158517 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:02.158485 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:02.185794 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:02.185750 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-tls\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:47:02.185953 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:02.185841 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3368babd-6ffc-4ec0-9dae-bcc00845b37f-metrics-tls\") pod \"dns-default-lk6f2\" (UID: \"3368babd-6ffc-4ec0-9dae-bcc00845b37f\") " pod="openshift-dns/dns-default-lk6f2" Jul 07 18:47:02.185953 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:47:02.185901 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 18:47:02.185953 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:47:02.185919 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69d6b476d6-9lkmx: secret "image-registry-tls" not found Jul 07 18:47:02.185953 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:47:02.185951 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 18:47:02.186133 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:47:02.185975 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-tls podName:37675bf2-2454-4e3c-9c53-9f7ae3517b6a nodeName:}" failed. No retries permitted until 2026-07-07 18:47:18.185957592 +0000 UTC m=+67.286976176 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-tls") pod "image-registry-69d6b476d6-9lkmx" (UID: "37675bf2-2454-4e3c-9c53-9f7ae3517b6a") : secret "image-registry-tls" not found Jul 07 18:47:02.186133 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:47:02.186005 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3368babd-6ffc-4ec0-9dae-bcc00845b37f-metrics-tls podName:3368babd-6ffc-4ec0-9dae-bcc00845b37f nodeName:}" failed. No retries permitted until 2026-07-07 18:47:18.185993632 +0000 UTC m=+67.287012185 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3368babd-6ffc-4ec0-9dae-bcc00845b37f-metrics-tls") pod "dns-default-lk6f2" (UID: "3368babd-6ffc-4ec0-9dae-bcc00845b37f") : secret "dns-default-metrics-tls" not found Jul 07 18:47:02.261991 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:02.261962 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:02.494396 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:02.494308 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:02.589214 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:02.589174 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/363d866e-b0a1-4351-8f85-b994a18200a7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-wszbl\" (UID: \"363d866e-b0a1-4351-8f85-b994a18200a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl" Jul 07 18:47:02.589406 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:47:02.589329 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 18:47:02.589448 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:47:02.589407 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/363d866e-b0a1-4351-8f85-b994a18200a7-cluster-monitoring-operator-tls podName:363d866e-b0a1-4351-8f85-b994a18200a7 nodeName:}" failed. No retries permitted until 2026-07-07 18:47:18.589391198 +0000 UTC m=+67.690409751 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/363d866e-b0a1-4351-8f85-b994a18200a7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-wszbl" (UID: "363d866e-b0a1-4351-8f85-b994a18200a7") : secret "cluster-monitoring-operator-tls" not found Jul 07 18:47:02.595693 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:02.595659 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:03.241725 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:03.241695 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:03.496131 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:03.496028 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dccf34c-d46e-4122-81a2-2794860331a8-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-7zg9v\" (UID: \"3dccf34c-d46e-4122-81a2-2794860331a8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zg9v" Jul 07 18:47:03.496339 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:47:03.496209 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 18:47:03.496339 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:47:03.496312 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dccf34c-d46e-4122-81a2-2794860331a8-samples-operator-tls podName:3dccf34c-d46e-4122-81a2-2794860331a8 nodeName:}" failed. No retries permitted until 2026-07-07 18:47:19.496291074 +0000 UTC m=+68.597309644 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3dccf34c-d46e-4122-81a2-2794860331a8-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-7zg9v" (UID: "3dccf34c-d46e-4122-81a2-2794860331a8") : secret "samples-operator-tls" not found Jul 07 18:47:03.596858 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:03.596815 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-metrics-certs\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:47:03.597055 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:03.596897 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-service-ca-bundle\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:47:03.597055 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:47:03.596996 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 18:47:03.597055 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:47:03.597020 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-service-ca-bundle podName:7ea1005f-1048-4ec3-8fb7-e5ed794095a7 nodeName:}" failed. No retries permitted until 2026-07-07 18:47:19.597003096 +0000 UTC m=+68.698021672 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-service-ca-bundle") pod "router-default-86c68c69b7-p7g8g" (UID: "7ea1005f-1048-4ec3-8fb7-e5ed794095a7") : configmap references non-existent config key: service-ca.crt Jul 07 18:47:03.597185 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:47:03.597061 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-metrics-certs podName:7ea1005f-1048-4ec3-8fb7-e5ed794095a7 nodeName:}" failed. No retries permitted until 2026-07-07 18:47:19.597044005 +0000 UTC m=+68.698062581 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-metrics-certs") pod "router-default-86c68c69b7-p7g8g" (UID: "7ea1005f-1048-4ec3-8fb7-e5ed794095a7") : secret "router-metrics-certs-default" not found Jul 07 18:47:04.557203 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:04.557168 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:04.658469 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:04.658435 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:04.815205 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:04.815127 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:04.944789 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:04.942108 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:05.020214 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:05.020178 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:05.436338 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:05.436302 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:05.542486 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:05.542456 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:06.264129 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:06.264094 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:06.459228 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:06.459196 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:06.857233 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:06.857193 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:06.946492 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:06.946459 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:07.056652 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:07.056623 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:07.286416 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:07.286382 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:07.450686 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:07.450596 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:08.106500 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:08.106469 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:08.245614 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:08.245583 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:08.385566 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:08.385465 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:08.487346 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:08.487310 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:08.643451 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:08.643357 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67c94c15-f630-41c3-9d1d-33332df68075-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t9wzd\" (UID: \"67c94c15-f630-41c3-9d1d-33332df68075\") " pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:47:08.643605 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:08.643498 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67c94c15-f630-41c3-9d1d-33332df68075-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t9wzd\" (UID: \"67c94c15-f630-41c3-9d1d-33332df68075\") " pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:47:08.643605 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:47:08.643601 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 18:47:08.643676 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:47:08.643659 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67c94c15-f630-41c3-9d1d-33332df68075-insights-runtime-extractor-tls podName:67c94c15-f630-41c3-9d1d-33332df68075 nodeName:}" failed. No retries permitted until 2026-07-07 18:47:24.64364299 +0000 UTC m=+73.744661562 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67c94c15-f630-41c3-9d1d-33332df68075-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t9wzd" (UID: "67c94c15-f630-41c3-9d1d-33332df68075") : secret "insights-runtime-extractor-tls" not found Jul 07 18:47:08.643928 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:08.643908 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67c94c15-f630-41c3-9d1d-33332df68075-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t9wzd\" (UID: \"67c94c15-f630-41c3-9d1d-33332df68075\") " pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:47:08.849840 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:08.849810 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:09.258276 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:09.258224 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:09.322829 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:09.322799 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:09.674899 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:09.674865 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:09.857701 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:09.857665 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:10.256554 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:10.256515 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:10.873392 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:10.873362 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:11.247684 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:11.247605 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:11.572506 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:11.572480 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9j7xz" Jul 07 18:47:11.653289 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:11.653238 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:11.847993 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:11.847906 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:11.949426 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:11.949388 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:12.052134 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:12.052103 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:12.133961 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:12.133892 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:12.322346 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:12.322314 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:12.409231 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:12.409162 2583 scope.go:117] "RemoveContainer" containerID="f48496f84e8787c551104f5dbcbbec779e8473f5646eb7615361427e7e9a79fa" Jul 07 18:47:12.672603 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:12.672517 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5x9w8_f390fdec-e425-4d23-a559-9681817515ec/console-operator/1.log" Jul 07 18:47:12.672603 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:12.672581 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-5x9w8" event={"ID":"f390fdec-e425-4d23-a559-9681817515ec","Type":"ContainerStarted","Data":"a980a91bea2b6cc1f47e9f887486e0e8b104d719e8f139c408ad2a6379c753e0"} Jul 07 18:47:12.672972 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:12.672949 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-5x9w8" Jul 07 18:47:12.693034 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:12.692977 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-5x9w8" podStartSLOduration=21.00274155 podStartE2EDuration="24.692960063s" totalImagesPullingTime="3.690218513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:47:12.691873657 +0000 UTC m=+61.792892233" watchObservedRunningTime="2026-07-07 18:47:12.692960063 +0000 UTC m=+61.793978651" Jul 07 18:47:12.852161 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:12.852131 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:13.249053 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:13.249022 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:13.352218 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:13.352191 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-5x9w8" Jul 07 18:47:13.552279 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:13.552181 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:13.653830 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:13.653785 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:13.735425 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:13.735394 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:13.851133 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:13.851103 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:13.951480 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:13.951453 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:14.052440 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:14.052410 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:14.153460 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:14.153378 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:14.253790 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:14.253757 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:14.357118 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:14.357085 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:14.455889 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:14.455810 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:14.866727 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:14.866684 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:15.061006 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:15.060967 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:15.270979 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:15.270896 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:15.391738 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:15.391704 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:15.493088 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:15.493055 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:15.640179 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:15.640147 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:15.740204 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:15.740161 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:17.119060 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:17.119020 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgl6b\" (UniqueName: \"kubernetes.io/projected/c7296533-de2d-44a3-8972-5e1b8baeaae2-kube-api-access-sgl6b\") pod \"network-check-target-l5d2z\" (UID: \"c7296533-de2d-44a3-8972-5e1b8baeaae2\") " pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:47:17.119456 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:17.119078 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cc80f14-06d5-483d-b15b-a44896b686a6-original-pull-secret\") pod \"global-pull-secret-syncer-c6dfn\" (UID: \"7cc80f14-06d5-483d-b15b-a44896b686a6\") " pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:47:17.119456 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:17.119121 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/329b4a4f-5489-4021-ab98-bcc7cc5b7dd0-metrics-certs\") pod \"network-metrics-daemon-cpp9l\" (UID: \"329b4a4f-5489-4021-ab98-bcc7cc5b7dd0\") " pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:47:17.121862 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:17.121840 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 18:47:17.122382 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:17.122365 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 18:47:17.122419 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:17.122365 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 18:47:17.132416 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:17.132390 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/329b4a4f-5489-4021-ab98-bcc7cc5b7dd0-metrics-certs\") pod \"network-metrics-daemon-cpp9l\" (UID: \"329b4a4f-5489-4021-ab98-bcc7cc5b7dd0\") " pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:47:17.132530 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:17.132463 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cc80f14-06d5-483d-b15b-a44896b686a6-original-pull-secret\") pod \"global-pull-secret-syncer-c6dfn\" (UID: \"7cc80f14-06d5-483d-b15b-a44896b686a6\") " pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:47:17.132710 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:17.132695 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 18:47:17.142708 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:17.142679 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgl6b\" (UniqueName: \"kubernetes.io/projected/c7296533-de2d-44a3-8972-5e1b8baeaae2-kube-api-access-sgl6b\") pod \"network-check-target-l5d2z\" (UID: \"c7296533-de2d-44a3-8972-5e1b8baeaae2\") " pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:47:17.319806 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:17.319752 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6dfn" Jul 07 18:47:17.328879 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:17.328854 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9vz2b\"" Jul 07 18:47:17.335483 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:17.335453 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8xhzt\"" Jul 07 18:47:17.337385 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:17.337364 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:47:17.343545 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:17.343514 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cpp9l" Jul 07 18:47:17.466751 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:17.466581 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-c6dfn"] Jul 07 18:47:17.470085 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:47:17.470059 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cc80f14_06d5_483d_b15b_a44896b686a6.slice/crio-ba4e2dd85b34f626776192c1a2fa9bfca00e880ec66542adaf1917675f964d2a WatchSource:0}: Error finding container ba4e2dd85b34f626776192c1a2fa9bfca00e880ec66542adaf1917675f964d2a: Status 404 returned error can't find the container with id ba4e2dd85b34f626776192c1a2fa9bfca00e880ec66542adaf1917675f964d2a Jul 07 18:47:17.484012 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:17.483988 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l5d2z"] Jul 07 18:47:17.487476 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:47:17.487451 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7296533_de2d_44a3_8972_5e1b8baeaae2.slice/crio-443b6bc5cfba3deebe8400576e888af2b4e90c57daf3952ed98bb85d851c8c2e WatchSource:0}: Error finding container 443b6bc5cfba3deebe8400576e888af2b4e90c57daf3952ed98bb85d851c8c2e: Status 404 returned error can't find the container with id 443b6bc5cfba3deebe8400576e888af2b4e90c57daf3952ed98bb85d851c8c2e Jul 07 18:47:17.505585 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:17.505555 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cpp9l"] Jul 07 18:47:17.508010 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:47:17.507987 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod329b4a4f_5489_4021_ab98_bcc7cc5b7dd0.slice/crio-b808ab0e1f497f3b238634f88a56b322b214ee351699bec24dd875caff0d43e1 WatchSource:0}: Error finding container b808ab0e1f497f3b238634f88a56b322b214ee351699bec24dd875caff0d43e1: Status 404 returned error can't find the container with id b808ab0e1f497f3b238634f88a56b322b214ee351699bec24dd875caff0d43e1 Jul 07 18:47:17.690886 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:17.690787 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cpp9l" event={"ID":"329b4a4f-5489-4021-ab98-bcc7cc5b7dd0","Type":"ContainerStarted","Data":"b808ab0e1f497f3b238634f88a56b322b214ee351699bec24dd875caff0d43e1"} Jul 07 18:47:17.691904 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:17.691883 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l5d2z" event={"ID":"c7296533-de2d-44a3-8972-5e1b8baeaae2","Type":"ContainerStarted","Data":"443b6bc5cfba3deebe8400576e888af2b4e90c57daf3952ed98bb85d851c8c2e"} Jul 07 18:47:17.692838 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:17.692819 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-c6dfn" event={"ID":"7cc80f14-06d5-483d-b15b-a44896b686a6","Type":"ContainerStarted","Data":"ba4e2dd85b34f626776192c1a2fa9bfca00e880ec66542adaf1917675f964d2a"} Jul 07 18:47:18.230102 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:18.230057 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-tls\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:47:18.230611 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:18.230165 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3368babd-6ffc-4ec0-9dae-bcc00845b37f-metrics-tls\") pod \"dns-default-lk6f2\" (UID: \"3368babd-6ffc-4ec0-9dae-bcc00845b37f\") " pod="openshift-dns/dns-default-lk6f2" Jul 07 18:47:18.233280 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:18.233186 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3368babd-6ffc-4ec0-9dae-bcc00845b37f-metrics-tls\") pod \"dns-default-lk6f2\" (UID: \"3368babd-6ffc-4ec0-9dae-bcc00845b37f\") " pod="openshift-dns/dns-default-lk6f2" Jul 07 18:47:18.234304 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:18.234276 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-tls\") pod \"image-registry-69d6b476d6-9lkmx\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:47:18.413465 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:18.413432 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4gs4q\"" Jul 07 18:47:18.421829 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:18.421585 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:47:18.437283 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:18.437068 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-x9x4r\"" Jul 07 18:47:18.445483 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:18.445148 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lk6f2" Jul 07 18:47:18.636803 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:18.636222 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/363d866e-b0a1-4351-8f85-b994a18200a7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-wszbl\" (UID: \"363d866e-b0a1-4351-8f85-b994a18200a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl" Jul 07 18:47:18.639489 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:18.639460 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/363d866e-b0a1-4351-8f85-b994a18200a7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-wszbl\" (UID: \"363d866e-b0a1-4351-8f85-b994a18200a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl" Jul 07 18:47:18.648721 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:18.648591 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69d6b476d6-9lkmx"] Jul 07 18:47:18.657579 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:47:18.657523 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37675bf2_2454_4e3c_9c53_9f7ae3517b6a.slice/crio-a584fa6a1c30592163cc2e960a008534a004dce84aa8d39670b7120b166d2d16 WatchSource:0}: Error finding container a584fa6a1c30592163cc2e960a008534a004dce84aa8d39670b7120b166d2d16: Status 404 returned error can't find the container with id a584fa6a1c30592163cc2e960a008534a004dce84aa8d39670b7120b166d2d16 Jul 07 18:47:18.699426 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:18.698604 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lk6f2"] Jul 07 18:47:18.699426 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:18.698843 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" event={"ID":"37675bf2-2454-4e3c-9c53-9f7ae3517b6a","Type":"ContainerStarted","Data":"a584fa6a1c30592163cc2e960a008534a004dce84aa8d39670b7120b166d2d16"} Jul 07 18:47:18.701987 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:18.701863 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cpp9l" event={"ID":"329b4a4f-5489-4021-ab98-bcc7cc5b7dd0","Type":"ContainerStarted","Data":"45ca36dc356d3ce75b40956bf85b4c158aaf8e2e3e3bdce06f166425b95facdf"} Jul 07 18:47:18.862891 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:18.862821 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-87mj9\"" Jul 07 18:47:18.871544 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:18.871495 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl" Jul 07 18:47:19.065336 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:19.065290 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl"] Jul 07 18:47:19.071142 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:47:19.071107 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod363d866e_b0a1_4351_8f85_b994a18200a7.slice/crio-e4a42e41d98edf2f4bb2ab4546566eeadc7b5f0e45f197161349e03a7c04f4dd WatchSource:0}: Error finding container e4a42e41d98edf2f4bb2ab4546566eeadc7b5f0e45f197161349e03a7c04f4dd: Status 404 returned error can't find the container with id e4a42e41d98edf2f4bb2ab4546566eeadc7b5f0e45f197161349e03a7c04f4dd Jul 07 18:47:19.544495 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:19.544404 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dccf34c-d46e-4122-81a2-2794860331a8-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-7zg9v\" (UID: \"3dccf34c-d46e-4122-81a2-2794860331a8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zg9v" Jul 07 18:47:19.549406 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:19.549368 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dccf34c-d46e-4122-81a2-2794860331a8-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-7zg9v\" (UID: \"3dccf34c-d46e-4122-81a2-2794860331a8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zg9v" Jul 07 18:47:19.646078 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:19.645952 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-metrics-certs\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:47:19.647153 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:19.646047 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-service-ca-bundle\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:47:19.647153 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:19.646887 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-service-ca-bundle\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:47:19.652990 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:19.652958 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ea1005f-1048-4ec3-8fb7-e5ed794095a7-metrics-certs\") pod \"router-default-86c68c69b7-p7g8g\" (UID: \"7ea1005f-1048-4ec3-8fb7-e5ed794095a7\") " pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:47:19.710737 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:19.710416 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" event={"ID":"37675bf2-2454-4e3c-9c53-9f7ae3517b6a","Type":"ContainerStarted","Data":"e2d189ee9afb63a61feb463f85017e7614b921cfe9c09d451802122761378169"} Jul 07 18:47:19.710737 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:19.710640 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:47:19.714617 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:19.714552 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cpp9l" event={"ID":"329b4a4f-5489-4021-ab98-bcc7cc5b7dd0","Type":"ContainerStarted","Data":"6b05ef0e47dd77e63888902570eefb985558261588c0b66c8268f3e65f12a224"} Jul 07 18:47:19.716128 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:19.716047 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl" event={"ID":"363d866e-b0a1-4351-8f85-b994a18200a7","Type":"ContainerStarted","Data":"e4a42e41d98edf2f4bb2ab4546566eeadc7b5f0e45f197161349e03a7c04f4dd"} Jul 07 18:47:19.717452 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:19.717345 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lk6f2" event={"ID":"3368babd-6ffc-4ec0-9dae-bcc00845b37f","Type":"ContainerStarted","Data":"dbc981c6da779105c37c87c36ab3a65fb1a4bbada0254bc28921b5238dc97557"} Jul 07 18:47:19.735635 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:19.735570 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" podStartSLOduration=67.735548998 podStartE2EDuration="1m7.735548998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:47:19.733619544 +0000 UTC m=+68.834638118" watchObservedRunningTime="2026-07-07 18:47:19.735548998 +0000 UTC m=+68.836567574" Jul 07 18:47:19.735962 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:19.735874 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-26cqm\"" Jul 07 18:47:19.744597 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:19.744560 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zg9v" Jul 07 18:47:19.751242 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:19.751182 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cpp9l" podStartSLOduration=66.71055921600001 podStartE2EDuration="1m7.751160369s" totalImagesPullingTime="1.040601153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:47:19.749796053 +0000 UTC m=+68.850814629" watchObservedRunningTime="2026-07-07 18:47:19.751160369 +0000 UTC m=+68.852178945" Jul 07 18:47:19.770025 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:19.769993 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-n9c8b\"" Jul 07 18:47:19.778342 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:19.778290 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:47:19.944905 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:19.943658 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zg9v"] Jul 07 18:47:21.725081 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:21.725040 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zg9v" event={"ID":"3dccf34c-d46e-4122-81a2-2794860331a8","Type":"ContainerStarted","Data":"1fe9a52e6affbf7101a74b7d91f7469c0a9a923da8535261a72598cb99177202"} Jul 07 18:47:22.826091 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:22.826044 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:22.937953 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:22.937904 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:23.041976 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:23.041938 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:23.140389 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:23.140323 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:24.241005 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.240930 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-86c68c69b7-p7g8g"] Jul 07 18:47:24.249047 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:47:24.249014 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ea1005f_1048_4ec3_8fb7_e5ed794095a7.slice/crio-fb6bfcae2574848effb83c347baddbb722f4ccb8f4b59a8d1d9bed3b0dd9a0a6 WatchSource:0}: Error finding container fb6bfcae2574848effb83c347baddbb722f4ccb8f4b59a8d1d9bed3b0dd9a0a6: Status 404 returned error can't find the container with id fb6bfcae2574848effb83c347baddbb722f4ccb8f4b59a8d1d9bed3b0dd9a0a6 Jul 07 18:47:24.472483 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.472442 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:24.581930 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.581901 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:24.673337 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.673304 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-mjtkj"] Jul 07 18:47:24.677679 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.677474 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-mjtkj" Jul 07 18:47:24.678848 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.678800 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:24.682362 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.682304 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-wmxsb\"" Jul 07 18:47:24.682503 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.682437 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 07 18:47:24.691427 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.691405 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-mjtkj"] Jul 07 18:47:24.694575 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.694537 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67c94c15-f630-41c3-9d1d-33332df68075-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t9wzd\" (UID: \"67c94c15-f630-41c3-9d1d-33332df68075\") " pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:47:24.697569 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.697542 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67c94c15-f630-41c3-9d1d-33332df68075-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t9wzd\" (UID: \"67c94c15-f630-41c3-9d1d-33332df68075\") " pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:47:24.740779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.740732 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-c6dfn" event={"ID":"7cc80f14-06d5-483d-b15b-a44896b686a6","Type":"ContainerStarted","Data":"fdff6b4570cf06699af430265e5173ea3c1a0f09768bd43418171c3025c3be8d"} Jul 07 18:47:24.742826 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.742787 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl" event={"ID":"363d866e-b0a1-4351-8f85-b994a18200a7","Type":"ContainerStarted","Data":"f9e0e0145f767319ec8f08dd48c95f332f8e639775fc76d6a0d95ea9fbd9b62b"} Jul 07 18:47:24.745437 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.745387 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-86c68c69b7-p7g8g" event={"ID":"7ea1005f-1048-4ec3-8fb7-e5ed794095a7","Type":"ContainerStarted","Data":"a720b7d25948a376b18a1d240951bfe412845f96520b815e6bcd4999f1ce2569"} Jul 07 18:47:24.745563 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.745456 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-86c68c69b7-p7g8g" event={"ID":"7ea1005f-1048-4ec3-8fb7-e5ed794095a7","Type":"ContainerStarted","Data":"fb6bfcae2574848effb83c347baddbb722f4ccb8f4b59a8d1d9bed3b0dd9a0a6"} Jul 07 18:47:24.748532 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.748507 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lk6f2" event={"ID":"3368babd-6ffc-4ec0-9dae-bcc00845b37f","Type":"ContainerStarted","Data":"6b013c302500612a461648dc3d8171505d9d54234174bfca9d3d2e642100dd5f"} Jul 07 18:47:24.748655 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.748552 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lk6f2" event={"ID":"3368babd-6ffc-4ec0-9dae-bcc00845b37f","Type":"ContainerStarted","Data":"75cc5d607b6e78673515968bd0e0eddfa8a3466b8504788d82e6bbb0f6736889"} Jul 07 18:47:24.748655 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.748621 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-lk6f2" Jul 07 18:47:24.750510 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.750487 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l5d2z" event={"ID":"c7296533-de2d-44a3-8972-5e1b8baeaae2","Type":"ContainerStarted","Data":"452f358a98388b38715e7fbde5c4ac5d53febeef9623bad83e0ca6f55adbe80f"} Jul 07 18:47:24.750647 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.750625 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:47:24.780510 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.780439 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:47:24.780704 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.780640 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-c6dfn" podStartSLOduration=66.19086061 podStartE2EDuration="1m12.780625836s" totalImagesPullingTime="6.589765226s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:46:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:47:24.780190546 +0000 UTC m=+73.881209122" watchObservedRunningTime="2026-07-07 18:47:24.780625836 +0000 UTC m=+73.881644405" Jul 07 18:47:24.783434 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.783396 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:47:24.794539 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.794485 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-l5d2z" podStartSLOduration=66.231360694 podStartE2EDuration="1m12.794467652s" totalImagesPullingTime="6.563106958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:47:24.793069917 +0000 UTC m=+73.894088491" watchObservedRunningTime="2026-07-07 18:47:24.794467652 +0000 UTC m=+73.895486227" Jul 07 18:47:24.795102 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.795071 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f5a4981e-bb5c-4860-8340-5a6c31c3ecfe-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-mjtkj\" (UID: \"f5a4981e-bb5c-4860-8340-5a6c31c3ecfe\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-mjtkj" Jul 07 18:47:24.800612 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.800576 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:24.808943 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.808869 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-lk6f2" podStartSLOduration=33.470067166 podStartE2EDuration="38.80885122s" totalImagesPullingTime="5.338784054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:47:24.808095688 +0000 UTC m=+73.909114307" watchObservedRunningTime="2026-07-07 18:47:24.80885122 +0000 UTC m=+73.909869797" Jul 07 18:47:24.824241 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.824186 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl" podStartSLOduration=33.836527706 podStartE2EDuration="38.824167163s" totalImagesPullingTime="4.987639457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:47:24.823887652 +0000 UTC m=+73.924906227" watchObservedRunningTime="2026-07-07 18:47:24.824167163 +0000 UTC m=+73.925185729" Jul 07 18:47:24.825410 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.825390 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8tq5g\"" Jul 07 18:47:24.834384 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.834307 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t9wzd" Jul 07 18:47:24.842386 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.842343 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-86c68c69b7-p7g8g" podStartSLOduration=38.842328371 podStartE2EDuration="38.842328371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:47:24.840881696 +0000 UTC m=+73.941900275" watchObservedRunningTime="2026-07-07 18:47:24.842328371 +0000 UTC m=+73.943346946" Jul 07 18:47:24.896135 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.896100 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f5a4981e-bb5c-4860-8340-5a6c31c3ecfe-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-mjtkj\" (UID: \"f5a4981e-bb5c-4860-8340-5a6c31c3ecfe\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-mjtkj" Jul 07 18:47:24.900526 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.900499 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f5a4981e-bb5c-4860-8340-5a6c31c3ecfe-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-mjtkj\" (UID: \"f5a4981e-bb5c-4860-8340-5a6c31c3ecfe\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-mjtkj" Jul 07 18:47:24.988131 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:24.988096 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-mjtkj" Jul 07 18:47:25.286816 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.286780 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t9wzd"] Jul 07 18:47:25.289204 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:47:25.289177 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67c94c15_f630_41c3_9d1d_33332df68075.slice/crio-120b4409e2d6fedfaaaf4d08a9b86ad6d405316d79048f298be41c03a9330570 WatchSource:0}: Error finding container 120b4409e2d6fedfaaaf4d08a9b86ad6d405316d79048f298be41c03a9330570: Status 404 returned error can't find the container with id 120b4409e2d6fedfaaaf4d08a9b86ad6d405316d79048f298be41c03a9330570 Jul 07 18:47:25.302097 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.302060 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-mjtkj"] Jul 07 18:47:25.305580 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:47:25.305556 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5a4981e_bb5c_4860_8340_5a6c31c3ecfe.slice/crio-49bda8ff76bd8eace88517c98a47e09d767cca6a2cd6b220b29c789c1bd70397 WatchSource:0}: Error finding container 49bda8ff76bd8eace88517c98a47e09d767cca6a2cd6b220b29c789c1bd70397: Status 404 returned error can't find the container with id 49bda8ff76bd8eace88517c98a47e09d767cca6a2cd6b220b29c789c1bd70397 Jul 07 18:47:25.665718 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.665684 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:25.684420 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.684389 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77b65d74fd-4flgm"] Jul 07 18:47:25.687458 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.687436 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b65d74fd-4flgm" Jul 07 18:47:25.690051 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.690012 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 18:47:25.690604 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.690579 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 18:47:25.690712 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.690605 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 18:47:25.690712 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.690638 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 18:47:25.690712 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.690644 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-cjnth\"" Jul 07 18:47:25.691898 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.691876 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 18:47:25.692017 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.691937 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 18:47:25.692017 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.691956 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 18:47:25.699649 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.699624 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77b65d74fd-4flgm"] Jul 07 18:47:25.756453 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.756375 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t9wzd" event={"ID":"67c94c15-f630-41c3-9d1d-33332df68075","Type":"ContainerStarted","Data":"120b4409e2d6fedfaaaf4d08a9b86ad6d405316d79048f298be41c03a9330570"} Jul 07 18:47:25.758473 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.758443 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zg9v" event={"ID":"3dccf34c-d46e-4122-81a2-2794860331a8","Type":"ContainerStarted","Data":"84e4fdacac8f20a594a303b0ed58ed14fa1abc90b590645cace1e8bd96f9b18b"} Jul 07 18:47:25.758614 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.758496 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zg9v" event={"ID":"3dccf34c-d46e-4122-81a2-2794860331a8","Type":"ContainerStarted","Data":"dfb17fd186982bb058070a7035722a4a16dc96b384d89bb7f487f2d21d4d7449"} Jul 07 18:47:25.760269 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.760198 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-mjtkj" event={"ID":"f5a4981e-bb5c-4860-8340-5a6c31c3ecfe","Type":"ContainerStarted","Data":"49bda8ff76bd8eace88517c98a47e09d767cca6a2cd6b220b29c789c1bd70397"} Jul 07 18:47:25.760955 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.760910 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:47:25.761579 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.761554 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:25.762326 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.762308 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-86c68c69b7-p7g8g" Jul 07 18:47:25.776722 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.776668 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zg9v" podStartSLOduration=34.734307341 podStartE2EDuration="38.776649106s" totalImagesPullingTime="4.042341765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:47:25.775730355 +0000 UTC m=+74.876748930" watchObservedRunningTime="2026-07-07 18:47:25.776649106 +0000 UTC m=+74.877667682" Jul 07 18:47:25.805898 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.805720 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6994cdae-7247-4e9c-8c95-5d409cea63bd-console-oauth-config\") pod \"console-77b65d74fd-4flgm\" (UID: \"6994cdae-7247-4e9c-8c95-5d409cea63bd\") " pod="openshift-console/console-77b65d74fd-4flgm" Jul 07 18:47:25.805898 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.805766 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6994cdae-7247-4e9c-8c95-5d409cea63bd-service-ca\") pod \"console-77b65d74fd-4flgm\" (UID: \"6994cdae-7247-4e9c-8c95-5d409cea63bd\") " pod="openshift-console/console-77b65d74fd-4flgm" Jul 07 18:47:25.805898 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.805814 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvcwt\" (UniqueName: \"kubernetes.io/projected/6994cdae-7247-4e9c-8c95-5d409cea63bd-kube-api-access-tvcwt\") pod \"console-77b65d74fd-4flgm\" (UID: \"6994cdae-7247-4e9c-8c95-5d409cea63bd\") " pod="openshift-console/console-77b65d74fd-4flgm" Jul 07 18:47:25.805898 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.805847 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6994cdae-7247-4e9c-8c95-5d409cea63bd-oauth-serving-cert\") pod \"console-77b65d74fd-4flgm\" (UID: \"6994cdae-7247-4e9c-8c95-5d409cea63bd\") " pod="openshift-console/console-77b65d74fd-4flgm" Jul 07 18:47:25.806243 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.805917 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6994cdae-7247-4e9c-8c95-5d409cea63bd-console-config\") pod \"console-77b65d74fd-4flgm\" (UID: \"6994cdae-7247-4e9c-8c95-5d409cea63bd\") " pod="openshift-console/console-77b65d74fd-4flgm" Jul 07 18:47:25.806243 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.805958 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6994cdae-7247-4e9c-8c95-5d409cea63bd-console-serving-cert\") pod \"console-77b65d74fd-4flgm\" (UID: \"6994cdae-7247-4e9c-8c95-5d409cea63bd\") " pod="openshift-console/console-77b65d74fd-4flgm" Jul 07 18:47:25.907219 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.907186 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6994cdae-7247-4e9c-8c95-5d409cea63bd-console-oauth-config\") pod \"console-77b65d74fd-4flgm\" (UID: \"6994cdae-7247-4e9c-8c95-5d409cea63bd\") " pod="openshift-console/console-77b65d74fd-4flgm" Jul 07 18:47:25.907398 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.907228 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6994cdae-7247-4e9c-8c95-5d409cea63bd-service-ca\") pod \"console-77b65d74fd-4flgm\" (UID: \"6994cdae-7247-4e9c-8c95-5d409cea63bd\") " pod="openshift-console/console-77b65d74fd-4flgm" Jul 07 18:47:25.908486 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.908160 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6994cdae-7247-4e9c-8c95-5d409cea63bd-service-ca\") pod \"console-77b65d74fd-4flgm\" (UID: \"6994cdae-7247-4e9c-8c95-5d409cea63bd\") " pod="openshift-console/console-77b65d74fd-4flgm" Jul 07 18:47:25.912288 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.908918 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvcwt\" (UniqueName: \"kubernetes.io/projected/6994cdae-7247-4e9c-8c95-5d409cea63bd-kube-api-access-tvcwt\") pod \"console-77b65d74fd-4flgm\" (UID: \"6994cdae-7247-4e9c-8c95-5d409cea63bd\") " pod="openshift-console/console-77b65d74fd-4flgm" Jul 07 18:47:25.912288 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.908976 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6994cdae-7247-4e9c-8c95-5d409cea63bd-oauth-serving-cert\") pod \"console-77b65d74fd-4flgm\" (UID: \"6994cdae-7247-4e9c-8c95-5d409cea63bd\") " pod="openshift-console/console-77b65d74fd-4flgm" Jul 07 18:47:25.912288 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.909068 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6994cdae-7247-4e9c-8c95-5d409cea63bd-console-config\") pod \"console-77b65d74fd-4flgm\" (UID: \"6994cdae-7247-4e9c-8c95-5d409cea63bd\") " pod="openshift-console/console-77b65d74fd-4flgm" Jul 07 18:47:25.912288 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.909117 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6994cdae-7247-4e9c-8c95-5d409cea63bd-console-serving-cert\") pod \"console-77b65d74fd-4flgm\" (UID: \"6994cdae-7247-4e9c-8c95-5d409cea63bd\") " pod="openshift-console/console-77b65d74fd-4flgm" Jul 07 18:47:25.912288 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.909996 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6994cdae-7247-4e9c-8c95-5d409cea63bd-oauth-serving-cert\") pod \"console-77b65d74fd-4flgm\" (UID: \"6994cdae-7247-4e9c-8c95-5d409cea63bd\") " pod="openshift-console/console-77b65d74fd-4flgm" Jul 07 18:47:25.912288 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.911546 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6994cdae-7247-4e9c-8c95-5d409cea63bd-console-config\") pod \"console-77b65d74fd-4flgm\" (UID: \"6994cdae-7247-4e9c-8c95-5d409cea63bd\") " pod="openshift-console/console-77b65d74fd-4flgm" Jul 07 18:47:25.916594 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.916517 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6994cdae-7247-4e9c-8c95-5d409cea63bd-console-oauth-config\") pod \"console-77b65d74fd-4flgm\" (UID: \"6994cdae-7247-4e9c-8c95-5d409cea63bd\") " pod="openshift-console/console-77b65d74fd-4flgm" Jul 07 18:47:25.918488 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.918465 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6994cdae-7247-4e9c-8c95-5d409cea63bd-console-serving-cert\") pod \"console-77b65d74fd-4flgm\" (UID: \"6994cdae-7247-4e9c-8c95-5d409cea63bd\") " pod="openshift-console/console-77b65d74fd-4flgm" Jul 07 18:47:25.920637 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.920588 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvcwt\" (UniqueName: \"kubernetes.io/projected/6994cdae-7247-4e9c-8c95-5d409cea63bd-kube-api-access-tvcwt\") pod \"console-77b65d74fd-4flgm\" (UID: \"6994cdae-7247-4e9c-8c95-5d409cea63bd\") " pod="openshift-console/console-77b65d74fd-4flgm" Jul 07 18:47:25.998035 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:25.997784 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b65d74fd-4flgm" Jul 07 18:47:26.166951 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:26.166875 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77b65d74fd-4flgm"] Jul 07 18:47:26.770102 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:26.770064 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-mjtkj" event={"ID":"f5a4981e-bb5c-4860-8340-5a6c31c3ecfe","Type":"ContainerStarted","Data":"739a858c94bc554303ddcf18a1f90ed9cd58b6603c2340a0027c0c64c0d4ca4d"} Jul 07 18:47:26.771025 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:26.771001 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-mjtkj" Jul 07 18:47:26.773427 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:26.771858 2583 patch_prober.go:28] interesting pod/prometheus-operator-admission-webhook-d559d66d5-mjtkj container/prometheus-operator-admission-webhook namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.132.0.16:8443/healthz\": dial tcp 10.132.0.16:8443: connect: connection refused" start-of-body= Jul 07 18:47:26.773556 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:26.773481 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-mjtkj" podUID="f5a4981e-bb5c-4860-8340-5a6c31c3ecfe" containerName="prometheus-operator-admission-webhook" probeResult="failure" output="Get \"https://10.132.0.16:8443/healthz\": dial tcp 10.132.0.16:8443: connect: connection refused" Jul 07 18:47:26.775369 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:26.775334 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b65d74fd-4flgm" event={"ID":"6994cdae-7247-4e9c-8c95-5d409cea63bd","Type":"ContainerStarted","Data":"2a80429a0f295c75aa97ae9843ec9238349ad51dc4ee68077a7629c06a41a558"} Jul 07 18:47:26.794924 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:26.794830 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-mjtkj" podStartSLOduration=1.423699435 podStartE2EDuration="2.794810376s" totalImagesPullingTime="1.371110941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:47:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:47:26.792997702 +0000 UTC m=+75.894016279" watchObservedRunningTime="2026-07-07 18:47:26.794810376 +0000 UTC m=+75.895828951" Jul 07 18:47:26.905310 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:26.905266 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:27.007886 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:27.007806 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:27.509380 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:27.509348 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:27.619049 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:27.618977 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:27.784991 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:27.784902 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-mjtkj" Jul 07 18:47:27.935850 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:27.935816 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:28.041795 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:28.041425 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:28.692594 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:28.692550 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:28.735988 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:28.735954 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-j2p4h"] Jul 07 18:47:28.739548 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:28.739522 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-j2p4h" Jul 07 18:47:28.742115 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:28.742089 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 07 18:47:28.742573 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:28.742546 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 18:47:28.742903 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:28.742878 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-p94sj\"" Jul 07 18:47:28.743049 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:28.742950 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 07 18:47:28.749130 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:28.749096 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-j2p4h"] Jul 07 18:47:28.766924 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:28.766885 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:28.835356 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:28.835317 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d33cb992-3e55-4a9d-8405-2aff78a850aa-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-j2p4h\" (UID: \"d33cb992-3e55-4a9d-8405-2aff78a850aa\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-j2p4h" Jul 07 18:47:28.835820 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:28.835371 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf29n\" (UniqueName: \"kubernetes.io/projected/d33cb992-3e55-4a9d-8405-2aff78a850aa-kube-api-access-zf29n\") pod \"prometheus-operator-567f7b8b7c-j2p4h\" (UID: \"d33cb992-3e55-4a9d-8405-2aff78a850aa\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-j2p4h" Jul 07 18:47:28.835820 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:28.835394 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d33cb992-3e55-4a9d-8405-2aff78a850aa-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-j2p4h\" (UID: \"d33cb992-3e55-4a9d-8405-2aff78a850aa\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-j2p4h" Jul 07 18:47:28.835820 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:28.835422 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d33cb992-3e55-4a9d-8405-2aff78a850aa-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-j2p4h\" (UID: \"d33cb992-3e55-4a9d-8405-2aff78a850aa\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-j2p4h" Jul 07 18:47:28.936674 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:28.936637 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d33cb992-3e55-4a9d-8405-2aff78a850aa-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-j2p4h\" (UID: \"d33cb992-3e55-4a9d-8405-2aff78a850aa\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-j2p4h" Jul 07 18:47:28.936843 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:28.936706 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf29n\" (UniqueName: \"kubernetes.io/projected/d33cb992-3e55-4a9d-8405-2aff78a850aa-kube-api-access-zf29n\") pod \"prometheus-operator-567f7b8b7c-j2p4h\" (UID: \"d33cb992-3e55-4a9d-8405-2aff78a850aa\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-j2p4h" Jul 07 18:47:28.936843 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:28.936740 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d33cb992-3e55-4a9d-8405-2aff78a850aa-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-j2p4h\" (UID: \"d33cb992-3e55-4a9d-8405-2aff78a850aa\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-j2p4h" Jul 07 18:47:28.936843 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:28.936765 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d33cb992-3e55-4a9d-8405-2aff78a850aa-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-j2p4h\" (UID: \"d33cb992-3e55-4a9d-8405-2aff78a850aa\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-j2p4h" Jul 07 18:47:28.937025 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:47:28.936915 2583 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 07 18:47:28.937025 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:47:28.936989 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d33cb992-3e55-4a9d-8405-2aff78a850aa-prometheus-operator-tls podName:d33cb992-3e55-4a9d-8405-2aff78a850aa nodeName:}" failed. No retries permitted until 2026-07-07 18:47:29.436967305 +0000 UTC m=+78.537985875 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/d33cb992-3e55-4a9d-8405-2aff78a850aa-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-j2p4h" (UID: "d33cb992-3e55-4a9d-8405-2aff78a850aa") : secret "prometheus-operator-tls" not found Jul 07 18:47:28.937515 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:28.937492 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d33cb992-3e55-4a9d-8405-2aff78a850aa-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-j2p4h\" (UID: \"d33cb992-3e55-4a9d-8405-2aff78a850aa\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-j2p4h" Jul 07 18:47:28.939632 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:28.939611 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d33cb992-3e55-4a9d-8405-2aff78a850aa-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-j2p4h\" (UID: \"d33cb992-3e55-4a9d-8405-2aff78a850aa\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-j2p4h" Jul 07 18:47:28.945490 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:28.945432 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf29n\" (UniqueName: \"kubernetes.io/projected/d33cb992-3e55-4a9d-8405-2aff78a850aa-kube-api-access-zf29n\") pod \"prometheus-operator-567f7b8b7c-j2p4h\" (UID: \"d33cb992-3e55-4a9d-8405-2aff78a850aa\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-j2p4h" Jul 07 18:47:29.427180 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:29.427137 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:29.443013 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:29.442982 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d33cb992-3e55-4a9d-8405-2aff78a850aa-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-j2p4h\" (UID: \"d33cb992-3e55-4a9d-8405-2aff78a850aa\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-j2p4h" Jul 07 18:47:29.445578 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:29.445553 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d33cb992-3e55-4a9d-8405-2aff78a850aa-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-j2p4h\" (UID: \"d33cb992-3e55-4a9d-8405-2aff78a850aa\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-j2p4h" Jul 07 18:47:29.650867 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:29.650834 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-j2p4h" Jul 07 18:47:29.689183 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:29.689097 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:29.754305 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:29.754241 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:30.538935 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:30.538901 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-j2p4h"] Jul 07 18:47:30.543916 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:47:30.543868 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd33cb992_3e55_4a9d_8405_2aff78a850aa.slice/crio-229a352592501528dac1fea456869c59c94f255fbd523949907d4d370f5116cc WatchSource:0}: Error finding container 229a352592501528dac1fea456869c59c94f255fbd523949907d4d370f5116cc: Status 404 returned error can't find the container with id 229a352592501528dac1fea456869c59c94f255fbd523949907d4d370f5116cc Jul 07 18:47:30.791732 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:30.791633 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b65d74fd-4flgm" event={"ID":"6994cdae-7247-4e9c-8c95-5d409cea63bd","Type":"ContainerStarted","Data":"0818e8f08cf827c1117958c3776f2e77a9c4a11945eb61d5b55d6ea27a125a5e"} Jul 07 18:47:30.792844 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:30.792815 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-j2p4h" event={"ID":"d33cb992-3e55-4a9d-8405-2aff78a850aa","Type":"ContainerStarted","Data":"229a352592501528dac1fea456869c59c94f255fbd523949907d4d370f5116cc"} Jul 07 18:47:30.794072 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:30.794049 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t9wzd" event={"ID":"67c94c15-f630-41c3-9d1d-33332df68075","Type":"ContainerStarted","Data":"9a36b7a28c7a62a4b7ca350cbe5cbe70e5a951cbd3736703a4fd61e87eaf70f3"} Jul 07 18:47:30.809711 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:30.809661 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77b65d74fd-4flgm" podStartSLOduration=1.583912781 podStartE2EDuration="5.809643635s" totalImagesPullingTime="4.225730854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:47:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:47:30.808506991 +0000 UTC m=+79.909525565" watchObservedRunningTime="2026-07-07 18:47:30.809643635 +0000 UTC m=+79.910662210" Jul 07 18:47:31.801092 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:31.801054 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t9wzd" event={"ID":"67c94c15-f630-41c3-9d1d-33332df68075","Type":"ContainerStarted","Data":"602eb0daefdb8a280d6411c53a0053ce6ca4b6ccd2721c4aa68366524ba235fd"} Jul 07 18:47:32.246089 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:32.246056 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:32.600961 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:32.600915 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:32.667217 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:32.667185 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:32.807652 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:32.807613 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-j2p4h" event={"ID":"d33cb992-3e55-4a9d-8405-2aff78a850aa","Type":"ContainerStarted","Data":"885233b7688d31e580ddc762b4d797cac4a16abb2b6b28103a417fd56bedb512"} Jul 07 18:47:32.808119 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:32.807679 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-j2p4h" event={"ID":"d33cb992-3e55-4a9d-8405-2aff78a850aa","Type":"ContainerStarted","Data":"247ff2a46023e13d8cde37615f5953f46c473daed7a99631086bf3d5ddcec8ea"} Jul 07 18:47:32.824777 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:32.824707 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-j2p4h" podStartSLOduration=3.4481141109999998 podStartE2EDuration="4.824684382s" totalImagesPullingTime="1.376570271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:47:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:47:32.823941547 +0000 UTC m=+81.924960123" watchObservedRunningTime="2026-07-07 18:47:32.824684382 +0000 UTC m=+81.925702978" Jul 07 18:47:33.820202 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:33.820159 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t9wzd" event={"ID":"67c94c15-f630-41c3-9d1d-33332df68075","Type":"ContainerStarted","Data":"56d3c5183636b5e26b560a212f9f8d63e6ac70def585031fbb2f7fe628cf59d4"} Jul 07 18:47:33.839141 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:33.839090 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-t9wzd" podStartSLOduration=33.608919325 podStartE2EDuration="41.839073285s" totalImagesPullingTime="8.23015396s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:46:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:47:33.837435727 +0000 UTC m=+82.938454303" watchObservedRunningTime="2026-07-07 18:47:33.839073285 +0000 UTC m=+82.940091861" Jul 07 18:47:34.462153 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:34.462112 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:34.518140 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:34.518111 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:34.763061 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:34.762969 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-lk6f2" Jul 07 18:47:34.843911 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:34.843867 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:35.037224 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.037137 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:35.101415 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.101379 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-rgvt7"] Jul 07 18:47:35.104671 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.104654 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-rgvt7" Jul 07 18:47:35.108768 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.108749 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 07 18:47:35.109092 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.109077 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-rvlvq\"" Jul 07 18:47:35.109225 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.109210 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 07 18:47:35.116122 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.116102 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-m9skr"] Jul 07 18:47:35.119197 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.119168 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.120345 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.120326 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-rgvt7"] Jul 07 18:47:35.121877 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.121854 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 18:47:35.121953 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.121905 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 18:47:35.122218 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.122201 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 18:47:35.122404 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.122389 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-q8t9z\"" Jul 07 18:47:35.155830 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.155794 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:35.189407 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.189376 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1d6d0aa-b29e-49f7-b83f-df34ef4a027a-metrics-client-ca\") pod \"node-exporter-m9skr\" (UID: \"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a\") " pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.189407 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.189411 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bf91236-1a18-4683-867f-88f7a67cba50-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-rgvt7\" (UID: \"2bf91236-1a18-4683-867f-88f7a67cba50\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-rgvt7" Jul 07 18:47:35.189600 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.189446 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c1d6d0aa-b29e-49f7-b83f-df34ef4a027a-sys\") pod \"node-exporter-m9skr\" (UID: \"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a\") " pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.189600 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.189500 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c1d6d0aa-b29e-49f7-b83f-df34ef4a027a-root\") pod \"node-exporter-m9skr\" (UID: \"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a\") " pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.189600 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.189540 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mdfx\" (UniqueName: \"kubernetes.io/projected/c1d6d0aa-b29e-49f7-b83f-df34ef4a027a-kube-api-access-5mdfx\") pod \"node-exporter-m9skr\" (UID: \"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a\") " pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.189600 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.189570 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2bf91236-1a18-4683-867f-88f7a67cba50-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-rgvt7\" (UID: \"2bf91236-1a18-4683-867f-88f7a67cba50\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-rgvt7" Jul 07 18:47:35.189600 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.189594 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c1d6d0aa-b29e-49f7-b83f-df34ef4a027a-node-exporter-textfile\") pod \"node-exporter-m9skr\" (UID: \"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a\") " pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.189974 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.189612 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c1d6d0aa-b29e-49f7-b83f-df34ef4a027a-node-exporter-tls\") pod \"node-exporter-m9skr\" (UID: \"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a\") " pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.189974 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.189630 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bf91236-1a18-4683-867f-88f7a67cba50-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-rgvt7\" (UID: \"2bf91236-1a18-4683-867f-88f7a67cba50\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-rgvt7" Jul 07 18:47:35.189974 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.189673 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1d6d0aa-b29e-49f7-b83f-df34ef4a027a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m9skr\" (UID: \"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a\") " pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.189974 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.189708 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ln6sz\" (UniqueName: \"kubernetes.io/projected/2bf91236-1a18-4683-867f-88f7a67cba50-kube-api-access-ln6sz\") pod \"openshift-state-metrics-54cf96dfcf-rgvt7\" (UID: \"2bf91236-1a18-4683-867f-88f7a67cba50\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-rgvt7" Jul 07 18:47:35.189974 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.189734 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c1d6d0aa-b29e-49f7-b83f-df34ef4a027a-node-exporter-accelerators-collector-config\") pod \"node-exporter-m9skr\" (UID: \"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a\") " pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.189974 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.189758 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c1d6d0aa-b29e-49f7-b83f-df34ef4a027a-node-exporter-wtmp\") pod \"node-exporter-m9skr\" (UID: \"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a\") " pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.276436 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.276396 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:35.290528 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.290466 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c1d6d0aa-b29e-49f7-b83f-df34ef4a027a-node-exporter-accelerators-collector-config\") pod \"node-exporter-m9skr\" (UID: \"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a\") " pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.290528 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.290506 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c1d6d0aa-b29e-49f7-b83f-df34ef4a027a-node-exporter-wtmp\") pod \"node-exporter-m9skr\" (UID: \"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a\") " pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.290663 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.290528 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1d6d0aa-b29e-49f7-b83f-df34ef4a027a-metrics-client-ca\") pod \"node-exporter-m9skr\" (UID: \"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a\") " pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.290663 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.290555 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bf91236-1a18-4683-867f-88f7a67cba50-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-rgvt7\" (UID: \"2bf91236-1a18-4683-867f-88f7a67cba50\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-rgvt7" Jul 07 18:47:35.290663 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.290597 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c1d6d0aa-b29e-49f7-b83f-df34ef4a027a-sys\") pod \"node-exporter-m9skr\" (UID: \"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a\") " pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.290663 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.290623 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c1d6d0aa-b29e-49f7-b83f-df34ef4a027a-root\") pod \"node-exporter-m9skr\" (UID: \"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a\") " pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.290663 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.290660 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mdfx\" (UniqueName: \"kubernetes.io/projected/c1d6d0aa-b29e-49f7-b83f-df34ef4a027a-kube-api-access-5mdfx\") pod \"node-exporter-m9skr\" (UID: \"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a\") " pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.290902 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.290664 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c1d6d0aa-b29e-49f7-b83f-df34ef4a027a-node-exporter-wtmp\") pod \"node-exporter-m9skr\" (UID: \"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a\") " pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.290902 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.290703 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c1d6d0aa-b29e-49f7-b83f-df34ef4a027a-sys\") pod \"node-exporter-m9skr\" (UID: \"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a\") " pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.290902 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.290741 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c1d6d0aa-b29e-49f7-b83f-df34ef4a027a-root\") pod \"node-exporter-m9skr\" (UID: \"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a\") " pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.290902 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.290777 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2bf91236-1a18-4683-867f-88f7a67cba50-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-rgvt7\" (UID: \"2bf91236-1a18-4683-867f-88f7a67cba50\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-rgvt7" Jul 07 18:47:35.290902 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.290825 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c1d6d0aa-b29e-49f7-b83f-df34ef4a027a-node-exporter-textfile\") pod \"node-exporter-m9skr\" (UID: \"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a\") " pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.290902 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.290851 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c1d6d0aa-b29e-49f7-b83f-df34ef4a027a-node-exporter-tls\") pod \"node-exporter-m9skr\" (UID: \"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a\") " pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.290902 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.290875 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bf91236-1a18-4683-867f-88f7a67cba50-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-rgvt7\" (UID: \"2bf91236-1a18-4683-867f-88f7a67cba50\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-rgvt7" Jul 07 18:47:35.290902 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.290901 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1d6d0aa-b29e-49f7-b83f-df34ef4a027a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m9skr\" (UID: \"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a\") " pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.291277 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.290936 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ln6sz\" (UniqueName: \"kubernetes.io/projected/2bf91236-1a18-4683-867f-88f7a67cba50-kube-api-access-ln6sz\") pod \"openshift-state-metrics-54cf96dfcf-rgvt7\" (UID: \"2bf91236-1a18-4683-867f-88f7a67cba50\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-rgvt7" Jul 07 18:47:35.291277 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:47:35.291049 2583 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 07 18:47:35.291277 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:47:35.291103 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bf91236-1a18-4683-867f-88f7a67cba50-openshift-state-metrics-tls podName:2bf91236-1a18-4683-867f-88f7a67cba50 nodeName:}" failed. No retries permitted until 2026-07-07 18:47:35.791085367 +0000 UTC m=+84.892103922 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/2bf91236-1a18-4683-867f-88f7a67cba50-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-rgvt7" (UID: "2bf91236-1a18-4683-867f-88f7a67cba50") : secret "openshift-state-metrics-tls" not found Jul 07 18:47:35.291277 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.291230 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c1d6d0aa-b29e-49f7-b83f-df34ef4a027a-node-exporter-accelerators-collector-config\") pod \"node-exporter-m9skr\" (UID: \"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a\") " pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.291277 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.291231 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c1d6d0aa-b29e-49f7-b83f-df34ef4a027a-node-exporter-textfile\") pod \"node-exporter-m9skr\" (UID: \"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a\") " pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.291277 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.291247 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1d6d0aa-b29e-49f7-b83f-df34ef4a027a-metrics-client-ca\") pod \"node-exporter-m9skr\" (UID: \"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a\") " pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.291542 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.291386 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bf91236-1a18-4683-867f-88f7a67cba50-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-rgvt7\" (UID: \"2bf91236-1a18-4683-867f-88f7a67cba50\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-rgvt7" Jul 07 18:47:35.293360 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.293338 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2bf91236-1a18-4683-867f-88f7a67cba50-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-rgvt7\" (UID: \"2bf91236-1a18-4683-867f-88f7a67cba50\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-rgvt7" Jul 07 18:47:35.293360 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.293352 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1d6d0aa-b29e-49f7-b83f-df34ef4a027a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m9skr\" (UID: \"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a\") " pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.293500 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.293428 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c1d6d0aa-b29e-49f7-b83f-df34ef4a027a-node-exporter-tls\") pod \"node-exporter-m9skr\" (UID: \"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a\") " pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.302250 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.302224 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mdfx\" (UniqueName: \"kubernetes.io/projected/c1d6d0aa-b29e-49f7-b83f-df34ef4a027a-kube-api-access-5mdfx\") pod \"node-exporter-m9skr\" (UID: \"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a\") " pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.302421 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.302294 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ln6sz\" (UniqueName: \"kubernetes.io/projected/2bf91236-1a18-4683-867f-88f7a67cba50-kube-api-access-ln6sz\") pod \"openshift-state-metrics-54cf96dfcf-rgvt7\" (UID: \"2bf91236-1a18-4683-867f-88f7a67cba50\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-rgvt7" Jul 07 18:47:35.382065 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.382021 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:35.428035 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.428001 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-m9skr" Jul 07 18:47:35.440124 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:47:35.440092 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1d6d0aa_b29e_49f7_b83f_df34ef4a027a.slice/crio-f8ed59c18a47f6aed0cde943ab052a407b87fddb0cb79c56466833cd8d67f639 WatchSource:0}: Error finding container f8ed59c18a47f6aed0cde943ab052a407b87fddb0cb79c56466833cd8d67f639: Status 404 returned error can't find the container with id f8ed59c18a47f6aed0cde943ab052a407b87fddb0cb79c56466833cd8d67f639 Jul 07 18:47:35.570712 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.570679 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:35.671773 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.671742 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:35.772305 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.772273 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:35.795783 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.795746 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bf91236-1a18-4683-867f-88f7a67cba50-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-rgvt7\" (UID: \"2bf91236-1a18-4683-867f-88f7a67cba50\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-rgvt7" Jul 07 18:47:35.798225 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.798200 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bf91236-1a18-4683-867f-88f7a67cba50-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-rgvt7\" (UID: \"2bf91236-1a18-4683-867f-88f7a67cba50\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-rgvt7" Jul 07 18:47:35.829908 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.829833 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m9skr" event={"ID":"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a","Type":"ContainerStarted","Data":"f8ed59c18a47f6aed0cde943ab052a407b87fddb0cb79c56466833cd8d67f639"} Jul 07 18:47:35.934903 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.934863 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:35.998462 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.998411 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77b65d74fd-4flgm" Jul 07 18:47:35.998629 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:35.998483 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77b65d74fd-4flgm" Jul 07 18:47:36.004274 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.004223 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77b65d74fd-4flgm" Jul 07 18:47:36.013106 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.013072 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-rgvt7" Jul 07 18:47:36.129995 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.128942 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:36.170460 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.170432 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-rgvt7"] Jul 07 18:47:36.228016 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.227974 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:36.257956 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.257916 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 18:47:36.263569 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.263543 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.266131 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.266093 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 07 18:47:36.266131 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.266112 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 07 18:47:36.266376 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.266097 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 07 18:47:36.266376 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.266289 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 07 18:47:36.266376 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.266306 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 07 18:47:36.266531 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.266380 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 07 18:47:36.266531 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.266311 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 07 18:47:36.266531 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.266480 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-s7fpk\"" Jul 07 18:47:36.266531 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.266527 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 07 18:47:36.267425 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.267280 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 07 18:47:36.282573 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.282542 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 18:47:36.300039 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.299997 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/00660473-d5d8-4e8c-a348-4a639947c47b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.300187 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.300045 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00660473-d5d8-4e8c-a348-4a639947c47b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.300187 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.300085 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00660473-d5d8-4e8c-a348-4a639947c47b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.300187 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.300126 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvrsm\" (UniqueName: \"kubernetes.io/projected/00660473-d5d8-4e8c-a348-4a639947c47b-kube-api-access-fvrsm\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.300454 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.300226 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/00660473-d5d8-4e8c-a348-4a639947c47b-config-out\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.300454 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.300340 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.300454 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.300369 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-web-config\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.300454 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.300396 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/00660473-d5d8-4e8c-a348-4a639947c47b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.300454 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.300415 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.300454 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.300430 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.300754 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.300520 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-config-volume\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.300754 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.300555 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.300754 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.300607 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.324755 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.324715 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:36.352761 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:47:36.352729 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bf91236_1a18_4683_867f_88f7a67cba50.slice/crio-9d8f7f2b2bc4805f24fd3326c8ab3660d6d6c939110bcb6597726993a045cf0e WatchSource:0}: Error finding container 9d8f7f2b2bc4805f24fd3326c8ab3660d6d6c939110bcb6597726993a045cf0e: Status 404 returned error can't find the container with id 9d8f7f2b2bc4805f24fd3326c8ab3660d6d6c939110bcb6597726993a045cf0e Jul 07 18:47:36.401157 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.401105 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvrsm\" (UniqueName: \"kubernetes.io/projected/00660473-d5d8-4e8c-a348-4a639947c47b-kube-api-access-fvrsm\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.401157 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.401153 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/00660473-d5d8-4e8c-a348-4a639947c47b-config-out\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.401157 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.401189 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.401561 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.401214 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-web-config\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.401561 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.401239 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/00660473-d5d8-4e8c-a348-4a639947c47b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.401561 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.401323 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.401561 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.401347 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.401561 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.401387 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-config-volume\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.401561 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.401411 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.401561 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.401452 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.401561 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.401483 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/00660473-d5d8-4e8c-a348-4a639947c47b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.401561 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.401510 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00660473-d5d8-4e8c-a348-4a639947c47b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.401561 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.401549 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00660473-d5d8-4e8c-a348-4a639947c47b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.402536 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.401931 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/00660473-d5d8-4e8c-a348-4a639947c47b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.402536 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.402303 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00660473-d5d8-4e8c-a348-4a639947c47b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.402536 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:47:36.402442 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/00660473-d5d8-4e8c-a348-4a639947c47b-alertmanager-trusted-ca-bundle podName:00660473-d5d8-4e8c-a348-4a639947c47b nodeName:}" failed. No retries permitted until 2026-07-07 18:47:36.902404925 +0000 UTC m=+86.003423494 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/00660473-d5d8-4e8c-a348-4a639947c47b-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "00660473-d5d8-4e8c-a348-4a639947c47b") : configmap references non-existent config key: ca-bundle.crt Jul 07 18:47:36.405115 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.404719 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/00660473-d5d8-4e8c-a348-4a639947c47b-config-out\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.405115 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.404760 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.405337 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.405287 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.406186 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.405370 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-config-volume\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.406186 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.405614 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.406186 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.405680 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.406186 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.406099 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-web-config\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.407519 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.407497 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.407657 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.407639 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/00660473-d5d8-4e8c-a348-4a639947c47b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.413479 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.413454 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvrsm\" (UniqueName: \"kubernetes.io/projected/00660473-d5d8-4e8c-a348-4a639947c47b-kube-api-access-fvrsm\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.835826 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.835794 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-rgvt7" event={"ID":"2bf91236-1a18-4683-867f-88f7a67cba50","Type":"ContainerStarted","Data":"4aeab44553889ac46a87b489e0389732876f7fcf21a69ee22a6af414cf7db140"} Jul 07 18:47:36.835986 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.835855 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-rgvt7" event={"ID":"2bf91236-1a18-4683-867f-88f7a67cba50","Type":"ContainerStarted","Data":"b788730a287b8862953807f5c2aa2aa608c127db3bd59148358a6ef99a5a463e"} Jul 07 18:47:36.835986 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.835879 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-rgvt7" event={"ID":"2bf91236-1a18-4683-867f-88f7a67cba50","Type":"ContainerStarted","Data":"9d8f7f2b2bc4805f24fd3326c8ab3660d6d6c939110bcb6597726993a045cf0e"} Jul 07 18:47:36.837431 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.837404 2583 generic.go:356] "Generic (PLEG): container finished" podID="c1d6d0aa-b29e-49f7-b83f-df34ef4a027a" containerID="d2fbef6cf0c439569805c9d38e18bbabfd37ec85dff50fd24046261cf6c5d609" exitCode=0 Jul 07 18:47:36.837547 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.837494 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m9skr" event={"ID":"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a","Type":"ContainerDied","Data":"d2fbef6cf0c439569805c9d38e18bbabfd37ec85dff50fd24046261cf6c5d609"} Jul 07 18:47:36.841974 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.841951 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77b65d74fd-4flgm" Jul 07 18:47:36.906710 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.906684 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00660473-d5d8-4e8c-a348-4a639947c47b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:36.907884 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:36.907861 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00660473-d5d8-4e8c-a348-4a639947c47b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:37.179746 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:37.179652 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:47:37.264996 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:37.264960 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:37.324310 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:37.324124 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 18:47:37.326923 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:47:37.326895 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00660473_d5d8_4e8c_a348_4a639947c47b.slice/crio-aa3c1f3403fa9f7f775a21a2a1d8a13a62924a32c112354f01bd81158d0880bb WatchSource:0}: Error finding container aa3c1f3403fa9f7f775a21a2a1d8a13a62924a32c112354f01bd81158d0880bb: Status 404 returned error can't find the container with id aa3c1f3403fa9f7f775a21a2a1d8a13a62924a32c112354f01bd81158d0880bb Jul 07 18:47:37.326923 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:37.326896 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:37.843912 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:37.843824 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-rgvt7" event={"ID":"2bf91236-1a18-4683-867f-88f7a67cba50","Type":"ContainerStarted","Data":"a104156d87bd21dc54c3bdc0c2d1012a44cf5a37245f409b290c1eff461e3d62"} Jul 07 18:47:37.845209 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:37.845182 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"00660473-d5d8-4e8c-a348-4a639947c47b","Type":"ContainerStarted","Data":"aa3c1f3403fa9f7f775a21a2a1d8a13a62924a32c112354f01bd81158d0880bb"} Jul 07 18:47:37.851779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:37.851736 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m9skr" event={"ID":"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a","Type":"ContainerStarted","Data":"e2c90bb22d67a256eb4902a58c2d64606c31a09c8a822f95be146f5aaf280458"} Jul 07 18:47:37.851891 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:37.851830 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m9skr" event={"ID":"c1d6d0aa-b29e-49f7-b83f-df34ef4a027a","Type":"ContainerStarted","Data":"5b39f77478f72d674e97613ac83e35115f5608fa3480d1d052285b6c5d2c7229"} Jul 07 18:47:37.864898 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:37.864847 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-rgvt7" podStartSLOduration=1.767965676 podStartE2EDuration="2.864832623s" totalImagesPullingTime="1.096866947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:47:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:47:37.863188579 +0000 UTC m=+86.964207155" watchObservedRunningTime="2026-07-07 18:47:37.864832623 +0000 UTC m=+86.965851197" Jul 07 18:47:38.046269 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:38.046217 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:38.147718 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:38.147630 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:38.425007 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:38.424882 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:38.526458 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:38.526423 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:38.856865 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:38.856830 2583 generic.go:356] "Generic (PLEG): container finished" podID="00660473-d5d8-4e8c-a348-4a639947c47b" containerID="4a14619d8bd34e0e9bf1a9452fbe038acfd73f281308639e780bfd8967bcccf5" exitCode=0 Jul 07 18:47:38.857066 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:38.856917 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"00660473-d5d8-4e8c-a348-4a639947c47b","Type":"ContainerDied","Data":"4a14619d8bd34e0e9bf1a9452fbe038acfd73f281308639e780bfd8967bcccf5"} Jul 07 18:47:38.885185 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:38.885135 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-m9skr" podStartSLOduration=2.847686687 podStartE2EDuration="3.88511786s" totalImagesPullingTime="968.581079ms" totalInitContainerRuntime="68.850094ms" isStatefulPod=true podCreationTimestamp="2026-07-07 18:47:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:47:37.885698592 +0000 UTC m=+86.986717166" watchObservedRunningTime="2026-07-07 18:47:38.88511786 +0000 UTC m=+87.986136445" Jul 07 18:47:39.147825 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:39.147741 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77b65d74fd-4flgm"] Jul 07 18:47:39.900594 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:39.900559 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:39.906352 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:39.906317 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-q45t8"] Jul 07 18:47:39.909599 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:39.909578 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-q45t8" Jul 07 18:47:39.911751 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:39.911719 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-jd22q\"" Jul 07 18:47:39.913900 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:39.912809 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 07 18:47:39.922811 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:39.922786 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-q45t8"] Jul 07 18:47:39.932682 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:39.932649 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e1e0387c-912d-4098-bc91-1865b90b26ad-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-q45t8\" (UID: \"e1e0387c-912d-4098-bc91-1865b90b26ad\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-q45t8" Jul 07 18:47:39.932835 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:39.932698 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74ljx\" (UniqueName: \"kubernetes.io/projected/e1e0387c-912d-4098-bc91-1865b90b26ad-kube-api-access-74ljx\") pod \"monitoring-plugin-d5dcff7cf-q45t8\" (UID: \"e1e0387c-912d-4098-bc91-1865b90b26ad\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-q45t8" Jul 07 18:47:39.988669 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:39.988634 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:40.033078 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:40.033048 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e1e0387c-912d-4098-bc91-1865b90b26ad-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-q45t8\" (UID: \"e1e0387c-912d-4098-bc91-1865b90b26ad\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-q45t8" Jul 07 18:47:40.033078 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:40.033082 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74ljx\" (UniqueName: \"kubernetes.io/projected/e1e0387c-912d-4098-bc91-1865b90b26ad-kube-api-access-74ljx\") pod \"monitoring-plugin-d5dcff7cf-q45t8\" (UID: \"e1e0387c-912d-4098-bc91-1865b90b26ad\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-q45t8" Jul 07 18:47:40.035589 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:40.035567 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e1e0387c-912d-4098-bc91-1865b90b26ad-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-q45t8\" (UID: \"e1e0387c-912d-4098-bc91-1865b90b26ad\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-q45t8" Jul 07 18:47:40.044225 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:40.044202 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74ljx\" (UniqueName: \"kubernetes.io/projected/e1e0387c-912d-4098-bc91-1865b90b26ad-kube-api-access-74ljx\") pod \"monitoring-plugin-d5dcff7cf-q45t8\" (UID: \"e1e0387c-912d-4098-bc91-1865b90b26ad\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-q45t8" Jul 07 18:47:40.060068 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:40.060039 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:40.220234 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:40.220152 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-q45t8" Jul 07 18:47:40.511422 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:40.511399 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-q45t8"] Jul 07 18:47:40.513944 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:47:40.513907 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1e0387c_912d_4098_bc91_1865b90b26ad.slice/crio-b1d1bd90384b19d3e0027f8bbceee816a5155b7d3d4c729ab418f9f89932e22c WatchSource:0}: Error finding container b1d1bd90384b19d3e0027f8bbceee816a5155b7d3d4c729ab418f9f89932e22c: Status 404 returned error can't find the container with id b1d1bd90384b19d3e0027f8bbceee816a5155b7d3d4c729ab418f9f89932e22c Jul 07 18:47:40.725351 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:40.725326 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:47:40.868061 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:40.868023 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-q45t8" event={"ID":"e1e0387c-912d-4098-bc91-1865b90b26ad","Type":"ContainerStarted","Data":"b1d1bd90384b19d3e0027f8bbceee816a5155b7d3d4c729ab418f9f89932e22c"} Jul 07 18:47:40.871067 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:40.871041 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"00660473-d5d8-4e8c-a348-4a639947c47b","Type":"ContainerStarted","Data":"fda5553304a9bc3c27e22321efd6bd5bfcdaf6d5df49f7c353425ed1388f62ec"} Jul 07 18:47:40.871191 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:40.871098 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"00660473-d5d8-4e8c-a348-4a639947c47b","Type":"ContainerStarted","Data":"477aef5dad0f30b94cfdabb463b57562dfc7fda94e329331aa33fadf04999ea5"} Jul 07 18:47:40.871191 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:40.871123 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"00660473-d5d8-4e8c-a348-4a639947c47b","Type":"ContainerStarted","Data":"4793e83a06f9be0fda81a9a7b49617efb74760d87986109d05fe1cd3227780ae"} Jul 07 18:47:40.871191 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:40.871148 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"00660473-d5d8-4e8c-a348-4a639947c47b","Type":"ContainerStarted","Data":"4b8dc7801709dd91aa05ecc3bf700668ec8e338b81e13e70e823985eb543ea1a"} Jul 07 18:47:40.871191 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:40.871171 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"00660473-d5d8-4e8c-a348-4a639947c47b","Type":"ContainerStarted","Data":"72574e624ed9452ba2a53b958c99337db314e1c97daefe5635213fd09a4a7244"} Jul 07 18:47:41.036654 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.036580 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:41.139494 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.139450 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:41.345942 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.345870 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:41.348168 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.348142 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 18:47:41.354476 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.354448 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.356702 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.356681 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 18:47:41.357346 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.357090 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 18:47:41.357346 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.357147 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 18:47:41.357651 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.357629 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-7xl68\"" Jul 07 18:47:41.357814 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.357752 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 18:47:41.357814 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.357799 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 18:47:41.357939 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.357872 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 18:47:41.359102 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.358209 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 18:47:41.359102 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.358342 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 18:47:41.359102 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.358425 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 18:47:41.359102 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.358450 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 18:47:41.359102 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.358519 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 18:47:41.359102 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.358638 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8re03dvljcvg9\"" Jul 07 18:47:41.359102 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.358705 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 18:47:41.359102 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.358899 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 18:47:41.363065 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.362890 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 18:47:41.363416 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.363394 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 18:47:41.370461 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.370357 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 18:47:41.450326 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.450290 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g54dj\" (UniqueName: \"kubernetes.io/projected/9d6992af-1c11-4cec-8f15-cf57722284c6-kube-api-access-g54dj\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.450518 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.450334 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.450518 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.450359 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.450518 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.450384 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.450518 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.450439 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.450518 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.450475 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.450756 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.450568 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.450756 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.450624 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-config\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.450756 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.450653 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.450756 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.450724 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.450917 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.450753 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-web-config\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.450917 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.450816 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.450917 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.450856 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.450917 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.450873 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9d6992af-1c11-4cec-8f15-cf57722284c6-config-out\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.450917 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.450893 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.450917 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.450910 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.451180 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.451084 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.451180 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.451128 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.451180 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.451161 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.451356 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.451201 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9d6992af-1c11-4cec-8f15-cf57722284c6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.551669 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.551634 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.551790 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.551683 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.551790 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.551714 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.551790 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.551736 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9d6992af-1c11-4cec-8f15-cf57722284c6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.551790 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.551759 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g54dj\" (UniqueName: \"kubernetes.io/projected/9d6992af-1c11-4cec-8f15-cf57722284c6-kube-api-access-g54dj\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.551790 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.551782 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.552062 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.551797 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.552062 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.551824 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.552062 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.551844 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.552062 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.551862 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.552062 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.551883 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.552062 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.551918 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-config\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.552062 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.551942 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.552062 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.551988 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.552062 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.552010 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-web-config\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.552062 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.552061 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.552588 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.552091 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.552588 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.552121 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9d6992af-1c11-4cec-8f15-cf57722284c6-config-out\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.552588 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.552154 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.552588 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.552198 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.560300 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.556062 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.562430 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.560762 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.562430 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.562323 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.570910 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.570577 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.571317 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.571231 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.571860 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.571804 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-config\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.572511 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.572096 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.575282 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.574005 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.575282 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.574032 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g54dj\" (UniqueName: \"kubernetes.io/projected/9d6992af-1c11-4cec-8f15-cf57722284c6-kube-api-access-g54dj\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.575282 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.574336 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.575282 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.574397 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.575282 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.574758 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.575282 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.574786 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-web-config\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.575282 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.575129 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.575282 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.575132 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9d6992af-1c11-4cec-8f15-cf57722284c6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.576950 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.576612 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.576950 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.576899 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.577099 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.577059 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9d6992af-1c11-4cec-8f15-cf57722284c6-config-out\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.577884 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.577787 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.579728 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.579705 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.672005 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.671953 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:41.830669 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.830638 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 18:47:41.881074 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.880985 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"00660473-d5d8-4e8c-a348-4a639947c47b","Type":"ContainerStarted","Data":"caa18de2649653321baa3a5b04b697a8287504041e8c52e445242ceeb46baa5e"} Jul 07 18:47:41.901230 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.901181 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:41.913755 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.913693 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.397029411 podStartE2EDuration="5.913672846s" totalImagesPullingTime="3.498741699s" totalInitContainerRuntime="17.901736ms" isStatefulPod=false podCreationTimestamp="2026-07-07 18:47:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:47:41.911172042 +0000 UTC m=+91.012190613" watchObservedRunningTime="2026-07-07 18:47:41.913672846 +0000 UTC m=+91.014691422" Jul 07 18:47:41.979617 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:41.979586 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:42.083145 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:47:42.083116 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d6992af_1c11_4cec_8f15_cf57722284c6.slice/crio-f3d330d2a83705feb1dfe5f1e39dea20a1d82ff30c6666a8a964bddc7a321472 WatchSource:0}: Error finding container f3d330d2a83705feb1dfe5f1e39dea20a1d82ff30c6666a8a964bddc7a321472: Status 404 returned error can't find the container with id f3d330d2a83705feb1dfe5f1e39dea20a1d82ff30c6666a8a964bddc7a321472 Jul 07 18:47:42.272754 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:42.272723 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:42.369162 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:42.369123 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:42.886167 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:42.886127 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-q45t8" event={"ID":"e1e0387c-912d-4098-bc91-1865b90b26ad","Type":"ContainerStarted","Data":"64c6d6680b1ed298bbd28e706a9b27571b80abf00b85df719f312b84df8d0c47"} Jul 07 18:47:42.886398 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:42.886201 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-q45t8" Jul 07 18:47:42.887751 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:42.887727 2583 generic.go:356] "Generic (PLEG): container finished" podID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerID="7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459" exitCode=0 Jul 07 18:47:42.887873 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:42.887816 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d6992af-1c11-4cec-8f15-cf57722284c6","Type":"ContainerDied","Data":"7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459"} Jul 07 18:47:42.887940 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:42.887874 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d6992af-1c11-4cec-8f15-cf57722284c6","Type":"ContainerStarted","Data":"f3d330d2a83705feb1dfe5f1e39dea20a1d82ff30c6666a8a964bddc7a321472"} Jul 07 18:47:42.891593 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:42.891570 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-q45t8" Jul 07 18:47:42.900671 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:42.900615 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-q45t8" podStartSLOduration=2.284858329 podStartE2EDuration="3.900598875s" totalImagesPullingTime="1.615740546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:47:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:47:42.900568333 +0000 UTC m=+92.001586910" watchObservedRunningTime="2026-07-07 18:47:42.900598875 +0000 UTC m=+92.001617453" Jul 07 18:47:44.582182 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:44.582146 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:46.104102 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:46.104070 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69d6b476d6-9lkmx"] Jul 07 18:47:46.298583 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:46.298537 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:46.400756 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:46.400593 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:46.499098 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:46.499061 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:46.912682 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:46.912646 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d6992af-1c11-4cec-8f15-cf57722284c6","Type":"ContainerStarted","Data":"407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b"} Jul 07 18:47:46.912872 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:46.912731 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d6992af-1c11-4cec-8f15-cf57722284c6","Type":"ContainerStarted","Data":"e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571"} Jul 07 18:47:48.615371 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:48.615220 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:48.678375 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:48.678336 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:48.927032 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:48.926951 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d6992af-1c11-4cec-8f15-cf57722284c6","Type":"ContainerStarted","Data":"3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98"} Jul 07 18:47:48.927032 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:48.927014 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d6992af-1c11-4cec-8f15-cf57722284c6","Type":"ContainerStarted","Data":"57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7"} Jul 07 18:47:48.927200 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:48.927042 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d6992af-1c11-4cec-8f15-cf57722284c6","Type":"ContainerStarted","Data":"8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f"} Jul 07 18:47:48.927200 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:48.927067 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d6992af-1c11-4cec-8f15-cf57722284c6","Type":"ContainerStarted","Data":"e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67"} Jul 07 18:47:48.955592 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:48.955510 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.905020637 podStartE2EDuration="7.955484943s" totalImagesPullingTime="5.034487491s" totalInitContainerRuntime="15.976815ms" isStatefulPod=false podCreationTimestamp="2026-07-07 18:47:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:47:48.955388923 +0000 UTC m=+98.056407501" watchObservedRunningTime="2026-07-07 18:47:48.955484943 +0000 UTC m=+98.056503519" Jul 07 18:47:51.672440 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:51.672403 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:53.388267 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:53.388190 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:53.490244 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:53.490215 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:53.588274 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:53.588224 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:55.146879 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:55.146834 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:55.246718 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:55.246685 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:55.766798 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:55.766767 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-l5d2z" Jul 07 18:47:56.898949 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:56.898914 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:57.000770 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:57.000737 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:57.694536 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:57.694498 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:47:57.794648 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:47:57.794607 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:48:04.168024 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:04.167980 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-77b65d74fd-4flgm" podUID="6994cdae-7247-4e9c-8c95-5d409cea63bd" containerName="console" containerID="cri-o://0818e8f08cf827c1117958c3776f2e77a9c4a11945eb61d5b55d6ea27a125a5e" gracePeriod=15 Jul 07 18:48:04.452907 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:04.452876 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b65d74fd-4flgm" Jul 07 18:48:04.568216 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:04.568109 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6994cdae-7247-4e9c-8c95-5d409cea63bd-console-oauth-config\") pod \"6994cdae-7247-4e9c-8c95-5d409cea63bd\" (UID: \"6994cdae-7247-4e9c-8c95-5d409cea63bd\") " Jul 07 18:48:04.568420 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:04.568295 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6994cdae-7247-4e9c-8c95-5d409cea63bd-console-serving-cert\") pod \"6994cdae-7247-4e9c-8c95-5d409cea63bd\" (UID: \"6994cdae-7247-4e9c-8c95-5d409cea63bd\") " Jul 07 18:48:04.568420 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:04.568324 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6994cdae-7247-4e9c-8c95-5d409cea63bd-console-config\") pod \"6994cdae-7247-4e9c-8c95-5d409cea63bd\" (UID: \"6994cdae-7247-4e9c-8c95-5d409cea63bd\") " Jul 07 18:48:04.568420 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:04.568371 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvcwt\" (UniqueName: \"kubernetes.io/projected/6994cdae-7247-4e9c-8c95-5d409cea63bd-kube-api-access-tvcwt\") pod \"6994cdae-7247-4e9c-8c95-5d409cea63bd\" (UID: \"6994cdae-7247-4e9c-8c95-5d409cea63bd\") " Jul 07 18:48:04.568420 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:04.568394 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6994cdae-7247-4e9c-8c95-5d409cea63bd-service-ca\") pod \"6994cdae-7247-4e9c-8c95-5d409cea63bd\" (UID: \"6994cdae-7247-4e9c-8c95-5d409cea63bd\") " Jul 07 18:48:04.568632 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:04.568446 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6994cdae-7247-4e9c-8c95-5d409cea63bd-oauth-serving-cert\") pod \"6994cdae-7247-4e9c-8c95-5d409cea63bd\" (UID: \"6994cdae-7247-4e9c-8c95-5d409cea63bd\") " Jul 07 18:48:04.568789 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:04.568754 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6994cdae-7247-4e9c-8c95-5d409cea63bd-console-config" (OuterVolumeSpecName: "console-config") pod "6994cdae-7247-4e9c-8c95-5d409cea63bd" (UID: "6994cdae-7247-4e9c-8c95-5d409cea63bd"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:48:04.568853 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:04.568819 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6994cdae-7247-4e9c-8c95-5d409cea63bd-service-ca" (OuterVolumeSpecName: "service-ca") pod "6994cdae-7247-4e9c-8c95-5d409cea63bd" (UID: "6994cdae-7247-4e9c-8c95-5d409cea63bd"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:48:04.569046 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:04.569024 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6994cdae-7247-4e9c-8c95-5d409cea63bd-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6994cdae-7247-4e9c-8c95-5d409cea63bd" (UID: "6994cdae-7247-4e9c-8c95-5d409cea63bd"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:48:04.570702 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:04.570660 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6994cdae-7247-4e9c-8c95-5d409cea63bd-kube-api-access-tvcwt" (OuterVolumeSpecName: "kube-api-access-tvcwt") pod "6994cdae-7247-4e9c-8c95-5d409cea63bd" (UID: "6994cdae-7247-4e9c-8c95-5d409cea63bd"). InnerVolumeSpecName "kube-api-access-tvcwt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:48:04.570801 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:04.570759 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6994cdae-7247-4e9c-8c95-5d409cea63bd-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6994cdae-7247-4e9c-8c95-5d409cea63bd" (UID: "6994cdae-7247-4e9c-8c95-5d409cea63bd"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:48:04.570844 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:04.570795 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6994cdae-7247-4e9c-8c95-5d409cea63bd-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6994cdae-7247-4e9c-8c95-5d409cea63bd" (UID: "6994cdae-7247-4e9c-8c95-5d409cea63bd"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:48:04.669451 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:04.669417 2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6994cdae-7247-4e9c-8c95-5d409cea63bd-console-serving-cert\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:48:04.669451 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:04.669449 2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6994cdae-7247-4e9c-8c95-5d409cea63bd-console-config\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:48:04.669657 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:04.669462 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvcwt\" (UniqueName: \"kubernetes.io/projected/6994cdae-7247-4e9c-8c95-5d409cea63bd-kube-api-access-tvcwt\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:48:04.669657 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:04.669476 2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6994cdae-7247-4e9c-8c95-5d409cea63bd-service-ca\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:48:04.669657 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:04.669487 2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6994cdae-7247-4e9c-8c95-5d409cea63bd-oauth-serving-cert\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:48:04.669657 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:04.669498 2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6994cdae-7247-4e9c-8c95-5d409cea63bd-console-oauth-config\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:48:04.994073 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:04.994040 2583 generic.go:356] "Generic (PLEG): container finished" podID="6994cdae-7247-4e9c-8c95-5d409cea63bd" containerID="0818e8f08cf827c1117958c3776f2e77a9c4a11945eb61d5b55d6ea27a125a5e" exitCode=0 Jul 07 18:48:04.994290 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:04.994081 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b65d74fd-4flgm" Jul 07 18:48:04.994290 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:04.994120 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b65d74fd-4flgm" event={"ID":"6994cdae-7247-4e9c-8c95-5d409cea63bd","Type":"ContainerDied","Data":"0818e8f08cf827c1117958c3776f2e77a9c4a11945eb61d5b55d6ea27a125a5e"} Jul 07 18:48:04.994290 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:04.994176 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b65d74fd-4flgm" event={"ID":"6994cdae-7247-4e9c-8c95-5d409cea63bd","Type":"ContainerDied","Data":"2a80429a0f295c75aa97ae9843ec9238349ad51dc4ee68077a7629c06a41a558"} Jul 07 18:48:04.994290 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:04.994205 2583 scope.go:117] "RemoveContainer" containerID="0818e8f08cf827c1117958c3776f2e77a9c4a11945eb61d5b55d6ea27a125a5e" Jul 07 18:48:05.003105 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:05.003087 2583 scope.go:117] "RemoveContainer" containerID="0818e8f08cf827c1117958c3776f2e77a9c4a11945eb61d5b55d6ea27a125a5e" Jul 07 18:48:05.003438 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:48:05.003404 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0818e8f08cf827c1117958c3776f2e77a9c4a11945eb61d5b55d6ea27a125a5e\": container with ID starting with 0818e8f08cf827c1117958c3776f2e77a9c4a11945eb61d5b55d6ea27a125a5e not found: ID does not exist" containerID="0818e8f08cf827c1117958c3776f2e77a9c4a11945eb61d5b55d6ea27a125a5e" Jul 07 18:48:05.003489 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:05.003446 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0818e8f08cf827c1117958c3776f2e77a9c4a11945eb61d5b55d6ea27a125a5e"} err="failed to get container status \"0818e8f08cf827c1117958c3776f2e77a9c4a11945eb61d5b55d6ea27a125a5e\": rpc error: code = NotFound desc = could not find container \"0818e8f08cf827c1117958c3776f2e77a9c4a11945eb61d5b55d6ea27a125a5e\": container with ID starting with 0818e8f08cf827c1117958c3776f2e77a9c4a11945eb61d5b55d6ea27a125a5e not found: ID does not exist" Jul 07 18:48:05.015043 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:05.015012 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77b65d74fd-4flgm"] Jul 07 18:48:05.017950 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:05.017920 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77b65d74fd-4flgm"] Jul 07 18:48:05.401327 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:05.401293 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:48:05.413166 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:05.413133 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6994cdae-7247-4e9c-8c95-5d409cea63bd" path="/var/lib/kubelet/pods/6994cdae-7247-4e9c-8c95-5d409cea63bd/volumes" Jul 07 18:48:05.497240 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:05.497210 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:48:05.599272 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:05.599229 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:48:07.004450 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:07.004416 2583 generic.go:356] "Generic (PLEG): container finished" podID="b8f4ce07-b53a-4e57-a9ca-e64da6750c7f" containerID="540ff09500b92833ea4d9d0d2a9228f110ec2651df611b077f6db3b21bba1109" exitCode=0 Jul 07 18:48:07.004788 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:07.004457 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-lqs4g" event={"ID":"b8f4ce07-b53a-4e57-a9ca-e64da6750c7f","Type":"ContainerDied","Data":"540ff09500b92833ea4d9d0d2a9228f110ec2651df611b077f6db3b21bba1109"} Jul 07 18:48:07.004830 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:07.004802 2583 scope.go:117] "RemoveContainer" containerID="540ff09500b92833ea4d9d0d2a9228f110ec2651df611b077f6db3b21bba1109" Jul 07 18:48:08.010735 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:08.010698 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:48:08.011173 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:08.011020 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-lqs4g" event={"ID":"b8f4ce07-b53a-4e57-a9ca-e64da6750c7f","Type":"ContainerStarted","Data":"c9e16b4fc43575e3bd9803f0ec7517dcc4cd40a8f8746800b52885131c0a6d93"} Jul 07 18:48:11.131145 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.131104 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" podUID="37675bf2-2454-4e3c-9c53-9f7ae3517b6a" containerName="registry" containerID="cri-o://e2d189ee9afb63a61feb463f85017e7614b921cfe9c09d451802122761378169" gracePeriod=30 Jul 07 18:48:11.398333 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.398306 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:48:11.529081 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.529046 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-certificates\") pod \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " Jul 07 18:48:11.529300 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.529092 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-bound-sa-token\") pod \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " Jul 07 18:48:11.529300 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.529114 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-ca-trust-extracted\") pod \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " Jul 07 18:48:11.529300 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.529133 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-trusted-ca\") pod \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " Jul 07 18:48:11.529300 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.529164 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-image-registry-private-configuration\") pod \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " Jul 07 18:48:11.529300 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.529188 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-tls\") pod \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " Jul 07 18:48:11.529300 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.529248 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-installation-pull-secrets\") pod \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " Jul 07 18:48:11.529616 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.529315 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7rdv\" (UniqueName: \"kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-kube-api-access-k7rdv\") pod \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\" (UID: \"37675bf2-2454-4e3c-9c53-9f7ae3517b6a\") " Jul 07 18:48:11.529616 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.529548 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "37675bf2-2454-4e3c-9c53-9f7ae3517b6a" (UID: "37675bf2-2454-4e3c-9c53-9f7ae3517b6a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:48:11.529722 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.529657 2583 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-certificates\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:48:11.529779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.529759 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "37675bf2-2454-4e3c-9c53-9f7ae3517b6a" (UID: "37675bf2-2454-4e3c-9c53-9f7ae3517b6a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:48:11.532046 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.532012 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "37675bf2-2454-4e3c-9c53-9f7ae3517b6a" (UID: "37675bf2-2454-4e3c-9c53-9f7ae3517b6a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:48:11.532149 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.532044 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "37675bf2-2454-4e3c-9c53-9f7ae3517b6a" (UID: "37675bf2-2454-4e3c-9c53-9f7ae3517b6a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:48:11.532149 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.532098 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "37675bf2-2454-4e3c-9c53-9f7ae3517b6a" (UID: "37675bf2-2454-4e3c-9c53-9f7ae3517b6a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:48:11.532307 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.532285 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-kube-api-access-k7rdv" (OuterVolumeSpecName: "kube-api-access-k7rdv") pod "37675bf2-2454-4e3c-9c53-9f7ae3517b6a" (UID: "37675bf2-2454-4e3c-9c53-9f7ae3517b6a"). InnerVolumeSpecName "kube-api-access-k7rdv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:48:11.532411 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.532390 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "37675bf2-2454-4e3c-9c53-9f7ae3517b6a" (UID: "37675bf2-2454-4e3c-9c53-9f7ae3517b6a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:48:11.537873 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.537850 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "37675bf2-2454-4e3c-9c53-9f7ae3517b6a" (UID: "37675bf2-2454-4e3c-9c53-9f7ae3517b6a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:48:11.630074 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.630044 2583 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-bound-sa-token\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:48:11.630074 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.630071 2583 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-ca-trust-extracted\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:48:11.630074 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.630080 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-trusted-ca\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:48:11.630327 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.630090 2583 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-image-registry-private-configuration\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:48:11.630327 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.630099 2583 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-registry-tls\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:48:11.630327 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.630109 2583 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-installation-pull-secrets\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:48:11.630327 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.630117 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7rdv\" (UniqueName: \"kubernetes.io/projected/37675bf2-2454-4e3c-9c53-9f7ae3517b6a-kube-api-access-k7rdv\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:48:11.894762 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.894731 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:48:11.951803 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:11.951769 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:48:12.027839 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:12.027804 2583 generic.go:356] "Generic (PLEG): container finished" podID="37675bf2-2454-4e3c-9c53-9f7ae3517b6a" containerID="e2d189ee9afb63a61feb463f85017e7614b921cfe9c09d451802122761378169" exitCode=0 Jul 07 18:48:12.027988 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:12.027849 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" event={"ID":"37675bf2-2454-4e3c-9c53-9f7ae3517b6a","Type":"ContainerDied","Data":"e2d189ee9afb63a61feb463f85017e7614b921cfe9c09d451802122761378169"} Jul 07 18:48:12.027988 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:12.027864 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" Jul 07 18:48:12.027988 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:12.027889 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69d6b476d6-9lkmx" event={"ID":"37675bf2-2454-4e3c-9c53-9f7ae3517b6a","Type":"ContainerDied","Data":"a584fa6a1c30592163cc2e960a008534a004dce84aa8d39670b7120b166d2d16"} Jul 07 18:48:12.027988 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:12.027912 2583 scope.go:117] "RemoveContainer" containerID="e2d189ee9afb63a61feb463f85017e7614b921cfe9c09d451802122761378169" Jul 07 18:48:12.036931 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:12.036915 2583 scope.go:117] "RemoveContainer" containerID="e2d189ee9afb63a61feb463f85017e7614b921cfe9c09d451802122761378169" Jul 07 18:48:12.037190 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:48:12.037169 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2d189ee9afb63a61feb463f85017e7614b921cfe9c09d451802122761378169\": container with ID starting with e2d189ee9afb63a61feb463f85017e7614b921cfe9c09d451802122761378169 not found: ID does not exist" containerID="e2d189ee9afb63a61feb463f85017e7614b921cfe9c09d451802122761378169" Jul 07 18:48:12.037240 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:12.037199 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2d189ee9afb63a61feb463f85017e7614b921cfe9c09d451802122761378169"} err="failed to get container status \"e2d189ee9afb63a61feb463f85017e7614b921cfe9c09d451802122761378169\": rpc error: code = NotFound desc = could not find container \"e2d189ee9afb63a61feb463f85017e7614b921cfe9c09d451802122761378169\": container with ID starting with e2d189ee9afb63a61feb463f85017e7614b921cfe9c09d451802122761378169 not found: ID does not exist" Jul 07 18:48:12.047997 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:12.047974 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69d6b476d6-9lkmx"] Jul 07 18:48:12.052285 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:12.052246 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-69d6b476d6-9lkmx"] Jul 07 18:48:13.418969 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:13.418129 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37675bf2-2454-4e3c-9c53-9f7ae3517b6a" path="/var/lib/kubelet/pods/37675bf2-2454-4e3c-9c53-9f7ae3517b6a/volumes" Jul 07 18:48:41.672803 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:41.672762 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:41.689087 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:41.689063 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:42.175871 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:42.175842 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:42.876312 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:42.876269 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:48:54.326366 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:54.326330 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:48:54.675399 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:54.675301 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:48:56.217821 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:56.217792 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-wszbl_363d866e-b0a1-4351-8f85-b994a18200a7/cluster-monitoring-operator/0.log" Jul 07 18:48:56.218184 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:56.217833 2583 generic.go:356] "Generic (PLEG): container finished" podID="363d866e-b0a1-4351-8f85-b994a18200a7" containerID="f9e0e0145f767319ec8f08dd48c95f332f8e639775fc76d6a0d95ea9fbd9b62b" exitCode=2 Jul 07 18:48:56.218184 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:56.217911 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl" event={"ID":"363d866e-b0a1-4351-8f85-b994a18200a7","Type":"ContainerDied","Data":"f9e0e0145f767319ec8f08dd48c95f332f8e639775fc76d6a0d95ea9fbd9b62b"} Jul 07 18:48:56.218280 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:56.218224 2583 scope.go:117] "RemoveContainer" containerID="f9e0e0145f767319ec8f08dd48c95f332f8e639775fc76d6a0d95ea9fbd9b62b" Jul 07 18:48:57.224283 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:57.224242 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-wszbl_363d866e-b0a1-4351-8f85-b994a18200a7/cluster-monitoring-operator/0.log" Jul 07 18:48:57.224698 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:48:57.224368 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl" event={"ID":"363d866e-b0a1-4351-8f85-b994a18200a7","Type":"ContainerStarted","Data":"b57650887b3b7b3a170bc20e72c3c6caafe3729d948d83e1510bb14c3898a481"} Jul 07 18:49:00.237737 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:00.237709 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-wszbl_363d866e-b0a1-4351-8f85-b994a18200a7/cluster-monitoring-operator/1.log" Jul 07 18:49:00.238144 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:00.238128 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-wszbl_363d866e-b0a1-4351-8f85-b994a18200a7/cluster-monitoring-operator/0.log" Jul 07 18:49:00.238187 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:00.238163 2583 generic.go:356] "Generic (PLEG): container finished" podID="363d866e-b0a1-4351-8f85-b994a18200a7" containerID="b57650887b3b7b3a170bc20e72c3c6caafe3729d948d83e1510bb14c3898a481" exitCode=2 Jul 07 18:49:00.238225 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:00.238207 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl" event={"ID":"363d866e-b0a1-4351-8f85-b994a18200a7","Type":"ContainerDied","Data":"b57650887b3b7b3a170bc20e72c3c6caafe3729d948d83e1510bb14c3898a481"} Jul 07 18:49:00.238285 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:00.238272 2583 scope.go:117] "RemoveContainer" containerID="f9e0e0145f767319ec8f08dd48c95f332f8e639775fc76d6a0d95ea9fbd9b62b" Jul 07 18:49:00.238579 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:00.238562 2583 scope.go:117] "RemoveContainer" containerID="b57650887b3b7b3a170bc20e72c3c6caafe3729d948d83e1510bb14c3898a481" Jul 07 18:49:00.238776 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:49:00.238756 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cluster-monitoring-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cluster-monitoring-operator pod=cluster-monitoring-operator-64786d9cbb-wszbl_openshift-monitoring(363d866e-b0a1-4351-8f85-b994a18200a7)\"" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl" podUID="363d866e-b0a1-4351-8f85-b994a18200a7" Jul 07 18:49:01.243081 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:01.243050 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-wszbl_363d866e-b0a1-4351-8f85-b994a18200a7/cluster-monitoring-operator/1.log" Jul 07 18:49:14.408803 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:14.408770 2583 scope.go:117] "RemoveContainer" containerID="b57650887b3b7b3a170bc20e72c3c6caafe3729d948d83e1510bb14c3898a481" Jul 07 18:49:15.302376 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:15.302349 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-wszbl_363d866e-b0a1-4351-8f85-b994a18200a7/cluster-monitoring-operator/1.log" Jul 07 18:49:15.302543 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:15.302396 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-wszbl" event={"ID":"363d866e-b0a1-4351-8f85-b994a18200a7","Type":"ContainerStarted","Data":"6847cf98292475713b363a738ac549e7638e5bc36b15a78fd55a03dd15ed2910"} Jul 07 18:49:18.956835 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:18.956797 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 18:49:18.957294 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:18.957151 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="00660473-d5d8-4e8c-a348-4a639947c47b" containerName="alertmanager" containerID="cri-o://72574e624ed9452ba2a53b958c99337db314e1c97daefe5635213fd09a4a7244" gracePeriod=120 Jul 07 18:49:18.957294 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:18.957196 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="00660473-d5d8-4e8c-a348-4a639947c47b" containerName="kube-rbac-proxy-metric" containerID="cri-o://fda5553304a9bc3c27e22321efd6bd5bfcdaf6d5df49f7c353425ed1388f62ec" gracePeriod=120 Jul 07 18:49:18.957294 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:18.957216 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="00660473-d5d8-4e8c-a348-4a639947c47b" containerName="kube-rbac-proxy-web" containerID="cri-o://4793e83a06f9be0fda81a9a7b49617efb74760d87986109d05fe1cd3227780ae" gracePeriod=120 Jul 07 18:49:18.957468 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:18.957250 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="00660473-d5d8-4e8c-a348-4a639947c47b" containerName="kube-rbac-proxy" containerID="cri-o://477aef5dad0f30b94cfdabb463b57562dfc7fda94e329331aa33fadf04999ea5" gracePeriod=120 Jul 07 18:49:18.957468 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:18.957329 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="00660473-d5d8-4e8c-a348-4a639947c47b" containerName="prom-label-proxy" containerID="cri-o://caa18de2649653321baa3a5b04b697a8287504041e8c52e445242ceeb46baa5e" gracePeriod=120 Jul 07 18:49:18.957468 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:18.957288 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="00660473-d5d8-4e8c-a348-4a639947c47b" containerName="config-reloader" containerID="cri-o://4b8dc7801709dd91aa05ecc3bf700668ec8e338b81e13e70e823985eb543ea1a" gracePeriod=120 Jul 07 18:49:19.324949 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:19.324913 2583 generic.go:356] "Generic (PLEG): container finished" podID="00660473-d5d8-4e8c-a348-4a639947c47b" containerID="caa18de2649653321baa3a5b04b697a8287504041e8c52e445242ceeb46baa5e" exitCode=0 Jul 07 18:49:19.324949 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:19.324942 2583 generic.go:356] "Generic (PLEG): container finished" podID="00660473-d5d8-4e8c-a348-4a639947c47b" containerID="477aef5dad0f30b94cfdabb463b57562dfc7fda94e329331aa33fadf04999ea5" exitCode=0 Jul 07 18:49:19.324949 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:19.324951 2583 generic.go:356] "Generic (PLEG): container finished" podID="00660473-d5d8-4e8c-a348-4a639947c47b" containerID="4b8dc7801709dd91aa05ecc3bf700668ec8e338b81e13e70e823985eb543ea1a" exitCode=0 Jul 07 18:49:19.324949 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:19.324959 2583 generic.go:356] "Generic (PLEG): container finished" podID="00660473-d5d8-4e8c-a348-4a639947c47b" containerID="72574e624ed9452ba2a53b958c99337db314e1c97daefe5635213fd09a4a7244" exitCode=0 Jul 07 18:49:19.325289 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:19.324997 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"00660473-d5d8-4e8c-a348-4a639947c47b","Type":"ContainerDied","Data":"caa18de2649653321baa3a5b04b697a8287504041e8c52e445242ceeb46baa5e"} Jul 07 18:49:19.325289 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:19.325084 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"00660473-d5d8-4e8c-a348-4a639947c47b","Type":"ContainerDied","Data":"477aef5dad0f30b94cfdabb463b57562dfc7fda94e329331aa33fadf04999ea5"} Jul 07 18:49:19.325289 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:19.325129 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"00660473-d5d8-4e8c-a348-4a639947c47b","Type":"ContainerDied","Data":"4b8dc7801709dd91aa05ecc3bf700668ec8e338b81e13e70e823985eb543ea1a"} Jul 07 18:49:19.325289 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:19.325169 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"00660473-d5d8-4e8c-a348-4a639947c47b","Type":"ContainerDied","Data":"72574e624ed9452ba2a53b958c99337db314e1c97daefe5635213fd09a4a7244"} Jul 07 18:49:20.306922 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.306898 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.328330 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.328294 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-cluster-tls-config\") pod \"00660473-d5d8-4e8c-a348-4a639947c47b\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " Jul 07 18:49:20.328330 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.328340 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00660473-d5d8-4e8c-a348-4a639947c47b-alertmanager-trusted-ca-bundle\") pod \"00660473-d5d8-4e8c-a348-4a639947c47b\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " Jul 07 18:49:20.328587 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.328358 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/00660473-d5d8-4e8c-a348-4a639947c47b-config-out\") pod \"00660473-d5d8-4e8c-a348-4a639947c47b\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " Jul 07 18:49:20.328587 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.328373 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-config-volume\") pod \"00660473-d5d8-4e8c-a348-4a639947c47b\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " Jul 07 18:49:20.328587 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.328402 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00660473-d5d8-4e8c-a348-4a639947c47b-metrics-client-ca\") pod \"00660473-d5d8-4e8c-a348-4a639947c47b\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " Jul 07 18:49:20.328587 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.328485 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/00660473-d5d8-4e8c-a348-4a639947c47b-alertmanager-main-db\") pod \"00660473-d5d8-4e8c-a348-4a639947c47b\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " Jul 07 18:49:20.328587 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.328510 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-secret-alertmanager-kube-rbac-proxy-web\") pod \"00660473-d5d8-4e8c-a348-4a639947c47b\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " Jul 07 18:49:20.328587 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.328536 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/00660473-d5d8-4e8c-a348-4a639947c47b-tls-assets\") pod \"00660473-d5d8-4e8c-a348-4a639947c47b\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " Jul 07 18:49:20.328587 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.328568 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-web-config\") pod \"00660473-d5d8-4e8c-a348-4a639947c47b\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " Jul 07 18:49:20.328919 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.328595 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-secret-alertmanager-main-tls\") pod \"00660473-d5d8-4e8c-a348-4a639947c47b\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " Jul 07 18:49:20.328919 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.328629 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvrsm\" (UniqueName: \"kubernetes.io/projected/00660473-d5d8-4e8c-a348-4a639947c47b-kube-api-access-fvrsm\") pod \"00660473-d5d8-4e8c-a348-4a639947c47b\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " Jul 07 18:49:20.328919 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.328659 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"00660473-d5d8-4e8c-a348-4a639947c47b\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " Jul 07 18:49:20.328919 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.328702 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-secret-alertmanager-kube-rbac-proxy\") pod \"00660473-d5d8-4e8c-a348-4a639947c47b\" (UID: \"00660473-d5d8-4e8c-a348-4a639947c47b\") " Jul 07 18:49:20.328919 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.328767 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00660473-d5d8-4e8c-a348-4a639947c47b-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "00660473-d5d8-4e8c-a348-4a639947c47b" (UID: "00660473-d5d8-4e8c-a348-4a639947c47b"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:49:20.329210 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.329003 2583 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00660473-d5d8-4e8c-a348-4a639947c47b-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:20.329210 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.329063 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00660473-d5d8-4e8c-a348-4a639947c47b-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "00660473-d5d8-4e8c-a348-4a639947c47b" (UID: "00660473-d5d8-4e8c-a348-4a639947c47b"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:49:20.329423 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.329400 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00660473-d5d8-4e8c-a348-4a639947c47b-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "00660473-d5d8-4e8c-a348-4a639947c47b" (UID: "00660473-d5d8-4e8c-a348-4a639947c47b"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:49:20.331082 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.331041 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-config-volume" (OuterVolumeSpecName: "config-volume") pod "00660473-d5d8-4e8c-a348-4a639947c47b" (UID: "00660473-d5d8-4e8c-a348-4a639947c47b"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:49:20.331621 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.331586 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00660473-d5d8-4e8c-a348-4a639947c47b-config-out" (OuterVolumeSpecName: "config-out") pod "00660473-d5d8-4e8c-a348-4a639947c47b" (UID: "00660473-d5d8-4e8c-a348-4a639947c47b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:49:20.332446 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.332418 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "00660473-d5d8-4e8c-a348-4a639947c47b" (UID: "00660473-d5d8-4e8c-a348-4a639947c47b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:49:20.332809 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.332782 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "00660473-d5d8-4e8c-a348-4a639947c47b" (UID: "00660473-d5d8-4e8c-a348-4a639947c47b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:49:20.333048 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.333000 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00660473-d5d8-4e8c-a348-4a639947c47b-kube-api-access-fvrsm" (OuterVolumeSpecName: "kube-api-access-fvrsm") pod "00660473-d5d8-4e8c-a348-4a639947c47b" (UID: "00660473-d5d8-4e8c-a348-4a639947c47b"). InnerVolumeSpecName "kube-api-access-fvrsm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:49:20.333613 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.333585 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "00660473-d5d8-4e8c-a348-4a639947c47b" (UID: "00660473-d5d8-4e8c-a348-4a639947c47b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:49:20.334301 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.334222 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "00660473-d5d8-4e8c-a348-4a639947c47b" (UID: "00660473-d5d8-4e8c-a348-4a639947c47b"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:49:20.334764 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.334720 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00660473-d5d8-4e8c-a348-4a639947c47b-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "00660473-d5d8-4e8c-a348-4a639947c47b" (UID: "00660473-d5d8-4e8c-a348-4a639947c47b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:49:20.334982 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.334940 2583 generic.go:356] "Generic (PLEG): container finished" podID="00660473-d5d8-4e8c-a348-4a639947c47b" containerID="fda5553304a9bc3c27e22321efd6bd5bfcdaf6d5df49f7c353425ed1388f62ec" exitCode=0 Jul 07 18:49:20.334982 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.334967 2583 generic.go:356] "Generic (PLEG): container finished" podID="00660473-d5d8-4e8c-a348-4a639947c47b" containerID="4793e83a06f9be0fda81a9a7b49617efb74760d87986109d05fe1cd3227780ae" exitCode=0 Jul 07 18:49:20.335117 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.335023 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"00660473-d5d8-4e8c-a348-4a639947c47b","Type":"ContainerDied","Data":"fda5553304a9bc3c27e22321efd6bd5bfcdaf6d5df49f7c353425ed1388f62ec"} Jul 07 18:49:20.335117 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.335107 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"00660473-d5d8-4e8c-a348-4a639947c47b","Type":"ContainerDied","Data":"4793e83a06f9be0fda81a9a7b49617efb74760d87986109d05fe1cd3227780ae"} Jul 07 18:49:20.335218 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.335154 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"00660473-d5d8-4e8c-a348-4a639947c47b","Type":"ContainerDied","Data":"aa3c1f3403fa9f7f775a21a2a1d8a13a62924a32c112354f01bd81158d0880bb"} Jul 07 18:49:20.335218 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.335204 2583 scope.go:117] "RemoveContainer" containerID="caa18de2649653321baa3a5b04b697a8287504041e8c52e445242ceeb46baa5e" Jul 07 18:49:20.335512 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.335446 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.338245 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.338217 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "00660473-d5d8-4e8c-a348-4a639947c47b" (UID: "00660473-d5d8-4e8c-a348-4a639947c47b"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:49:20.349567 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.349512 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-web-config" (OuterVolumeSpecName: "web-config") pod "00660473-d5d8-4e8c-a348-4a639947c47b" (UID: "00660473-d5d8-4e8c-a348-4a639947c47b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:49:20.353240 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.353219 2583 scope.go:117] "RemoveContainer" containerID="fda5553304a9bc3c27e22321efd6bd5bfcdaf6d5df49f7c353425ed1388f62ec" Jul 07 18:49:20.362443 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.362419 2583 scope.go:117] "RemoveContainer" containerID="477aef5dad0f30b94cfdabb463b57562dfc7fda94e329331aa33fadf04999ea5" Jul 07 18:49:20.370323 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.370302 2583 scope.go:117] "RemoveContainer" containerID="4793e83a06f9be0fda81a9a7b49617efb74760d87986109d05fe1cd3227780ae" Jul 07 18:49:20.378374 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.378349 2583 scope.go:117] "RemoveContainer" containerID="4b8dc7801709dd91aa05ecc3bf700668ec8e338b81e13e70e823985eb543ea1a" Jul 07 18:49:20.385934 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.385911 2583 scope.go:117] "RemoveContainer" containerID="72574e624ed9452ba2a53b958c99337db314e1c97daefe5635213fd09a4a7244" Jul 07 18:49:20.393639 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.393620 2583 scope.go:117] "RemoveContainer" containerID="4a14619d8bd34e0e9bf1a9452fbe038acfd73f281308639e780bfd8967bcccf5" Jul 07 18:49:20.400695 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.400677 2583 scope.go:117] "RemoveContainer" containerID="caa18de2649653321baa3a5b04b697a8287504041e8c52e445242ceeb46baa5e" Jul 07 18:49:20.400961 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:49:20.400941 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caa18de2649653321baa3a5b04b697a8287504041e8c52e445242ceeb46baa5e\": container with ID starting with caa18de2649653321baa3a5b04b697a8287504041e8c52e445242ceeb46baa5e not found: ID does not exist" containerID="caa18de2649653321baa3a5b04b697a8287504041e8c52e445242ceeb46baa5e" Jul 07 18:49:20.401010 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.400970 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caa18de2649653321baa3a5b04b697a8287504041e8c52e445242ceeb46baa5e"} err="failed to get container status \"caa18de2649653321baa3a5b04b697a8287504041e8c52e445242ceeb46baa5e\": rpc error: code = NotFound desc = could not find container \"caa18de2649653321baa3a5b04b697a8287504041e8c52e445242ceeb46baa5e\": container with ID starting with caa18de2649653321baa3a5b04b697a8287504041e8c52e445242ceeb46baa5e not found: ID does not exist" Jul 07 18:49:20.401010 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.400990 2583 scope.go:117] "RemoveContainer" containerID="fda5553304a9bc3c27e22321efd6bd5bfcdaf6d5df49f7c353425ed1388f62ec" Jul 07 18:49:20.401226 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:49:20.401210 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fda5553304a9bc3c27e22321efd6bd5bfcdaf6d5df49f7c353425ed1388f62ec\": container with ID starting with fda5553304a9bc3c27e22321efd6bd5bfcdaf6d5df49f7c353425ed1388f62ec not found: ID does not exist" containerID="fda5553304a9bc3c27e22321efd6bd5bfcdaf6d5df49f7c353425ed1388f62ec" Jul 07 18:49:20.401295 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.401230 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fda5553304a9bc3c27e22321efd6bd5bfcdaf6d5df49f7c353425ed1388f62ec"} err="failed to get container status \"fda5553304a9bc3c27e22321efd6bd5bfcdaf6d5df49f7c353425ed1388f62ec\": rpc error: code = NotFound desc = could not find container \"fda5553304a9bc3c27e22321efd6bd5bfcdaf6d5df49f7c353425ed1388f62ec\": container with ID starting with fda5553304a9bc3c27e22321efd6bd5bfcdaf6d5df49f7c353425ed1388f62ec not found: ID does not exist" Jul 07 18:49:20.401295 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.401243 2583 scope.go:117] "RemoveContainer" containerID="477aef5dad0f30b94cfdabb463b57562dfc7fda94e329331aa33fadf04999ea5" Jul 07 18:49:20.401476 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:49:20.401459 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"477aef5dad0f30b94cfdabb463b57562dfc7fda94e329331aa33fadf04999ea5\": container with ID starting with 477aef5dad0f30b94cfdabb463b57562dfc7fda94e329331aa33fadf04999ea5 not found: ID does not exist" containerID="477aef5dad0f30b94cfdabb463b57562dfc7fda94e329331aa33fadf04999ea5" Jul 07 18:49:20.401642 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.401481 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"477aef5dad0f30b94cfdabb463b57562dfc7fda94e329331aa33fadf04999ea5"} err="failed to get container status \"477aef5dad0f30b94cfdabb463b57562dfc7fda94e329331aa33fadf04999ea5\": rpc error: code = NotFound desc = could not find container \"477aef5dad0f30b94cfdabb463b57562dfc7fda94e329331aa33fadf04999ea5\": container with ID starting with 477aef5dad0f30b94cfdabb463b57562dfc7fda94e329331aa33fadf04999ea5 not found: ID does not exist" Jul 07 18:49:20.401642 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.401495 2583 scope.go:117] "RemoveContainer" containerID="4793e83a06f9be0fda81a9a7b49617efb74760d87986109d05fe1cd3227780ae" Jul 07 18:49:20.401713 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:49:20.401689 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4793e83a06f9be0fda81a9a7b49617efb74760d87986109d05fe1cd3227780ae\": container with ID starting with 4793e83a06f9be0fda81a9a7b49617efb74760d87986109d05fe1cd3227780ae not found: ID does not exist" containerID="4793e83a06f9be0fda81a9a7b49617efb74760d87986109d05fe1cd3227780ae" Jul 07 18:49:20.401713 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.401705 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4793e83a06f9be0fda81a9a7b49617efb74760d87986109d05fe1cd3227780ae"} err="failed to get container status \"4793e83a06f9be0fda81a9a7b49617efb74760d87986109d05fe1cd3227780ae\": rpc error: code = NotFound desc = could not find container \"4793e83a06f9be0fda81a9a7b49617efb74760d87986109d05fe1cd3227780ae\": container with ID starting with 4793e83a06f9be0fda81a9a7b49617efb74760d87986109d05fe1cd3227780ae not found: ID does not exist" Jul 07 18:49:20.401784 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.401718 2583 scope.go:117] "RemoveContainer" containerID="4b8dc7801709dd91aa05ecc3bf700668ec8e338b81e13e70e823985eb543ea1a" Jul 07 18:49:20.401970 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:49:20.401952 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b8dc7801709dd91aa05ecc3bf700668ec8e338b81e13e70e823985eb543ea1a\": container with ID starting with 4b8dc7801709dd91aa05ecc3bf700668ec8e338b81e13e70e823985eb543ea1a not found: ID does not exist" containerID="4b8dc7801709dd91aa05ecc3bf700668ec8e338b81e13e70e823985eb543ea1a" Jul 07 18:49:20.402022 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.401973 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b8dc7801709dd91aa05ecc3bf700668ec8e338b81e13e70e823985eb543ea1a"} err="failed to get container status \"4b8dc7801709dd91aa05ecc3bf700668ec8e338b81e13e70e823985eb543ea1a\": rpc error: code = NotFound desc = could not find container \"4b8dc7801709dd91aa05ecc3bf700668ec8e338b81e13e70e823985eb543ea1a\": container with ID starting with 4b8dc7801709dd91aa05ecc3bf700668ec8e338b81e13e70e823985eb543ea1a not found: ID does not exist" Jul 07 18:49:20.402022 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.401986 2583 scope.go:117] "RemoveContainer" containerID="72574e624ed9452ba2a53b958c99337db314e1c97daefe5635213fd09a4a7244" Jul 07 18:49:20.402177 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:49:20.402163 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72574e624ed9452ba2a53b958c99337db314e1c97daefe5635213fd09a4a7244\": container with ID starting with 72574e624ed9452ba2a53b958c99337db314e1c97daefe5635213fd09a4a7244 not found: ID does not exist" containerID="72574e624ed9452ba2a53b958c99337db314e1c97daefe5635213fd09a4a7244" Jul 07 18:49:20.402215 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.402179 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72574e624ed9452ba2a53b958c99337db314e1c97daefe5635213fd09a4a7244"} err="failed to get container status \"72574e624ed9452ba2a53b958c99337db314e1c97daefe5635213fd09a4a7244\": rpc error: code = NotFound desc = could not find container \"72574e624ed9452ba2a53b958c99337db314e1c97daefe5635213fd09a4a7244\": container with ID starting with 72574e624ed9452ba2a53b958c99337db314e1c97daefe5635213fd09a4a7244 not found: ID does not exist" Jul 07 18:49:20.402215 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.402190 2583 scope.go:117] "RemoveContainer" containerID="4a14619d8bd34e0e9bf1a9452fbe038acfd73f281308639e780bfd8967bcccf5" Jul 07 18:49:20.402411 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:49:20.402394 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a14619d8bd34e0e9bf1a9452fbe038acfd73f281308639e780bfd8967bcccf5\": container with ID starting with 4a14619d8bd34e0e9bf1a9452fbe038acfd73f281308639e780bfd8967bcccf5 not found: ID does not exist" containerID="4a14619d8bd34e0e9bf1a9452fbe038acfd73f281308639e780bfd8967bcccf5" Jul 07 18:49:20.402451 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.402418 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a14619d8bd34e0e9bf1a9452fbe038acfd73f281308639e780bfd8967bcccf5"} err="failed to get container status \"4a14619d8bd34e0e9bf1a9452fbe038acfd73f281308639e780bfd8967bcccf5\": rpc error: code = NotFound desc = could not find container \"4a14619d8bd34e0e9bf1a9452fbe038acfd73f281308639e780bfd8967bcccf5\": container with ID starting with 4a14619d8bd34e0e9bf1a9452fbe038acfd73f281308639e780bfd8967bcccf5 not found: ID does not exist" Jul 07 18:49:20.402451 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.402440 2583 scope.go:117] "RemoveContainer" containerID="caa18de2649653321baa3a5b04b697a8287504041e8c52e445242ceeb46baa5e" Jul 07 18:49:20.402648 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.402632 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caa18de2649653321baa3a5b04b697a8287504041e8c52e445242ceeb46baa5e"} err="failed to get container status \"caa18de2649653321baa3a5b04b697a8287504041e8c52e445242ceeb46baa5e\": rpc error: code = NotFound desc = could not find container \"caa18de2649653321baa3a5b04b697a8287504041e8c52e445242ceeb46baa5e\": container with ID starting with caa18de2649653321baa3a5b04b697a8287504041e8c52e445242ceeb46baa5e not found: ID does not exist" Jul 07 18:49:20.402648 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.402648 2583 scope.go:117] "RemoveContainer" containerID="fda5553304a9bc3c27e22321efd6bd5bfcdaf6d5df49f7c353425ed1388f62ec" Jul 07 18:49:20.402844 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.402825 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fda5553304a9bc3c27e22321efd6bd5bfcdaf6d5df49f7c353425ed1388f62ec"} err="failed to get container status \"fda5553304a9bc3c27e22321efd6bd5bfcdaf6d5df49f7c353425ed1388f62ec\": rpc error: code = NotFound desc = could not find container \"fda5553304a9bc3c27e22321efd6bd5bfcdaf6d5df49f7c353425ed1388f62ec\": container with ID starting with fda5553304a9bc3c27e22321efd6bd5bfcdaf6d5df49f7c353425ed1388f62ec not found: ID does not exist" Jul 07 18:49:20.402885 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.402846 2583 scope.go:117] "RemoveContainer" containerID="477aef5dad0f30b94cfdabb463b57562dfc7fda94e329331aa33fadf04999ea5" Jul 07 18:49:20.402994 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.402978 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"477aef5dad0f30b94cfdabb463b57562dfc7fda94e329331aa33fadf04999ea5"} err="failed to get container status \"477aef5dad0f30b94cfdabb463b57562dfc7fda94e329331aa33fadf04999ea5\": rpc error: code = NotFound desc = could not find container \"477aef5dad0f30b94cfdabb463b57562dfc7fda94e329331aa33fadf04999ea5\": container with ID starting with 477aef5dad0f30b94cfdabb463b57562dfc7fda94e329331aa33fadf04999ea5 not found: ID does not exist" Jul 07 18:49:20.403033 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.402994 2583 scope.go:117] "RemoveContainer" containerID="4793e83a06f9be0fda81a9a7b49617efb74760d87986109d05fe1cd3227780ae" Jul 07 18:49:20.403209 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.403188 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4793e83a06f9be0fda81a9a7b49617efb74760d87986109d05fe1cd3227780ae"} err="failed to get container status \"4793e83a06f9be0fda81a9a7b49617efb74760d87986109d05fe1cd3227780ae\": rpc error: code = NotFound desc = could not find container \"4793e83a06f9be0fda81a9a7b49617efb74760d87986109d05fe1cd3227780ae\": container with ID starting with 4793e83a06f9be0fda81a9a7b49617efb74760d87986109d05fe1cd3227780ae not found: ID does not exist" Jul 07 18:49:20.403209 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.403209 2583 scope.go:117] "RemoveContainer" containerID="4b8dc7801709dd91aa05ecc3bf700668ec8e338b81e13e70e823985eb543ea1a" Jul 07 18:49:20.403417 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.403396 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b8dc7801709dd91aa05ecc3bf700668ec8e338b81e13e70e823985eb543ea1a"} err="failed to get container status \"4b8dc7801709dd91aa05ecc3bf700668ec8e338b81e13e70e823985eb543ea1a\": rpc error: code = NotFound desc = could not find container \"4b8dc7801709dd91aa05ecc3bf700668ec8e338b81e13e70e823985eb543ea1a\": container with ID starting with 4b8dc7801709dd91aa05ecc3bf700668ec8e338b81e13e70e823985eb543ea1a not found: ID does not exist" Jul 07 18:49:20.403460 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.403419 2583 scope.go:117] "RemoveContainer" containerID="72574e624ed9452ba2a53b958c99337db314e1c97daefe5635213fd09a4a7244" Jul 07 18:49:20.403613 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.403597 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72574e624ed9452ba2a53b958c99337db314e1c97daefe5635213fd09a4a7244"} err="failed to get container status \"72574e624ed9452ba2a53b958c99337db314e1c97daefe5635213fd09a4a7244\": rpc error: code = NotFound desc = could not find container \"72574e624ed9452ba2a53b958c99337db314e1c97daefe5635213fd09a4a7244\": container with ID starting with 72574e624ed9452ba2a53b958c99337db314e1c97daefe5635213fd09a4a7244 not found: ID does not exist" Jul 07 18:49:20.403651 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.403614 2583 scope.go:117] "RemoveContainer" containerID="4a14619d8bd34e0e9bf1a9452fbe038acfd73f281308639e780bfd8967bcccf5" Jul 07 18:49:20.403790 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.403776 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a14619d8bd34e0e9bf1a9452fbe038acfd73f281308639e780bfd8967bcccf5"} err="failed to get container status \"4a14619d8bd34e0e9bf1a9452fbe038acfd73f281308639e780bfd8967bcccf5\": rpc error: code = NotFound desc = could not find container \"4a14619d8bd34e0e9bf1a9452fbe038acfd73f281308639e780bfd8967bcccf5\": container with ID starting with 4a14619d8bd34e0e9bf1a9452fbe038acfd73f281308639e780bfd8967bcccf5 not found: ID does not exist" Jul 07 18:49:20.429604 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.429554 2583 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/00660473-d5d8-4e8c-a348-4a639947c47b-alertmanager-main-db\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:20.429604 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.429601 2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:20.429604 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.429612 2583 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/00660473-d5d8-4e8c-a348-4a639947c47b-tls-assets\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:20.429604 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.429621 2583 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-web-config\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:20.429874 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.429629 2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-secret-alertmanager-main-tls\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:20.429874 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.429639 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvrsm\" (UniqueName: \"kubernetes.io/projected/00660473-d5d8-4e8c-a348-4a639947c47b-kube-api-access-fvrsm\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:20.429874 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.429648 2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:20.429874 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.429657 2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:20.429874 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.429665 2583 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-cluster-tls-config\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:20.429874 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.429673 2583 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/00660473-d5d8-4e8c-a348-4a639947c47b-config-out\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:20.429874 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.429681 2583 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/00660473-d5d8-4e8c-a348-4a639947c47b-config-volume\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:20.429874 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.429688 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00660473-d5d8-4e8c-a348-4a639947c47b-metrics-client-ca\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:20.659450 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.659419 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 18:49:20.663495 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.663466 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 18:49:20.695596 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.695557 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 18:49:20.695936 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.695918 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6994cdae-7247-4e9c-8c95-5d409cea63bd" containerName="console" Jul 07 18:49:20.696022 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.695938 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6994cdae-7247-4e9c-8c95-5d409cea63bd" containerName="console" Jul 07 18:49:20.696022 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.695954 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00660473-d5d8-4e8c-a348-4a639947c47b" containerName="kube-rbac-proxy-web" Jul 07 18:49:20.696022 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.695964 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="00660473-d5d8-4e8c-a348-4a639947c47b" containerName="kube-rbac-proxy-web" Jul 07 18:49:20.696022 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.695979 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00660473-d5d8-4e8c-a348-4a639947c47b" containerName="prom-label-proxy" Jul 07 18:49:20.696022 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.695988 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="00660473-d5d8-4e8c-a348-4a639947c47b" containerName="prom-label-proxy" Jul 07 18:49:20.696022 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.695998 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37675bf2-2454-4e3c-9c53-9f7ae3517b6a" containerName="registry" Jul 07 18:49:20.696022 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.696007 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="37675bf2-2454-4e3c-9c53-9f7ae3517b6a" containerName="registry" Jul 07 18:49:20.696022 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.696018 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00660473-d5d8-4e8c-a348-4a639947c47b" containerName="alertmanager" Jul 07 18:49:20.696420 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.696027 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="00660473-d5d8-4e8c-a348-4a639947c47b" containerName="alertmanager" Jul 07 18:49:20.696420 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.696036 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00660473-d5d8-4e8c-a348-4a639947c47b" containerName="config-reloader" Jul 07 18:49:20.696420 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.696044 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="00660473-d5d8-4e8c-a348-4a639947c47b" containerName="config-reloader" Jul 07 18:49:20.696420 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.696062 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00660473-d5d8-4e8c-a348-4a639947c47b" containerName="init-config-reloader" Jul 07 18:49:20.696420 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.696070 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="00660473-d5d8-4e8c-a348-4a639947c47b" containerName="init-config-reloader" Jul 07 18:49:20.696420 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.696080 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00660473-d5d8-4e8c-a348-4a639947c47b" containerName="kube-rbac-proxy" Jul 07 18:49:20.696420 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.696089 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="00660473-d5d8-4e8c-a348-4a639947c47b" containerName="kube-rbac-proxy" Jul 07 18:49:20.696420 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.696103 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00660473-d5d8-4e8c-a348-4a639947c47b" containerName="kube-rbac-proxy-metric" Jul 07 18:49:20.696420 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.696111 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="00660473-d5d8-4e8c-a348-4a639947c47b" containerName="kube-rbac-proxy-metric" Jul 07 18:49:20.696420 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.696193 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="00660473-d5d8-4e8c-a348-4a639947c47b" containerName="kube-rbac-proxy" Jul 07 18:49:20.696420 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.696204 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="37675bf2-2454-4e3c-9c53-9f7ae3517b6a" containerName="registry" Jul 07 18:49:20.696420 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.696216 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6994cdae-7247-4e9c-8c95-5d409cea63bd" containerName="console" Jul 07 18:49:20.696420 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.696227 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="00660473-d5d8-4e8c-a348-4a639947c47b" containerName="kube-rbac-proxy-metric" Jul 07 18:49:20.696420 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.696235 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="00660473-d5d8-4e8c-a348-4a639947c47b" containerName="prom-label-proxy" Jul 07 18:49:20.696420 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.696245 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="00660473-d5d8-4e8c-a348-4a639947c47b" containerName="config-reloader" Jul 07 18:49:20.696420 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.696270 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="00660473-d5d8-4e8c-a348-4a639947c47b" containerName="alertmanager" Jul 07 18:49:20.696420 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.696278 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="00660473-d5d8-4e8c-a348-4a639947c47b" containerName="kube-rbac-proxy-web" Jul 07 18:49:20.702481 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.701912 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.704451 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.704427 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 07 18:49:20.704557 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.704475 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 07 18:49:20.704732 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.704694 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 07 18:49:20.704732 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.704694 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-s7fpk\"" Jul 07 18:49:20.704885 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.704836 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 07 18:49:20.704885 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.704855 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 07 18:49:20.704980 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.704913 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 07 18:49:20.704980 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.704959 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 07 18:49:20.704980 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.704969 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 07 18:49:20.710658 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.710630 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 07 18:49:20.714787 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.714767 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 18:49:20.732365 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.732333 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d25d201c-3b25-47f0-b918-c8f3980e2da4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.732365 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.732370 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d25d201c-3b25-47f0-b918-c8f3980e2da4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.732557 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.732391 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g5t4\" (UniqueName: \"kubernetes.io/projected/d25d201c-3b25-47f0-b918-c8f3980e2da4-kube-api-access-6g5t4\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.732557 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.732467 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d25d201c-3b25-47f0-b918-c8f3980e2da4-web-config\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.732557 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.732495 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d25d201c-3b25-47f0-b918-c8f3980e2da4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.732655 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.732566 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d25d201c-3b25-47f0-b918-c8f3980e2da4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.732655 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.732586 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d25d201c-3b25-47f0-b918-c8f3980e2da4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.732655 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.732617 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d25d201c-3b25-47f0-b918-c8f3980e2da4-config-volume\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.732655 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.732639 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d25d201c-3b25-47f0-b918-c8f3980e2da4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.732769 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.732659 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d25d201c-3b25-47f0-b918-c8f3980e2da4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.732769 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.732676 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d25d201c-3b25-47f0-b918-c8f3980e2da4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.732769 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.732697 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d25d201c-3b25-47f0-b918-c8f3980e2da4-config-out\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.732769 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.732738 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d25d201c-3b25-47f0-b918-c8f3980e2da4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.833808 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.833772 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d25d201c-3b25-47f0-b918-c8f3980e2da4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.833808 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.833812 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d25d201c-3b25-47f0-b918-c8f3980e2da4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.834086 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.833944 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d25d201c-3b25-47f0-b918-c8f3980e2da4-config-volume\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.834086 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.833986 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d25d201c-3b25-47f0-b918-c8f3980e2da4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.834086 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.834011 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d25d201c-3b25-47f0-b918-c8f3980e2da4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.834086 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.834034 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d25d201c-3b25-47f0-b918-c8f3980e2da4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.834086 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.834067 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d25d201c-3b25-47f0-b918-c8f3980e2da4-config-out\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.834360 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.834099 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d25d201c-3b25-47f0-b918-c8f3980e2da4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.834489 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.834468 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d25d201c-3b25-47f0-b918-c8f3980e2da4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.835194 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.834609 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d25d201c-3b25-47f0-b918-c8f3980e2da4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.835194 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.834669 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d25d201c-3b25-47f0-b918-c8f3980e2da4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.835194 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.834732 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g5t4\" (UniqueName: \"kubernetes.io/projected/d25d201c-3b25-47f0-b918-c8f3980e2da4-kube-api-access-6g5t4\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.835194 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.834768 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d25d201c-3b25-47f0-b918-c8f3980e2da4-web-config\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.835194 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.834792 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d25d201c-3b25-47f0-b918-c8f3980e2da4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.835194 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.834896 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d25d201c-3b25-47f0-b918-c8f3980e2da4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.836850 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.836820 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d25d201c-3b25-47f0-b918-c8f3980e2da4-config-out\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.836968 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.836885 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d25d201c-3b25-47f0-b918-c8f3980e2da4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.837084 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.837062 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d25d201c-3b25-47f0-b918-c8f3980e2da4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.837199 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.837172 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d25d201c-3b25-47f0-b918-c8f3980e2da4-config-volume\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.837285 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.837225 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d25d201c-3b25-47f0-b918-c8f3980e2da4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.837515 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.837490 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d25d201c-3b25-47f0-b918-c8f3980e2da4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.837596 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.837516 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d25d201c-3b25-47f0-b918-c8f3980e2da4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.837751 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.837732 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d25d201c-3b25-47f0-b918-c8f3980e2da4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.838178 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.838156 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d25d201c-3b25-47f0-b918-c8f3980e2da4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.839047 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.839026 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d25d201c-3b25-47f0-b918-c8f3980e2da4-web-config\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:20.844780 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:20.844759 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g5t4\" (UniqueName: \"kubernetes.io/projected/d25d201c-3b25-47f0-b918-c8f3980e2da4-kube-api-access-6g5t4\") pod \"alertmanager-main-0\" (UID: \"d25d201c-3b25-47f0-b918-c8f3980e2da4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:21.012403 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:21.012307 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:49:21.152986 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:21.152952 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 18:49:21.156830 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:49:21.156799 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd25d201c_3b25_47f0_b918_c8f3980e2da4.slice/crio-231282a547b4e38cda19f25a1a2c2f05a7c265a69c73386f8e909e5a3af41d70 WatchSource:0}: Error finding container 231282a547b4e38cda19f25a1a2c2f05a7c265a69c73386f8e909e5a3af41d70: Status 404 returned error can't find the container with id 231282a547b4e38cda19f25a1a2c2f05a7c265a69c73386f8e909e5a3af41d70 Jul 07 18:49:21.342018 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:21.341981 2583 generic.go:356] "Generic (PLEG): container finished" podID="d25d201c-3b25-47f0-b918-c8f3980e2da4" containerID="699c3a487ba1197ead3a6e16316c8fedcd51f65d2f5d67968afdaa6c1490bba8" exitCode=0 Jul 07 18:49:21.342464 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:21.342042 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d25d201c-3b25-47f0-b918-c8f3980e2da4","Type":"ContainerDied","Data":"699c3a487ba1197ead3a6e16316c8fedcd51f65d2f5d67968afdaa6c1490bba8"} Jul 07 18:49:21.342464 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:21.342120 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d25d201c-3b25-47f0-b918-c8f3980e2da4","Type":"ContainerStarted","Data":"231282a547b4e38cda19f25a1a2c2f05a7c265a69c73386f8e909e5a3af41d70"} Jul 07 18:49:21.416270 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:21.416230 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00660473-d5d8-4e8c-a348-4a639947c47b" path="/var/lib/kubelet/pods/00660473-d5d8-4e8c-a348-4a639947c47b/volumes" Jul 07 18:49:22.350241 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:22.350207 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d25d201c-3b25-47f0-b918-c8f3980e2da4","Type":"ContainerStarted","Data":"c6b1b9720c26c05ae0caa788bc6b3e56646510484a13777f6390ce5ae1ea3a65"} Jul 07 18:49:22.350667 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:22.350303 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d25d201c-3b25-47f0-b918-c8f3980e2da4","Type":"ContainerStarted","Data":"5b42fe353ce41a61388d973be792cc8a8f7b7dc4f534242b3cd7c29f6a54d8fc"} Jul 07 18:49:22.350667 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:22.350333 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d25d201c-3b25-47f0-b918-c8f3980e2da4","Type":"ContainerStarted","Data":"9f94c43ae640ba7735e628b07990a4df4f67fe0a43d76b48972f603fa4a5c9a4"} Jul 07 18:49:22.350667 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:22.350359 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d25d201c-3b25-47f0-b918-c8f3980e2da4","Type":"ContainerStarted","Data":"f67466ecd7bd964a6a221f345e2d827fa9deec5fd6471401c55f9c797876df31"} Jul 07 18:49:22.350667 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:22.350385 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d25d201c-3b25-47f0-b918-c8f3980e2da4","Type":"ContainerStarted","Data":"f8ba62aba7bab9d5dd934d1ae2c372e0253ba713cc32436441d89535f0300ea1"} Jul 07 18:49:22.350667 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:22.350410 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d25d201c-3b25-47f0-b918-c8f3980e2da4","Type":"ContainerStarted","Data":"b4cb2c6d376614c04570a82e3cedbef4daddc86da09e2321fa4918aa3d027223"} Jul 07 18:49:22.373823 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:22.373693 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.359054521 podStartE2EDuration="2.373660961s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.60644ms" isStatefulPod=false podCreationTimestamp="2026-07-07 18:49:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:49:22.372515949 +0000 UTC m=+191.473534548" watchObservedRunningTime="2026-07-07 18:49:22.373660961 +0000 UTC m=+191.474679538" Jul 07 18:49:23.282047 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.282014 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 18:49:23.282357 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.282326 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerName="prometheus" containerID="cri-o://e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571" gracePeriod=600 Jul 07 18:49:23.282452 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.282343 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerName="kube-rbac-proxy" containerID="cri-o://57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7" gracePeriod=600 Jul 07 18:49:23.282452 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.282359 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerName="thanos-sidecar" containerID="cri-o://e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67" gracePeriod=600 Jul 07 18:49:23.282452 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.282399 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerName="kube-rbac-proxy-thanos" containerID="cri-o://3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98" gracePeriod=600 Jul 07 18:49:23.282452 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.282432 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerName="config-reloader" containerID="cri-o://407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b" gracePeriod=600 Jul 07 18:49:23.282650 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.282397 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerName="kube-rbac-proxy-web" containerID="cri-o://8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f" gracePeriod=600 Jul 07 18:49:23.561405 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.561372 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:23.660333 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.660302 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-metrics-client-certs\") pod \"9d6992af-1c11-4cec-8f15-cf57722284c6\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " Jul 07 18:49:23.660333 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.660339 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-k8s-rulefiles-0\") pod \"9d6992af-1c11-4cec-8f15-cf57722284c6\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " Jul 07 18:49:23.660569 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.660364 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-k8s-rulefiles-2\") pod \"9d6992af-1c11-4cec-8f15-cf57722284c6\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " Jul 07 18:49:23.660569 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.660388 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9d6992af-1c11-4cec-8f15-cf57722284c6-config-out\") pod \"9d6992af-1c11-4cec-8f15-cf57722284c6\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " Jul 07 18:49:23.660569 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.660415 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-kube-rbac-proxy\") pod \"9d6992af-1c11-4cec-8f15-cf57722284c6\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " Jul 07 18:49:23.660569 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.660431 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"9d6992af-1c11-4cec-8f15-cf57722284c6\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " Jul 07 18:49:23.660569 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.660450 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"9d6992af-1c11-4cec-8f15-cf57722284c6\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " Jul 07 18:49:23.660569 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.660471 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-prometheus-k8s-tls\") pod \"9d6992af-1c11-4cec-8f15-cf57722284c6\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " Jul 07 18:49:23.660569 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.660511 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-configmap-kubelet-serving-ca-bundle\") pod \"9d6992af-1c11-4cec-8f15-cf57722284c6\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " Jul 07 18:49:23.660569 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.660545 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-config\") pod \"9d6992af-1c11-4cec-8f15-cf57722284c6\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " Jul 07 18:49:23.660993 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.660598 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-k8s-db\") pod \"9d6992af-1c11-4cec-8f15-cf57722284c6\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " Jul 07 18:49:23.660993 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.660626 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-k8s-rulefiles-1\") pod \"9d6992af-1c11-4cec-8f15-cf57722284c6\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " Jul 07 18:49:23.660993 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.660652 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-configmap-serving-certs-ca-bundle\") pod \"9d6992af-1c11-4cec-8f15-cf57722284c6\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " Jul 07 18:49:23.660993 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.660690 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-trusted-ca-bundle\") pod \"9d6992af-1c11-4cec-8f15-cf57722284c6\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " Jul 07 18:49:23.660993 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.660689 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "9d6992af-1c11-4cec-8f15-cf57722284c6" (UID: "9d6992af-1c11-4cec-8f15-cf57722284c6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:49:23.660993 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.660720 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-thanos-prometheus-http-client-file\") pod \"9d6992af-1c11-4cec-8f15-cf57722284c6\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " Jul 07 18:49:23.660993 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.660754 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g54dj\" (UniqueName: \"kubernetes.io/projected/9d6992af-1c11-4cec-8f15-cf57722284c6-kube-api-access-g54dj\") pod \"9d6992af-1c11-4cec-8f15-cf57722284c6\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " Jul 07 18:49:23.660993 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.660780 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9d6992af-1c11-4cec-8f15-cf57722284c6-tls-assets\") pod \"9d6992af-1c11-4cec-8f15-cf57722284c6\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " Jul 07 18:49:23.660993 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.660818 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-configmap-metrics-client-ca\") pod \"9d6992af-1c11-4cec-8f15-cf57722284c6\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " Jul 07 18:49:23.660993 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.660855 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-web-config\") pod \"9d6992af-1c11-4cec-8f15-cf57722284c6\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " Jul 07 18:49:23.660993 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.660882 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-grpc-tls\") pod \"9d6992af-1c11-4cec-8f15-cf57722284c6\" (UID: \"9d6992af-1c11-4cec-8f15-cf57722284c6\") " Jul 07 18:49:23.661544 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.661170 2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:23.662242 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.661971 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "9d6992af-1c11-4cec-8f15-cf57722284c6" (UID: "9d6992af-1c11-4cec-8f15-cf57722284c6"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:49:23.662242 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.662094 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "9d6992af-1c11-4cec-8f15-cf57722284c6" (UID: "9d6992af-1c11-4cec-8f15-cf57722284c6"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:49:23.662242 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.662227 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "9d6992af-1c11-4cec-8f15-cf57722284c6" (UID: "9d6992af-1c11-4cec-8f15-cf57722284c6"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:49:23.662594 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.662533 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "9d6992af-1c11-4cec-8f15-cf57722284c6" (UID: "9d6992af-1c11-4cec-8f15-cf57722284c6"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:49:23.662651 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.662618 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "9d6992af-1c11-4cec-8f15-cf57722284c6" (UID: "9d6992af-1c11-4cec-8f15-cf57722284c6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:49:23.663004 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.662980 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "9d6992af-1c11-4cec-8f15-cf57722284c6" (UID: "9d6992af-1c11-4cec-8f15-cf57722284c6"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:49:23.663144 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.663113 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "9d6992af-1c11-4cec-8f15-cf57722284c6" (UID: "9d6992af-1c11-4cec-8f15-cf57722284c6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:49:23.663216 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.663176 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d6992af-1c11-4cec-8f15-cf57722284c6-config-out" (OuterVolumeSpecName: "config-out") pod "9d6992af-1c11-4cec-8f15-cf57722284c6" (UID: "9d6992af-1c11-4cec-8f15-cf57722284c6"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:49:23.663303 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.663282 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "9d6992af-1c11-4cec-8f15-cf57722284c6" (UID: "9d6992af-1c11-4cec-8f15-cf57722284c6"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:49:23.663627 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.663589 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "9d6992af-1c11-4cec-8f15-cf57722284c6" (UID: "9d6992af-1c11-4cec-8f15-cf57722284c6"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:49:23.664779 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.664742 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "9d6992af-1c11-4cec-8f15-cf57722284c6" (UID: "9d6992af-1c11-4cec-8f15-cf57722284c6"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:49:23.664950 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.664907 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "9d6992af-1c11-4cec-8f15-cf57722284c6" (UID: "9d6992af-1c11-4cec-8f15-cf57722284c6"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:49:23.665065 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.664997 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "9d6992af-1c11-4cec-8f15-cf57722284c6" (UID: "9d6992af-1c11-4cec-8f15-cf57722284c6"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:49:23.665118 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.665094 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d6992af-1c11-4cec-8f15-cf57722284c6-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9d6992af-1c11-4cec-8f15-cf57722284c6" (UID: "9d6992af-1c11-4cec-8f15-cf57722284c6"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:49:23.665118 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.665099 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "9d6992af-1c11-4cec-8f15-cf57722284c6" (UID: "9d6992af-1c11-4cec-8f15-cf57722284c6"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:49:23.665232 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.665213 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-config" (OuterVolumeSpecName: "config") pod "9d6992af-1c11-4cec-8f15-cf57722284c6" (UID: "9d6992af-1c11-4cec-8f15-cf57722284c6"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:49:23.665599 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.665581 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "9d6992af-1c11-4cec-8f15-cf57722284c6" (UID: "9d6992af-1c11-4cec-8f15-cf57722284c6"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:49:23.666492 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.666474 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d6992af-1c11-4cec-8f15-cf57722284c6-kube-api-access-g54dj" (OuterVolumeSpecName: "kube-api-access-g54dj") pod "9d6992af-1c11-4cec-8f15-cf57722284c6" (UID: "9d6992af-1c11-4cec-8f15-cf57722284c6"). InnerVolumeSpecName "kube-api-access-g54dj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:49:23.675201 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.675179 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-web-config" (OuterVolumeSpecName: "web-config") pod "9d6992af-1c11-4cec-8f15-cf57722284c6" (UID: "9d6992af-1c11-4cec-8f15-cf57722284c6"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:49:23.761878 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.761834 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g54dj\" (UniqueName: \"kubernetes.io/projected/9d6992af-1c11-4cec-8f15-cf57722284c6-kube-api-access-g54dj\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:23.761878 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.761870 2583 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9d6992af-1c11-4cec-8f15-cf57722284c6-tls-assets\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:23.761878 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.761883 2583 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-configmap-metrics-client-ca\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:23.762148 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.761896 2583 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-web-config\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:23.762148 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.761909 2583 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-grpc-tls\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:23.762148 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.761920 2583 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-metrics-client-certs\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:23.762148 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.761932 2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:23.762148 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.761943 2583 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9d6992af-1c11-4cec-8f15-cf57722284c6-config-out\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:23.762148 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.761954 2583 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-kube-rbac-proxy\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:23.762148 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.761969 2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:23.762148 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.761983 2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:23.762148 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.761996 2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-secret-prometheus-k8s-tls\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:23.762148 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.762009 2583 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:23.762148 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.762021 2583 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-config\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:23.762148 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.762032 2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-k8s-db\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:23.762148 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.762045 2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:23.762148 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.762058 2583 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:23.762148 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.762070 2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d6992af-1c11-4cec-8f15-cf57722284c6-prometheus-trusted-ca-bundle\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:23.762148 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:23.762082 2583 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9d6992af-1c11-4cec-8f15-cf57722284c6-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:49:24.363192 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.363156 2583 generic.go:356] "Generic (PLEG): container finished" podID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerID="3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98" exitCode=0 Jul 07 18:49:24.363192 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.363183 2583 generic.go:356] "Generic (PLEG): container finished" podID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerID="57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7" exitCode=0 Jul 07 18:49:24.363192 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.363193 2583 generic.go:356] "Generic (PLEG): container finished" podID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerID="8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f" exitCode=0 Jul 07 18:49:24.363192 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.363201 2583 generic.go:356] "Generic (PLEG): container finished" podID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerID="e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67" exitCode=0 Jul 07 18:49:24.363473 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.363208 2583 generic.go:356] "Generic (PLEG): container finished" podID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerID="407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b" exitCode=0 Jul 07 18:49:24.363473 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.363216 2583 generic.go:356] "Generic (PLEG): container finished" podID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerID="e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571" exitCode=0 Jul 07 18:49:24.363473 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.363246 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.363473 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.363238 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d6992af-1c11-4cec-8f15-cf57722284c6","Type":"ContainerDied","Data":"3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98"} Jul 07 18:49:24.363473 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.363420 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d6992af-1c11-4cec-8f15-cf57722284c6","Type":"ContainerDied","Data":"57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7"} Jul 07 18:49:24.363473 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.363467 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d6992af-1c11-4cec-8f15-cf57722284c6","Type":"ContainerDied","Data":"8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f"} Jul 07 18:49:24.363731 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.363487 2583 scope.go:117] "RemoveContainer" containerID="3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98" Jul 07 18:49:24.363731 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.363508 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d6992af-1c11-4cec-8f15-cf57722284c6","Type":"ContainerDied","Data":"e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67"} Jul 07 18:49:24.363731 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.363550 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d6992af-1c11-4cec-8f15-cf57722284c6","Type":"ContainerDied","Data":"407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b"} Jul 07 18:49:24.363731 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.363593 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d6992af-1c11-4cec-8f15-cf57722284c6","Type":"ContainerDied","Data":"e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571"} Jul 07 18:49:24.363731 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.363628 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d6992af-1c11-4cec-8f15-cf57722284c6","Type":"ContainerDied","Data":"f3d330d2a83705feb1dfe5f1e39dea20a1d82ff30c6666a8a964bddc7a321472"} Jul 07 18:49:24.371963 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.371944 2583 scope.go:117] "RemoveContainer" containerID="57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7" Jul 07 18:49:24.379316 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.379294 2583 scope.go:117] "RemoveContainer" containerID="8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f" Jul 07 18:49:24.386219 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.386199 2583 scope.go:117] "RemoveContainer" containerID="e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67" Jul 07 18:49:24.387011 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.386990 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 18:49:24.394174 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.394155 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 18:49:24.412348 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.412326 2583 scope.go:117] "RemoveContainer" containerID="407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b" Jul 07 18:49:24.418455 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.418430 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 18:49:24.418758 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.418744 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerName="thanos-sidecar" Jul 07 18:49:24.418802 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.418759 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerName="thanos-sidecar" Jul 07 18:49:24.418802 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.418768 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerName="kube-rbac-proxy-web" Jul 07 18:49:24.418802 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.418775 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerName="kube-rbac-proxy-web" Jul 07 18:49:24.418802 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.418785 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerName="kube-rbac-proxy" Jul 07 18:49:24.418802 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.418792 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerName="kube-rbac-proxy" Jul 07 18:49:24.418945 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.418807 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerName="init-config-reloader" Jul 07 18:49:24.418945 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.418813 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerName="init-config-reloader" Jul 07 18:49:24.418945 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.418820 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerName="kube-rbac-proxy-thanos" Jul 07 18:49:24.418945 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.418825 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerName="kube-rbac-proxy-thanos" Jul 07 18:49:24.418945 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.418833 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerName="config-reloader" Jul 07 18:49:24.418945 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.418838 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerName="config-reloader" Jul 07 18:49:24.418945 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.418845 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerName="prometheus" Jul 07 18:49:24.418945 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.418850 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerName="prometheus" Jul 07 18:49:24.418945 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.418908 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerName="prometheus" Jul 07 18:49:24.418945 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.418918 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerName="thanos-sidecar" Jul 07 18:49:24.418945 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.418926 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerName="kube-rbac-proxy" Jul 07 18:49:24.418945 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.418937 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerName="config-reloader" Jul 07 18:49:24.419284 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.418947 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerName="kube-rbac-proxy-web" Jul 07 18:49:24.419284 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.418959 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d6992af-1c11-4cec-8f15-cf57722284c6" containerName="kube-rbac-proxy-thanos" Jul 07 18:49:24.421169 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.421153 2583 scope.go:117] "RemoveContainer" containerID="e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571" Jul 07 18:49:24.425810 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.425788 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.428437 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.428411 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 18:49:24.428550 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.428452 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 18:49:24.429140 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.428894 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-7xl68\"" Jul 07 18:49:24.429140 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.428911 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8re03dvljcvg9\"" Jul 07 18:49:24.429140 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.428977 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 18:49:24.429140 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.429003 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 18:49:24.429140 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.429026 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 18:49:24.429140 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.429026 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 18:49:24.429528 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.429315 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 18:49:24.429528 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.429319 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 18:49:24.429694 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.429657 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 18:49:24.429694 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.429690 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 18:49:24.429848 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.429690 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 18:49:24.429909 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.429884 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 18:49:24.429961 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.429885 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 18:49:24.431040 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.431022 2583 scope.go:117] "RemoveContainer" containerID="7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459" Jul 07 18:49:24.433018 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.432999 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 18:49:24.435813 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.435797 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 18:49:24.439869 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.439852 2583 scope.go:117] "RemoveContainer" containerID="3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98" Jul 07 18:49:24.440175 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:49:24.440153 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98\": container with ID starting with 3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98 not found: ID does not exist" containerID="3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98" Jul 07 18:49:24.440280 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.440183 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98"} err="failed to get container status \"3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98\": rpc error: code = NotFound desc = could not find container \"3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98\": container with ID starting with 3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98 not found: ID does not exist" Jul 07 18:49:24.440280 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.440203 2583 scope.go:117] "RemoveContainer" containerID="57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7" Jul 07 18:49:24.440679 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:49:24.440659 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7\": container with ID starting with 57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7 not found: ID does not exist" containerID="57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7" Jul 07 18:49:24.440767 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.440683 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7"} err="failed to get container status \"57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7\": rpc error: code = NotFound desc = could not find container \"57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7\": container with ID starting with 57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7 not found: ID does not exist" Jul 07 18:49:24.440767 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.440698 2583 scope.go:117] "RemoveContainer" containerID="8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f" Jul 07 18:49:24.440991 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:49:24.440969 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f\": container with ID starting with 8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f not found: ID does not exist" containerID="8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f" Jul 07 18:49:24.441047 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.440998 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f"} err="failed to get container status \"8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f\": rpc error: code = NotFound desc = could not find container \"8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f\": container with ID starting with 8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f not found: ID does not exist" Jul 07 18:49:24.441047 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.441019 2583 scope.go:117] "RemoveContainer" containerID="e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67" Jul 07 18:49:24.441326 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:49:24.441301 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67\": container with ID starting with e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67 not found: ID does not exist" containerID="e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67" Jul 07 18:49:24.441391 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.441334 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67"} err="failed to get container status \"e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67\": rpc error: code = NotFound desc = could not find container \"e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67\": container with ID starting with e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67 not found: ID does not exist" Jul 07 18:49:24.441391 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.441357 2583 scope.go:117] "RemoveContainer" containerID="407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b" Jul 07 18:49:24.441620 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:49:24.441592 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b\": container with ID starting with 407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b not found: ID does not exist" containerID="407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b" Jul 07 18:49:24.441682 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.441624 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b"} err="failed to get container status \"407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b\": rpc error: code = NotFound desc = could not find container \"407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b\": container with ID starting with 407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b not found: ID does not exist" Jul 07 18:49:24.441682 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.441637 2583 scope.go:117] "RemoveContainer" containerID="e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571" Jul 07 18:49:24.441949 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:49:24.441917 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571\": container with ID starting with e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571 not found: ID does not exist" containerID="e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571" Jul 07 18:49:24.442009 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.441956 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571"} err="failed to get container status \"e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571\": rpc error: code = NotFound desc = could not find container \"e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571\": container with ID starting with e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571 not found: ID does not exist" Jul 07 18:49:24.442009 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.441976 2583 scope.go:117] "RemoveContainer" containerID="7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459" Jul 07 18:49:24.442283 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:49:24.442245 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459\": container with ID starting with 7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459 not found: ID does not exist" containerID="7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459" Jul 07 18:49:24.442352 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.442292 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459"} err="failed to get container status \"7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459\": rpc error: code = NotFound desc = could not find container \"7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459\": container with ID starting with 7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459 not found: ID does not exist" Jul 07 18:49:24.442352 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.442309 2583 scope.go:117] "RemoveContainer" containerID="3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98" Jul 07 18:49:24.442562 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.442539 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98"} err="failed to get container status \"3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98\": rpc error: code = NotFound desc = could not find container \"3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98\": container with ID starting with 3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98 not found: ID does not exist" Jul 07 18:49:24.442620 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.442567 2583 scope.go:117] "RemoveContainer" containerID="57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7" Jul 07 18:49:24.443042 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.442925 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7"} err="failed to get container status \"57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7\": rpc error: code = NotFound desc = could not find container \"57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7\": container with ID starting with 57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7 not found: ID does not exist" Jul 07 18:49:24.443042 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.442949 2583 scope.go:117] "RemoveContainer" containerID="8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f" Jul 07 18:49:24.443247 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.443198 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f"} err="failed to get container status \"8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f\": rpc error: code = NotFound desc = could not find container \"8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f\": container with ID starting with 8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f not found: ID does not exist" Jul 07 18:49:24.443247 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.443217 2583 scope.go:117] "RemoveContainer" containerID="e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67" Jul 07 18:49:24.443556 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.443528 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67"} err="failed to get container status \"e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67\": rpc error: code = NotFound desc = could not find container \"e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67\": container with ID starting with e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67 not found: ID does not exist" Jul 07 18:49:24.443556 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.443552 2583 scope.go:117] "RemoveContainer" containerID="407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b" Jul 07 18:49:24.443823 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.443785 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b"} err="failed to get container status \"407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b\": rpc error: code = NotFound desc = could not find container \"407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b\": container with ID starting with 407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b not found: ID does not exist" Jul 07 18:49:24.443823 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.443815 2583 scope.go:117] "RemoveContainer" containerID="e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571" Jul 07 18:49:24.444549 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.444524 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571"} err="failed to get container status \"e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571\": rpc error: code = NotFound desc = could not find container \"e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571\": container with ID starting with e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571 not found: ID does not exist" Jul 07 18:49:24.444625 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.444550 2583 scope.go:117] "RemoveContainer" containerID="7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459" Jul 07 18:49:24.444805 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.444781 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459"} err="failed to get container status \"7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459\": rpc error: code = NotFound desc = could not find container \"7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459\": container with ID starting with 7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459 not found: ID does not exist" Jul 07 18:49:24.444874 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.444807 2583 scope.go:117] "RemoveContainer" containerID="3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98" Jul 07 18:49:24.445045 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.445028 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98"} err="failed to get container status \"3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98\": rpc error: code = NotFound desc = could not find container \"3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98\": container with ID starting with 3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98 not found: ID does not exist" Jul 07 18:49:24.445101 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.445045 2583 scope.go:117] "RemoveContainer" containerID="57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7" Jul 07 18:49:24.445305 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.445284 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7"} err="failed to get container status \"57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7\": rpc error: code = NotFound desc = could not find container \"57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7\": container with ID starting with 57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7 not found: ID does not exist" Jul 07 18:49:24.445390 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.445306 2583 scope.go:117] "RemoveContainer" containerID="8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f" Jul 07 18:49:24.445552 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.445535 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f"} err="failed to get container status \"8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f\": rpc error: code = NotFound desc = could not find container \"8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f\": container with ID starting with 8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f not found: ID does not exist" Jul 07 18:49:24.445629 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.445554 2583 scope.go:117] "RemoveContainer" containerID="e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67" Jul 07 18:49:24.445794 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.445775 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67"} err="failed to get container status \"e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67\": rpc error: code = NotFound desc = could not find container \"e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67\": container with ID starting with e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67 not found: ID does not exist" Jul 07 18:49:24.445868 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.445796 2583 scope.go:117] "RemoveContainer" containerID="407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b" Jul 07 18:49:24.446041 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.446020 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b"} err="failed to get container status \"407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b\": rpc error: code = NotFound desc = could not find container \"407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b\": container with ID starting with 407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b not found: ID does not exist" Jul 07 18:49:24.446041 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.446040 2583 scope.go:117] "RemoveContainer" containerID="e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571" Jul 07 18:49:24.446289 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.446250 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571"} err="failed to get container status \"e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571\": rpc error: code = NotFound desc = could not find container \"e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571\": container with ID starting with e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571 not found: ID does not exist" Jul 07 18:49:24.446289 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.446289 2583 scope.go:117] "RemoveContainer" containerID="7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459" Jul 07 18:49:24.446537 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.446517 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 18:49:24.446607 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.446521 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459"} err="failed to get container status \"7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459\": rpc error: code = NotFound desc = could not find container \"7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459\": container with ID starting with 7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459 not found: ID does not exist" Jul 07 18:49:24.446668 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.446610 2583 scope.go:117] "RemoveContainer" containerID="3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98" Jul 07 18:49:24.446993 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.446971 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98"} err="failed to get container status \"3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98\": rpc error: code = NotFound desc = could not find container \"3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98\": container with ID starting with 3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98 not found: ID does not exist" Jul 07 18:49:24.446993 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.446993 2583 scope.go:117] "RemoveContainer" containerID="57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7" Jul 07 18:49:24.447226 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.447209 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7"} err="failed to get container status \"57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7\": rpc error: code = NotFound desc = could not find container \"57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7\": container with ID starting with 57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7 not found: ID does not exist" Jul 07 18:49:24.447315 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.447227 2583 scope.go:117] "RemoveContainer" containerID="8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f" Jul 07 18:49:24.447511 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.447492 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f"} err="failed to get container status \"8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f\": rpc error: code = NotFound desc = could not find container \"8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f\": container with ID starting with 8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f not found: ID does not exist" Jul 07 18:49:24.447561 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.447512 2583 scope.go:117] "RemoveContainer" containerID="e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67" Jul 07 18:49:24.447744 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.447727 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67"} err="failed to get container status \"e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67\": rpc error: code = NotFound desc = could not find container \"e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67\": container with ID starting with e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67 not found: ID does not exist" Jul 07 18:49:24.447744 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.447743 2583 scope.go:117] "RemoveContainer" containerID="407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b" Jul 07 18:49:24.447956 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.447935 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b"} err="failed to get container status \"407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b\": rpc error: code = NotFound desc = could not find container \"407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b\": container with ID starting with 407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b not found: ID does not exist" Jul 07 18:49:24.448029 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.447957 2583 scope.go:117] "RemoveContainer" containerID="e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571" Jul 07 18:49:24.448368 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.448229 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571"} err="failed to get container status \"e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571\": rpc error: code = NotFound desc = could not find container \"e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571\": container with ID starting with e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571 not found: ID does not exist" Jul 07 18:49:24.448368 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.448293 2583 scope.go:117] "RemoveContainer" containerID="7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459" Jul 07 18:49:24.448636 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.448616 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459"} err="failed to get container status \"7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459\": rpc error: code = NotFound desc = could not find container \"7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459\": container with ID starting with 7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459 not found: ID does not exist" Jul 07 18:49:24.448696 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.448637 2583 scope.go:117] "RemoveContainer" containerID="3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98" Jul 07 18:49:24.448869 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.448848 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98"} err="failed to get container status \"3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98\": rpc error: code = NotFound desc = could not find container \"3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98\": container with ID starting with 3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98 not found: ID does not exist" Jul 07 18:49:24.448922 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.448869 2583 scope.go:117] "RemoveContainer" containerID="57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7" Jul 07 18:49:24.449115 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.449095 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7"} err="failed to get container status \"57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7\": rpc error: code = NotFound desc = could not find container \"57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7\": container with ID starting with 57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7 not found: ID does not exist" Jul 07 18:49:24.449190 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.449117 2583 scope.go:117] "RemoveContainer" containerID="8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f" Jul 07 18:49:24.449348 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.449327 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f"} err="failed to get container status \"8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f\": rpc error: code = NotFound desc = could not find container \"8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f\": container with ID starting with 8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f not found: ID does not exist" Jul 07 18:49:24.449403 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.449349 2583 scope.go:117] "RemoveContainer" containerID="e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67" Jul 07 18:49:24.449569 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.449552 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67"} err="failed to get container status \"e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67\": rpc error: code = NotFound desc = could not find container \"e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67\": container with ID starting with e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67 not found: ID does not exist" Jul 07 18:49:24.449626 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.449569 2583 scope.go:117] "RemoveContainer" containerID="407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b" Jul 07 18:49:24.449785 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.449764 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b"} err="failed to get container status \"407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b\": rpc error: code = NotFound desc = could not find container \"407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b\": container with ID starting with 407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b not found: ID does not exist" Jul 07 18:49:24.449860 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.449787 2583 scope.go:117] "RemoveContainer" containerID="e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571" Jul 07 18:49:24.450040 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.450025 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571"} err="failed to get container status \"e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571\": rpc error: code = NotFound desc = could not find container \"e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571\": container with ID starting with e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571 not found: ID does not exist" Jul 07 18:49:24.450082 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.450040 2583 scope.go:117] "RemoveContainer" containerID="7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459" Jul 07 18:49:24.450248 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.450230 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459"} err="failed to get container status \"7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459\": rpc error: code = NotFound desc = could not find container \"7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459\": container with ID starting with 7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459 not found: ID does not exist" Jul 07 18:49:24.450314 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.450248 2583 scope.go:117] "RemoveContainer" containerID="3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98" Jul 07 18:49:24.450497 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.450480 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98"} err="failed to get container status \"3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98\": rpc error: code = NotFound desc = could not find container \"3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98\": container with ID starting with 3bb10459639e13fe90ee30e96786d03730ec0a3067f2396ba6047b2282387f98 not found: ID does not exist" Jul 07 18:49:24.450537 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.450497 2583 scope.go:117] "RemoveContainer" containerID="57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7" Jul 07 18:49:24.450720 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.450701 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7"} err="failed to get container status \"57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7\": rpc error: code = NotFound desc = could not find container \"57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7\": container with ID starting with 57e1545c51200e13604d9e682714d783e6607bff60b71fa777e8eb252f4fc7d7 not found: ID does not exist" Jul 07 18:49:24.450763 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.450721 2583 scope.go:117] "RemoveContainer" containerID="8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f" Jul 07 18:49:24.450912 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.450894 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f"} err="failed to get container status \"8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f\": rpc error: code = NotFound desc = could not find container \"8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f\": container with ID starting with 8cc3266d369a2232518649721e41eb2ea0018871266725e9dfb8c94997bd859f not found: ID does not exist" Jul 07 18:49:24.450953 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.450911 2583 scope.go:117] "RemoveContainer" containerID="e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67" Jul 07 18:49:24.451099 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.451083 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67"} err="failed to get container status \"e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67\": rpc error: code = NotFound desc = could not find container \"e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67\": container with ID starting with e6380ac03f60f546e70396ec55f86106173aa1f6f2ee3ae7fb99fe7a4b288a67 not found: ID does not exist" Jul 07 18:49:24.451140 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.451099 2583 scope.go:117] "RemoveContainer" containerID="407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b" Jul 07 18:49:24.451299 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.451284 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b"} err="failed to get container status \"407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b\": rpc error: code = NotFound desc = could not find container \"407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b\": container with ID starting with 407c317d3f9b03733a2d4f97cabb9907c2cdd99a329edbdfa4d0153a00e3e91b not found: ID does not exist" Jul 07 18:49:24.451338 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.451299 2583 scope.go:117] "RemoveContainer" containerID="e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571" Jul 07 18:49:24.451475 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.451460 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571"} err="failed to get container status \"e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571\": rpc error: code = NotFound desc = could not find container \"e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571\": container with ID starting with e7b9e1e79daa6114f1fe3a924f96437c891d955f60115121e83999cf99a9a571 not found: ID does not exist" Jul 07 18:49:24.451517 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.451474 2583 scope.go:117] "RemoveContainer" containerID="7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459" Jul 07 18:49:24.451671 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.451657 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459"} err="failed to get container status \"7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459\": rpc error: code = NotFound desc = could not find container \"7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459\": container with ID starting with 7b41ce7daf7b2fb787c8fab314920550b13b669036c755a3b93b7fdd0bc74459 not found: ID does not exist" Jul 07 18:49:24.466090 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.466067 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bc446e38-4de2-40c0-a548-0f306024a7b4-web-config\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.466090 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.466095 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh5lq\" (UniqueName: \"kubernetes.io/projected/bc446e38-4de2-40c0-a548-0f306024a7b4-kube-api-access-bh5lq\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.466293 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.466113 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bc446e38-4de2-40c0-a548-0f306024a7b4-config-out\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.466293 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.466131 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bc446e38-4de2-40c0-a548-0f306024a7b4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.466293 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.466152 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bc446e38-4de2-40c0-a548-0f306024a7b4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.466293 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.466192 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bc446e38-4de2-40c0-a548-0f306024a7b4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.466464 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.466316 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bc446e38-4de2-40c0-a548-0f306024a7b4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.466464 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.466364 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bc446e38-4de2-40c0-a548-0f306024a7b4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.466464 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.466397 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bc446e38-4de2-40c0-a548-0f306024a7b4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.466464 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.466415 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bc446e38-4de2-40c0-a548-0f306024a7b4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.466825 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.466532 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc446e38-4de2-40c0-a548-0f306024a7b4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.466825 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.466581 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc446e38-4de2-40c0-a548-0f306024a7b4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.466825 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.466631 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bc446e38-4de2-40c0-a548-0f306024a7b4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.466825 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.466661 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bc446e38-4de2-40c0-a548-0f306024a7b4-config\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.467034 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.466892 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc446e38-4de2-40c0-a548-0f306024a7b4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.467034 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.466926 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bc446e38-4de2-40c0-a548-0f306024a7b4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.467133 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.467087 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc446e38-4de2-40c0-a548-0f306024a7b4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.467184 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.467127 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bc446e38-4de2-40c0-a548-0f306024a7b4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.467184 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.467160 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bc446e38-4de2-40c0-a548-0f306024a7b4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.467286 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.467195 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bc446e38-4de2-40c0-a548-0f306024a7b4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.567958 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.567924 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bc446e38-4de2-40c0-a548-0f306024a7b4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.567958 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.567967 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bc446e38-4de2-40c0-a548-0f306024a7b4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.568493 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.567993 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bc446e38-4de2-40c0-a548-0f306024a7b4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.568493 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.568010 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bc446e38-4de2-40c0-a548-0f306024a7b4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.568493 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.568153 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc446e38-4de2-40c0-a548-0f306024a7b4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.568493 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.568199 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc446e38-4de2-40c0-a548-0f306024a7b4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.568493 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.568220 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bc446e38-4de2-40c0-a548-0f306024a7b4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.568493 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.568248 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bc446e38-4de2-40c0-a548-0f306024a7b4-config\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.568493 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.568290 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc446e38-4de2-40c0-a548-0f306024a7b4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.568493 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.568316 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bc446e38-4de2-40c0-a548-0f306024a7b4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.568493 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.568408 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bc446e38-4de2-40c0-a548-0f306024a7b4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.569011 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.568982 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc446e38-4de2-40c0-a548-0f306024a7b4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.569078 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.569040 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc446e38-4de2-40c0-a548-0f306024a7b4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.569132 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.569078 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bc446e38-4de2-40c0-a548-0f306024a7b4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.569132 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.569109 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bc446e38-4de2-40c0-a548-0f306024a7b4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.569229 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.569139 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bc446e38-4de2-40c0-a548-0f306024a7b4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.569229 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.569175 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bc446e38-4de2-40c0-a548-0f306024a7b4-web-config\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.569229 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.569199 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh5lq\" (UniqueName: \"kubernetes.io/projected/bc446e38-4de2-40c0-a548-0f306024a7b4-kube-api-access-bh5lq\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.569416 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.569226 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bc446e38-4de2-40c0-a548-0f306024a7b4-config-out\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.569416 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.569280 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bc446e38-4de2-40c0-a548-0f306024a7b4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.569416 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.569295 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc446e38-4de2-40c0-a548-0f306024a7b4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.569416 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.569308 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bc446e38-4de2-40c0-a548-0f306024a7b4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.569416 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.569334 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bc446e38-4de2-40c0-a548-0f306024a7b4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.569667 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.569484 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bc446e38-4de2-40c0-a548-0f306024a7b4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.571464 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.570564 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc446e38-4de2-40c0-a548-0f306024a7b4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.571464 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.570866 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc446e38-4de2-40c0-a548-0f306024a7b4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.571464 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.571064 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bc446e38-4de2-40c0-a548-0f306024a7b4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.572756 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.571936 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bc446e38-4de2-40c0-a548-0f306024a7b4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.572756 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.572281 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bc446e38-4de2-40c0-a548-0f306024a7b4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.572756 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.572695 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/bc446e38-4de2-40c0-a548-0f306024a7b4-config\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.572962 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.572781 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bc446e38-4de2-40c0-a548-0f306024a7b4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.573021 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.572960 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bc446e38-4de2-40c0-a548-0f306024a7b4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.573214 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.573187 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bc446e38-4de2-40c0-a548-0f306024a7b4-config-out\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.573462 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.573443 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bc446e38-4de2-40c0-a548-0f306024a7b4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.573684 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.573667 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bc446e38-4de2-40c0-a548-0f306024a7b4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.573751 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.573719 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bc446e38-4de2-40c0-a548-0f306024a7b4-web-config\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.573915 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.573900 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bc446e38-4de2-40c0-a548-0f306024a7b4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.574241 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.574224 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bc446e38-4de2-40c0-a548-0f306024a7b4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.574365 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.574320 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bc446e38-4de2-40c0-a548-0f306024a7b4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.580238 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.580219 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh5lq\" (UniqueName: \"kubernetes.io/projected/bc446e38-4de2-40c0-a548-0f306024a7b4-kube-api-access-bh5lq\") pod \"prometheus-k8s-0\" (UID: \"bc446e38-4de2-40c0-a548-0f306024a7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.736886 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.736848 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:24.876352 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:24.876326 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 18:49:24.877845 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:49:24.877817 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc446e38_4de2_40c0_a548_0f306024a7b4.slice/crio-04faec6521ff5582355dd93a882f16951cbd9ef85090a63ab4a6c4797e758388 WatchSource:0}: Error finding container 04faec6521ff5582355dd93a882f16951cbd9ef85090a63ab4a6c4797e758388: Status 404 returned error can't find the container with id 04faec6521ff5582355dd93a882f16951cbd9ef85090a63ab4a6c4797e758388 Jul 07 18:49:25.369431 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:25.369396 2583 generic.go:356] "Generic (PLEG): container finished" podID="bc446e38-4de2-40c0-a548-0f306024a7b4" containerID="6a12642d8b6f041452903edd2893d4c5a7766b819625e8393d24edb3f002831c" exitCode=0 Jul 07 18:49:25.369626 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:25.369488 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc446e38-4de2-40c0-a548-0f306024a7b4","Type":"ContainerDied","Data":"6a12642d8b6f041452903edd2893d4c5a7766b819625e8393d24edb3f002831c"} Jul 07 18:49:25.369626 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:25.369606 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc446e38-4de2-40c0-a548-0f306024a7b4","Type":"ContainerStarted","Data":"04faec6521ff5582355dd93a882f16951cbd9ef85090a63ab4a6c4797e758388"} Jul 07 18:49:25.415185 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:25.414978 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d6992af-1c11-4cec-8f15-cf57722284c6" path="/var/lib/kubelet/pods/9d6992af-1c11-4cec-8f15-cf57722284c6/volumes" Jul 07 18:49:26.380611 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:26.380577 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc446e38-4de2-40c0-a548-0f306024a7b4","Type":"ContainerStarted","Data":"dd8b5946fa29fe4a22bd778c29207c8658e8b769dc0789e67b02e2303c4f675f"} Jul 07 18:49:26.380994 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:26.380658 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc446e38-4de2-40c0-a548-0f306024a7b4","Type":"ContainerStarted","Data":"d96c6e6829f489172fa6d97b4cae707289d473190c497367d0891c6510b3012d"} Jul 07 18:49:26.380994 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:26.380686 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc446e38-4de2-40c0-a548-0f306024a7b4","Type":"ContainerStarted","Data":"15f7fd6d88793f5b7de8ddf2d71c6841a9aa698465c406cb79f7d41ab8e668dd"} Jul 07 18:49:26.380994 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:26.380716 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc446e38-4de2-40c0-a548-0f306024a7b4","Type":"ContainerStarted","Data":"fbd6eb4196b4e1b757b39fba269da332dd99bae90b7e4a12324c41c17244c266"} Jul 07 18:49:26.380994 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:26.380761 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc446e38-4de2-40c0-a548-0f306024a7b4","Type":"ContainerStarted","Data":"b03543601b26951f60bd3324bff50bce63fd9cf5482e8c56a4f9550394dcf2a6"} Jul 07 18:49:26.380994 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:26.380788 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bc446e38-4de2-40c0-a548-0f306024a7b4","Type":"ContainerStarted","Data":"f9eb9a4653fa039393a67529bc96519dfe56743a3e252caa49851feb3fe8b7e4"} Jul 07 18:49:26.409270 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:26.409188 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.3907526199999998 podStartE2EDuration="2.409168507s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.415887ms" isStatefulPod=false podCreationTimestamp="2026-07-07 18:49:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:49:26.407615206 +0000 UTC m=+195.508633784" watchObservedRunningTime="2026-07-07 18:49:26.409168507 +0000 UTC m=+195.510187083" Jul 07 18:49:27.289250 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:27.289210 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:49:27.348554 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:27.348516 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:49:27.410559 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:27.410518 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:49:28.008316 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:28.008283 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:49:28.061598 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:28.061568 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:49:29.737626 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:29.737586 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:37.827585 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:37.827543 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:49:37.928381 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:37.928344 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:49:37.984447 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:37.984416 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:49:51.315199 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:49:51.315165 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:13.800305 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:50:13.800248 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:13.856188 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:50:13.856136 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:24.737243 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:50:24.737200 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:50:24.752924 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:50:24.752897 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:50:25.633079 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:50:25.633052 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:50:33.174503 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:50:33.174423 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:33.238994 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:50:33.238963 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:35.765808 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:50:35.765777 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:36.482910 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:50:36.482868 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:36.532765 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:50:36.532735 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:36.626279 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:50:36.626227 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:36.677655 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:50:36.677619 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:37.225182 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:50:37.225150 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:45.117130 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:50:45.117091 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:45.172305 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:50:45.172242 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:47.081790 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:50:47.081758 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:47.183205 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:50:47.183169 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:47.243767 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:50:47.243733 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:56.918033 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:50:56.917989 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:57.015761 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:50:57.015727 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:57.085240 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:50:57.085209 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:51:03.311528 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:51:03.311496 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:51:11.260275 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:51:11.260222 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-wszbl_363d866e-b0a1-4351-8f85-b994a18200a7/cluster-monitoring-operator/1.log" Jul 07 18:51:11.262306 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:51:11.262286 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5x9w8_f390fdec-e425-4d23-a559-9681817515ec/console-operator/1.log" Jul 07 18:51:11.266534 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:51:11.266512 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9j7xz_bff07fe8-5476-4202-bd92-39b4a2ab6d8c/ovn-acl-logging/0.log" Jul 07 18:51:11.267224 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:51:11.267206 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-wszbl_363d866e-b0a1-4351-8f85-b994a18200a7/cluster-monitoring-operator/1.log" Jul 07 18:51:11.269384 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:51:11.269366 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5x9w8_f390fdec-e425-4d23-a559-9681817515ec/console-operator/1.log" Jul 07 18:51:11.272446 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:51:11.272424 2583 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 18:51:11.274300 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:51:11.274280 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9j7xz_bff07fe8-5476-4202-bd92-39b4a2ab6d8c/ovn-acl-logging/0.log" Jul 07 18:52:10.784223 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:52:10.784140 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:52:11.898536 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:52:11.898499 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:52:32.874398 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:52:32.874366 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:52:32.965637 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:52:32.965598 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:52:39.398639 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:52:39.398603 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:52:39.708279 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:52:39.708171 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:52:39.805435 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:52:39.805400 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:52:40.624184 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:52:40.624156 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:52:40.735471 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:52:40.735439 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:52:40.890598 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:52:40.890504 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:52:41.017004 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:52:41.016972 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:52:41.394957 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:52:41.394925 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:52:41.491074 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:52:41.491039 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:52:41.588770 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:52:41.588735 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:52:41.795763 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:52:41.795681 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:52:41.893039 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:52:41.893005 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:53:38.923792 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:53:38.923698 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:53:39.023499 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:53:39.023467 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:53:41.126819 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:53:41.126785 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:53:41.230383 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:53:41.230353 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:53:41.334574 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:53:41.334529 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:53:41.443909 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:53:41.443810 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:53:41.542409 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:53:41.542380 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:53:41.605692 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:53:41.605660 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:54:17.532799 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:17.532764 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:54:17.633090 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:17.633057 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:54:17.800739 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:17.800646 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:54:17.896208 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:17.896171 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:54:31.877622 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:31.877584 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:54:31.943016 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:31.942975 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:54:43.082684 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:43.082655 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:54:43.183611 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:43.183575 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:54:43.255596 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:43.255564 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:54:49.322827 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:49.322794 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4bn7v/must-gather-rplh6"] Jul 07 18:54:49.326123 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:49.326108 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4bn7v/must-gather-rplh6" Jul 07 18:54:49.328354 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:49.328329 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:54:49.328480 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:49.328441 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4bn7v\"/\"openshift-service-ca.crt\"" Jul 07 18:54:49.328480 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:49.328445 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4bn7v\"/\"kube-root-ca.crt\"" Jul 07 18:54:49.329168 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:49.329149 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4bn7v\"/\"default-dockercfg-2cltl\"" Jul 07 18:54:49.333354 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:49.333328 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4bn7v/must-gather-rplh6"] Jul 07 18:54:49.384717 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:49.384685 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:54:49.470848 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:49.470814 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r86sh\" (UniqueName: \"kubernetes.io/projected/d663a7bc-4487-4d5c-9367-2ba9440f10df-kube-api-access-r86sh\") pod \"must-gather-rplh6\" (UID: \"d663a7bc-4487-4d5c-9367-2ba9440f10df\") " pod="openshift-must-gather-4bn7v/must-gather-rplh6" Jul 07 18:54:49.471020 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:49.470862 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d663a7bc-4487-4d5c-9367-2ba9440f10df-must-gather-output\") pod \"must-gather-rplh6\" (UID: \"d663a7bc-4487-4d5c-9367-2ba9440f10df\") " pod="openshift-must-gather-4bn7v/must-gather-rplh6" Jul 07 18:54:49.571658 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:49.571628 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r86sh\" (UniqueName: \"kubernetes.io/projected/d663a7bc-4487-4d5c-9367-2ba9440f10df-kube-api-access-r86sh\") pod \"must-gather-rplh6\" (UID: \"d663a7bc-4487-4d5c-9367-2ba9440f10df\") " pod="openshift-must-gather-4bn7v/must-gather-rplh6" Jul 07 18:54:49.571835 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:49.571676 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d663a7bc-4487-4d5c-9367-2ba9440f10df-must-gather-output\") pod \"must-gather-rplh6\" (UID: \"d663a7bc-4487-4d5c-9367-2ba9440f10df\") " pod="openshift-must-gather-4bn7v/must-gather-rplh6" Jul 07 18:54:49.572037 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:49.572017 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d663a7bc-4487-4d5c-9367-2ba9440f10df-must-gather-output\") pod \"must-gather-rplh6\" (UID: \"d663a7bc-4487-4d5c-9367-2ba9440f10df\") " pod="openshift-must-gather-4bn7v/must-gather-rplh6" Jul 07 18:54:49.579561 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:49.579501 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r86sh\" (UniqueName: \"kubernetes.io/projected/d663a7bc-4487-4d5c-9367-2ba9440f10df-kube-api-access-r86sh\") pod \"must-gather-rplh6\" (UID: \"d663a7bc-4487-4d5c-9367-2ba9440f10df\") " pod="openshift-must-gather-4bn7v/must-gather-rplh6" Jul 07 18:54:49.651959 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:49.651923 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4bn7v/must-gather-rplh6" Jul 07 18:54:49.779333 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:49.779227 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4bn7v/must-gather-rplh6"] Jul 07 18:54:49.782050 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:54:49.782015 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd663a7bc_4487_4d5c_9367_2ba9440f10df.slice/crio-0cd25d381469fe8f7cebd228ebe8e21770b0bd94d296f90bcf85bae7806ec5e2 WatchSource:0}: Error finding container 0cd25d381469fe8f7cebd228ebe8e21770b0bd94d296f90bcf85bae7806ec5e2: Status 404 returned error can't find the container with id 0cd25d381469fe8f7cebd228ebe8e21770b0bd94d296f90bcf85bae7806ec5e2 Jul 07 18:54:49.783863 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:49.783847 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 18:54:50.651180 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:50.651115 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4bn7v/must-gather-rplh6" event={"ID":"d663a7bc-4487-4d5c-9367-2ba9440f10df","Type":"ContainerStarted","Data":"0cd25d381469fe8f7cebd228ebe8e21770b0bd94d296f90bcf85bae7806ec5e2"} Jul 07 18:54:56.682759 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:56.682722 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4bn7v/must-gather-rplh6" event={"ID":"d663a7bc-4487-4d5c-9367-2ba9440f10df","Type":"ContainerStarted","Data":"5806278ada00e0a73ad9fed77ee26dc41e709096e5204f0269fc3b2ca4fe1059"} Jul 07 18:54:56.682759 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:56.682766 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4bn7v/must-gather-rplh6" event={"ID":"d663a7bc-4487-4d5c-9367-2ba9440f10df","Type":"ContainerStarted","Data":"ffd9f9046dcd234a64d09eefed1778fcc357c9b75672292bcf1b0a5454459a48"} Jul 07 18:54:56.699207 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:54:56.699156 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4bn7v/must-gather-rplh6" podStartSLOduration=1.77692355 podStartE2EDuration="7.699140478s" totalImagesPullingTime="5.922216928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:54:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:54:56.697596465 +0000 UTC m=+525.798615039" watchObservedRunningTime="2026-07-07 18:54:56.699140478 +0000 UTC m=+525.800159095" Jul 07 18:55:06.290995 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:06.290909 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:55:09.737586 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:09.737556 2583 generic.go:356] "Generic (PLEG): container finished" podID="d663a7bc-4487-4d5c-9367-2ba9440f10df" containerID="ffd9f9046dcd234a64d09eefed1778fcc357c9b75672292bcf1b0a5454459a48" exitCode=0 Jul 07 18:55:09.738026 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:09.737602 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4bn7v/must-gather-rplh6" event={"ID":"d663a7bc-4487-4d5c-9367-2ba9440f10df","Type":"ContainerDied","Data":"ffd9f9046dcd234a64d09eefed1778fcc357c9b75672292bcf1b0a5454459a48"} Jul 07 18:55:09.738026 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:09.737916 2583 scope.go:117] "RemoveContainer" containerID="ffd9f9046dcd234a64d09eefed1778fcc357c9b75672292bcf1b0a5454459a48" Jul 07 18:55:09.790780 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:09.790743 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-4bn7v_must-gather-rplh6_d663a7bc-4487-4d5c-9367-2ba9440f10df/gather/0.log" Jul 07 18:55:10.587275 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:10.587178 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:55:10.792670 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:10.792636 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:55:10.882742 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:10.882650 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:55:13.347957 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:13.347928 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-c6dfn_7cc80f14-06d5-483d-b15b-a44896b686a6/global-pull-secret-syncer/0.log" Jul 07 18:55:13.395912 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:13.395882 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" Jul 07 18:55:13.495883 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:13.495856 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-120.ec2.internal_ee9c1d39d9fd4f723b1434f6d29f4c61/haproxy/0.log" Jul 07 18:55:15.559750 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:15.559712 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-4bn7v/must-gather-rplh6"] Jul 07 18:55:15.560188 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:15.559901 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-must-gather-4bn7v/must-gather-rplh6" podUID="d663a7bc-4487-4d5c-9367-2ba9440f10df" containerName="copy" containerID="cri-o://5806278ada00e0a73ad9fed77ee26dc41e709096e5204f0269fc3b2ca4fe1059" gracePeriod=2 Jul 07 18:55:15.564206 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:15.564101 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-4bn7v/must-gather-rplh6"] Jul 07 18:55:15.762000 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:15.761972 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-4bn7v_must-gather-rplh6_d663a7bc-4487-4d5c-9367-2ba9440f10df/copy/0.log" Jul 07 18:55:15.762421 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:15.762399 2583 generic.go:356] "Generic (PLEG): container finished" podID="d663a7bc-4487-4d5c-9367-2ba9440f10df" containerID="5806278ada00e0a73ad9fed77ee26dc41e709096e5204f0269fc3b2ca4fe1059" exitCode=143 Jul 07 18:55:15.792629 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:15.792606 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-4bn7v_must-gather-rplh6_d663a7bc-4487-4d5c-9367-2ba9440f10df/copy/0.log" Jul 07 18:55:15.795071 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:15.795054 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4bn7v/must-gather-rplh6" Jul 07 18:55:15.797030 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:55:15.797002 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-rplh6\" is forbidden: User \"system:node:ip-10-0-140-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-4bn7v\": no relationship found between node 'ip-10-0-140-120.ec2.internal' and this object" podUID="d663a7bc-4487-4d5c-9367-2ba9440f10df" pod="openshift-must-gather-4bn7v/must-gather-rplh6" Jul 07 18:55:15.913824 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:15.913736 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d663a7bc-4487-4d5c-9367-2ba9440f10df-must-gather-output\") pod \"d663a7bc-4487-4d5c-9367-2ba9440f10df\" (UID: \"d663a7bc-4487-4d5c-9367-2ba9440f10df\") " Jul 07 18:55:15.913824 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:15.913812 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r86sh\" (UniqueName: \"kubernetes.io/projected/d663a7bc-4487-4d5c-9367-2ba9440f10df-kube-api-access-r86sh\") pod \"d663a7bc-4487-4d5c-9367-2ba9440f10df\" (UID: \"d663a7bc-4487-4d5c-9367-2ba9440f10df\") " Jul 07 18:55:15.914219 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:15.914185 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d663a7bc-4487-4d5c-9367-2ba9440f10df-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "d663a7bc-4487-4d5c-9367-2ba9440f10df" (UID: "d663a7bc-4487-4d5c-9367-2ba9440f10df"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:55:15.915935 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:15.915913 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d663a7bc-4487-4d5c-9367-2ba9440f10df-kube-api-access-r86sh" (OuterVolumeSpecName: "kube-api-access-r86sh") pod "d663a7bc-4487-4d5c-9367-2ba9440f10df" (UID: "d663a7bc-4487-4d5c-9367-2ba9440f10df"). InnerVolumeSpecName "kube-api-access-r86sh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:55:16.014569 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:16.014529 2583 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d663a7bc-4487-4d5c-9367-2ba9440f10df-must-gather-output\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:55:16.014569 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:16.014561 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r86sh\" (UniqueName: \"kubernetes.io/projected/d663a7bc-4487-4d5c-9367-2ba9440f10df-kube-api-access-r86sh\") on node \"ip-10-0-140-120.ec2.internal\" DevicePath \"\"" Jul 07 18:55:16.170604 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:16.170527 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:55:16.224185 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:16.224157 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8khjj_e48b5a6a-4b69-4e2a-823c-656ce51dc9e8/konnectivity-agent/0.log" limit=1024 Jul 07 18:55:16.767331 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:16.767302 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-4bn7v_must-gather-rplh6_d663a7bc-4487-4d5c-9367-2ba9440f10df/copy/0.log" Jul 07 18:55:16.767761 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:16.767683 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4bn7v/must-gather-rplh6" Jul 07 18:55:16.767761 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:16.767702 2583 scope.go:117] "RemoveContainer" containerID="5806278ada00e0a73ad9fed77ee26dc41e709096e5204f0269fc3b2ca4fe1059" Jul 07 18:55:16.769996 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:55:16.769965 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-rplh6\" is forbidden: User \"system:node:ip-10-0-140-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-4bn7v\": no relationship found between node 'ip-10-0-140-120.ec2.internal' and this object" podUID="d663a7bc-4487-4d5c-9367-2ba9440f10df" pod="openshift-must-gather-4bn7v/must-gather-rplh6" Jul 07 18:55:16.775475 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:16.775203 2583 scope.go:117] "RemoveContainer" containerID="ffd9f9046dcd234a64d09eefed1778fcc357c9b75672292bcf1b0a5454459a48" Jul 07 18:55:16.777976 ip-10-0-140-120 kubenswrapper[2583]: E0707 18:55:16.777952 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-rplh6\" is forbidden: User \"system:node:ip-10-0-140-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-4bn7v\": no relationship found between node 'ip-10-0-140-120.ec2.internal' and this object" podUID="d663a7bc-4487-4d5c-9367-2ba9440f10df" pod="openshift-must-gather-4bn7v/must-gather-rplh6" Jul 07 18:55:17.337150 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:17.337041 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d25d201c-3b25-47f0-b918-c8f3980e2da4/alertmanager/0.log" Jul 07 18:55:17.371891 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:17.371864 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d25d201c-3b25-47f0-b918-c8f3980e2da4/config-reloader/0.log" Jul 07 18:55:17.399182 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:17.399151 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d25d201c-3b25-47f0-b918-c8f3980e2da4/kube-rbac-proxy-web/0.log" Jul 07 18:55:17.413705 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:17.413674 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d663a7bc-4487-4d5c-9367-2ba9440f10df" path="/var/lib/kubelet/pods/d663a7bc-4487-4d5c-9367-2ba9440f10df/volumes" Jul 07 18:55:17.426922 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:17.426882 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d25d201c-3b25-47f0-b918-c8f3980e2da4/kube-rbac-proxy/0.log" Jul 07 18:55:17.462555 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:17.462529 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d25d201c-3b25-47f0-b918-c8f3980e2da4/kube-rbac-proxy-metric/0.log" Jul 07 18:55:17.487472 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:17.487440 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d25d201c-3b25-47f0-b918-c8f3980e2da4/prom-label-proxy/0.log" Jul 07 18:55:17.518103 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:17.518073 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d25d201c-3b25-47f0-b918-c8f3980e2da4/init-config-reloader/0.log" Jul 07 18:55:17.557105 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:17.557076 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-wszbl_363d866e-b0a1-4351-8f85-b994a18200a7/cluster-monitoring-operator/2.log" Jul 07 18:55:17.639883 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:17.639795 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-wszbl_363d866e-b0a1-4351-8f85-b994a18200a7/cluster-monitoring-operator/1.log" Jul 07 18:55:17.773460 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:17.773427 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-q45t8_e1e0387c-912d-4098-bc91-1865b90b26ad/monitoring-plugin/0.log" Jul 07 18:55:17.884556 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:17.884522 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m9skr_c1d6d0aa-b29e-49f7-b83f-df34ef4a027a/node-exporter/0.log" Jul 07 18:55:17.911672 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:17.911599 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m9skr_c1d6d0aa-b29e-49f7-b83f-df34ef4a027a/kube-rbac-proxy/0.log" Jul 07 18:55:17.938115 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:17.938090 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m9skr_c1d6d0aa-b29e-49f7-b83f-df34ef4a027a/init-textfile/0.log" Jul 07 18:55:18.049377 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:18.049349 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-rgvt7_2bf91236-1a18-4683-867f-88f7a67cba50/kube-rbac-proxy-main/0.log" Jul 07 18:55:18.072491 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:18.072458 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-rgvt7_2bf91236-1a18-4683-867f-88f7a67cba50/kube-rbac-proxy-self/0.log" Jul 07 18:55:18.094265 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:18.094216 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-rgvt7_2bf91236-1a18-4683-867f-88f7a67cba50/openshift-state-metrics/0.log" Jul 07 18:55:18.131971 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:18.131946 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bc446e38-4de2-40c0-a548-0f306024a7b4/prometheus/0.log" Jul 07 18:55:18.157585 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:18.157561 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bc446e38-4de2-40c0-a548-0f306024a7b4/config-reloader/0.log" Jul 07 18:55:18.182579 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:18.182504 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bc446e38-4de2-40c0-a548-0f306024a7b4/thanos-sidecar/0.log" Jul 07 18:55:18.214751 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:18.214721 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bc446e38-4de2-40c0-a548-0f306024a7b4/kube-rbac-proxy-web/0.log" Jul 07 18:55:18.255883 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:18.255860 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bc446e38-4de2-40c0-a548-0f306024a7b4/kube-rbac-proxy/0.log" Jul 07 18:55:18.286343 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:18.286315 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bc446e38-4de2-40c0-a548-0f306024a7b4/kube-rbac-proxy-thanos/0.log" Jul 07 18:55:18.316766 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:18.316743 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bc446e38-4de2-40c0-a548-0f306024a7b4/init-config-reloader/0.log" Jul 07 18:55:18.348355 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:18.348323 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-j2p4h_d33cb992-3e55-4a9d-8405-2aff78a850aa/prometheus-operator/0.log" Jul 07 18:55:18.375740 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:18.375717 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-j2p4h_d33cb992-3e55-4a9d-8405-2aff78a850aa/kube-rbac-proxy/0.log" Jul 07 18:55:18.413840 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:18.413808 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-mjtkj_f5a4981e-bb5c-4860-8340-5a6c31c3ecfe/prometheus-operator-admission-webhook/0.log" Jul 07 18:55:20.541619 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:20.541507 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5x9w8_f390fdec-e425-4d23-a559-9681817515ec/console-operator/1.log" Jul 07 18:55:20.546599 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:20.546577 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5x9w8_f390fdec-e425-4d23-a559-9681817515ec/console-operator/2.log" Jul 07 18:55:21.332451 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.332421 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nlzrd/perf-node-gather-daemonset-8x6kc"] Jul 07 18:55:21.332774 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.332762 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d663a7bc-4487-4d5c-9367-2ba9440f10df" containerName="gather" Jul 07 18:55:21.332829 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.332776 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d663a7bc-4487-4d5c-9367-2ba9440f10df" containerName="gather" Jul 07 18:55:21.332829 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.332791 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d663a7bc-4487-4d5c-9367-2ba9440f10df" containerName="copy" Jul 07 18:55:21.332829 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.332796 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d663a7bc-4487-4d5c-9367-2ba9440f10df" containerName="copy" Jul 07 18:55:21.332933 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.332856 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d663a7bc-4487-4d5c-9367-2ba9440f10df" containerName="copy" Jul 07 18:55:21.332933 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.332867 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d663a7bc-4487-4d5c-9367-2ba9440f10df" containerName="gather" Jul 07 18:55:21.337810 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.337793 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nlzrd/perf-node-gather-daemonset-8x6kc" Jul 07 18:55:21.340309 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.340282 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nlzrd\"/\"default-dockercfg-zkgrc\"" Jul 07 18:55:21.340421 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.340284 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nlzrd\"/\"openshift-service-ca.crt\"" Jul 07 18:55:21.341015 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.340999 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nlzrd\"/\"kube-root-ca.crt\"" Jul 07 18:55:21.346219 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.346199 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nlzrd/perf-node-gather-daemonset-8x6kc"] Jul 07 18:55:21.460441 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.460406 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0eea44fd-a0c3-4380-9566-cb7f9c5bba07-podres\") pod \"perf-node-gather-daemonset-8x6kc\" (UID: \"0eea44fd-a0c3-4380-9566-cb7f9c5bba07\") " pod="openshift-must-gather-nlzrd/perf-node-gather-daemonset-8x6kc" Jul 07 18:55:21.460639 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.460477 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0eea44fd-a0c3-4380-9566-cb7f9c5bba07-sys\") pod \"perf-node-gather-daemonset-8x6kc\" (UID: \"0eea44fd-a0c3-4380-9566-cb7f9c5bba07\") " pod="openshift-must-gather-nlzrd/perf-node-gather-daemonset-8x6kc" Jul 07 18:55:21.460639 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.460522 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0eea44fd-a0c3-4380-9566-cb7f9c5bba07-lib-modules\") pod \"perf-node-gather-daemonset-8x6kc\" (UID: \"0eea44fd-a0c3-4380-9566-cb7f9c5bba07\") " pod="openshift-must-gather-nlzrd/perf-node-gather-daemonset-8x6kc" Jul 07 18:55:21.460639 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.460550 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp4p2\" (UniqueName: \"kubernetes.io/projected/0eea44fd-a0c3-4380-9566-cb7f9c5bba07-kube-api-access-gp4p2\") pod \"perf-node-gather-daemonset-8x6kc\" (UID: \"0eea44fd-a0c3-4380-9566-cb7f9c5bba07\") " pod="openshift-must-gather-nlzrd/perf-node-gather-daemonset-8x6kc" Jul 07 18:55:21.460639 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.460582 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0eea44fd-a0c3-4380-9566-cb7f9c5bba07-proc\") pod \"perf-node-gather-daemonset-8x6kc\" (UID: \"0eea44fd-a0c3-4380-9566-cb7f9c5bba07\") " pod="openshift-must-gather-nlzrd/perf-node-gather-daemonset-8x6kc" Jul 07 18:55:21.504868 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.504840 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-cbv87_fe9ddaa2-77d8-4b20-866d-caa5fce1427c/volume-data-source-validator/0.log" Jul 07 18:55:21.561989 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.561953 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0eea44fd-a0c3-4380-9566-cb7f9c5bba07-sys\") pod \"perf-node-gather-daemonset-8x6kc\" (UID: \"0eea44fd-a0c3-4380-9566-cb7f9c5bba07\") " pod="openshift-must-gather-nlzrd/perf-node-gather-daemonset-8x6kc" Jul 07 18:55:21.562397 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.561992 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0eea44fd-a0c3-4380-9566-cb7f9c5bba07-lib-modules\") pod \"perf-node-gather-daemonset-8x6kc\" (UID: \"0eea44fd-a0c3-4380-9566-cb7f9c5bba07\") " pod="openshift-must-gather-nlzrd/perf-node-gather-daemonset-8x6kc" Jul 07 18:55:21.562397 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.562013 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp4p2\" (UniqueName: \"kubernetes.io/projected/0eea44fd-a0c3-4380-9566-cb7f9c5bba07-kube-api-access-gp4p2\") pod \"perf-node-gather-daemonset-8x6kc\" (UID: \"0eea44fd-a0c3-4380-9566-cb7f9c5bba07\") " pod="openshift-must-gather-nlzrd/perf-node-gather-daemonset-8x6kc" Jul 07 18:55:21.562397 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.562045 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0eea44fd-a0c3-4380-9566-cb7f9c5bba07-proc\") pod \"perf-node-gather-daemonset-8x6kc\" (UID: \"0eea44fd-a0c3-4380-9566-cb7f9c5bba07\") " pod="openshift-must-gather-nlzrd/perf-node-gather-daemonset-8x6kc" Jul 07 18:55:21.562397 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.562082 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0eea44fd-a0c3-4380-9566-cb7f9c5bba07-sys\") pod \"perf-node-gather-daemonset-8x6kc\" (UID: \"0eea44fd-a0c3-4380-9566-cb7f9c5bba07\") " pod="openshift-must-gather-nlzrd/perf-node-gather-daemonset-8x6kc" Jul 07 18:55:21.562397 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.562099 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0eea44fd-a0c3-4380-9566-cb7f9c5bba07-podres\") pod \"perf-node-gather-daemonset-8x6kc\" (UID: \"0eea44fd-a0c3-4380-9566-cb7f9c5bba07\") " pod="openshift-must-gather-nlzrd/perf-node-gather-daemonset-8x6kc" Jul 07 18:55:21.562397 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.562149 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0eea44fd-a0c3-4380-9566-cb7f9c5bba07-lib-modules\") pod \"perf-node-gather-daemonset-8x6kc\" (UID: \"0eea44fd-a0c3-4380-9566-cb7f9c5bba07\") " pod="openshift-must-gather-nlzrd/perf-node-gather-daemonset-8x6kc" Jul 07 18:55:21.562397 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.562164 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0eea44fd-a0c3-4380-9566-cb7f9c5bba07-podres\") pod \"perf-node-gather-daemonset-8x6kc\" (UID: \"0eea44fd-a0c3-4380-9566-cb7f9c5bba07\") " pod="openshift-must-gather-nlzrd/perf-node-gather-daemonset-8x6kc" Jul 07 18:55:21.562397 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.562188 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0eea44fd-a0c3-4380-9566-cb7f9c5bba07-proc\") pod \"perf-node-gather-daemonset-8x6kc\" (UID: \"0eea44fd-a0c3-4380-9566-cb7f9c5bba07\") " pod="openshift-must-gather-nlzrd/perf-node-gather-daemonset-8x6kc" Jul 07 18:55:21.570682 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.570659 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp4p2\" (UniqueName: \"kubernetes.io/projected/0eea44fd-a0c3-4380-9566-cb7f9c5bba07-kube-api-access-gp4p2\") pod \"perf-node-gather-daemonset-8x6kc\" (UID: \"0eea44fd-a0c3-4380-9566-cb7f9c5bba07\") " pod="openshift-must-gather-nlzrd/perf-node-gather-daemonset-8x6kc" Jul 07 18:55:21.648560 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.648475 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nlzrd/perf-node-gather-daemonset-8x6kc" Jul 07 18:55:21.775013 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.774988 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nlzrd/perf-node-gather-daemonset-8x6kc"] Jul 07 18:55:21.777516 ip-10-0-140-120 kubenswrapper[2583]: W0707 18:55:21.777488 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0eea44fd_a0c3_4380_9566_cb7f9c5bba07.slice/crio-222c7c6a19789c4eff90407d38bd7bd4f53e415a29222e93c0af1c003566f5a8 WatchSource:0}: Error finding container 222c7c6a19789c4eff90407d38bd7bd4f53e415a29222e93c0af1c003566f5a8: Status 404 returned error can't find the container with id 222c7c6a19789c4eff90407d38bd7bd4f53e415a29222e93c0af1c003566f5a8 Jul 07 18:55:21.790702 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:21.790676 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nlzrd/perf-node-gather-daemonset-8x6kc" event={"ID":"0eea44fd-a0c3-4380-9566-cb7f9c5bba07","Type":"ContainerStarted","Data":"222c7c6a19789c4eff90407d38bd7bd4f53e415a29222e93c0af1c003566f5a8"} Jul 07 18:55:22.475313 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:22.475283 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lk6f2_3368babd-6ffc-4ec0-9dae-bcc00845b37f/dns/0.log" Jul 07 18:55:22.494959 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:22.494933 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lk6f2_3368babd-6ffc-4ec0-9dae-bcc00845b37f/kube-rbac-proxy/0.log" Jul 07 18:55:22.568471 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:22.568437 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ztg6h_8b9fb9da-073a-44c8-bf4d-2ee70fa748b9/dns-node-resolver/0.log" Jul 07 18:55:22.795662 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:22.795565 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nlzrd/perf-node-gather-daemonset-8x6kc" event={"ID":"0eea44fd-a0c3-4380-9566-cb7f9c5bba07","Type":"ContainerStarted","Data":"f240af288d5cdaa1fdde058156fb76d7fce0f88e1f0176775cea9c3c1fb8d541"} Jul 07 18:55:22.795810 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:22.795736 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nlzrd/perf-node-gather-daemonset-8x6kc" Jul 07 18:55:22.811635 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:22.811583 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nlzrd/perf-node-gather-daemonset-8x6kc" podStartSLOduration=1.811566805 podStartE2EDuration="1.811566805s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:55:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:55:22.810432574 +0000 UTC m=+551.911451149" watchObservedRunningTime="2026-07-07 18:55:22.811566805 +0000 UTC m=+551.912585380" Jul 07 18:55:23.050318 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:23.050236 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rwjnv_40b947e9-e3e6-43ec-8b88-ff421f627227/node-ca/0.log" Jul 07 18:55:23.866206 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:23.866174 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-86c68c69b7-p7g8g_7ea1005f-1048-4ec3-8fb7-e5ed794095a7/router/0.log" Jul 07 18:55:24.197350 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:24.197279 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5vxxf_b8d04c06-cf5f-4fe2-a384-a0420d3e4157/serve-healthcheck-canary/0.log" Jul 07 18:55:24.580620 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:24.580587 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-lqs4g_b8f4ce07-b53a-4e57-a9ca-e64da6750c7f/insights-operator/1.log" Jul 07 18:55:24.580792 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:24.580757 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-lqs4g_b8f4ce07-b53a-4e57-a9ca-e64da6750c7f/insights-operator/0.log" Jul 07 18:55:24.729449 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:24.729420 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t9wzd_67c94c15-f630-41c3-9d1d-33332df68075/kube-rbac-proxy/0.log" Jul 07 18:55:24.750807 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:24.750773 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t9wzd_67c94c15-f630-41c3-9d1d-33332df68075/exporter/0.log" Jul 07 18:55:24.771316 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:24.771289 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t9wzd_67c94c15-f630-41c3-9d1d-33332df68075/extractor/0.log" Jul 07 18:55:28.809527 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:28.809501 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nlzrd/perf-node-gather-daemonset-8x6kc" Jul 07 18:55:31.558847 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:31.558820 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-84cwc_30caa190-8d65-4825-a681-4af09975ef3e/kube-multus-additional-cni-plugins/0.log" Jul 07 18:55:31.595150 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:31.595076 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-84cwc_30caa190-8d65-4825-a681-4af09975ef3e/egress-router-binary-copy/0.log" Jul 07 18:55:31.624236 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:31.624213 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-84cwc_30caa190-8d65-4825-a681-4af09975ef3e/cni-plugins/0.log" Jul 07 18:55:31.644349 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:31.644324 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-84cwc_30caa190-8d65-4825-a681-4af09975ef3e/bond-cni-plugin/0.log" Jul 07 18:55:31.666388 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:31.666360 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-84cwc_30caa190-8d65-4825-a681-4af09975ef3e/routeoverride-cni/0.log" Jul 07 18:55:31.686308 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:31.686284 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-84cwc_30caa190-8d65-4825-a681-4af09975ef3e/whereabouts-cni-bincopy/0.log" Jul 07 18:55:31.706588 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:31.706563 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-84cwc_30caa190-8d65-4825-a681-4af09975ef3e/whereabouts-cni/0.log" Jul 07 18:55:31.734489 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:31.734463 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-f9lks_c1e5d07f-f71c-4423-9ef0-882c19222314/kube-multus/0.log" Jul 07 18:55:31.805622 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:31.805578 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cpp9l_329b4a4f-5489-4021-ab98-bcc7cc5b7dd0/network-metrics-daemon/0.log" Jul 07 18:55:31.825461 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:31.825434 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cpp9l_329b4a4f-5489-4021-ab98-bcc7cc5b7dd0/kube-rbac-proxy/0.log" Jul 07 18:55:33.163123 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:33.163041 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9j7xz_bff07fe8-5476-4202-bd92-39b4a2ab6d8c/ovn-controller/0.log" Jul 07 18:55:33.182152 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:33.182125 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9j7xz_bff07fe8-5476-4202-bd92-39b4a2ab6d8c/ovn-acl-logging/1.log" Jul 07 18:55:33.182152 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:33.182145 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9j7xz_bff07fe8-5476-4202-bd92-39b4a2ab6d8c/ovn-acl-logging/0.log" Jul 07 18:55:33.198938 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:33.198909 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9j7xz_bff07fe8-5476-4202-bd92-39b4a2ab6d8c/kube-rbac-proxy-node/0.log" Jul 07 18:55:33.219729 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:33.219697 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9j7xz_bff07fe8-5476-4202-bd92-39b4a2ab6d8c/kube-rbac-proxy-ovn-metrics/0.log" Jul 07 18:55:33.238949 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:33.238918 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9j7xz_bff07fe8-5476-4202-bd92-39b4a2ab6d8c/northd/0.log" Jul 07 18:55:33.259068 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:33.259043 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9j7xz_bff07fe8-5476-4202-bd92-39b4a2ab6d8c/nbdb/0.log" Jul 07 18:55:33.278401 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:33.278377 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9j7xz_bff07fe8-5476-4202-bd92-39b4a2ab6d8c/sbdb/0.log" Jul 07 18:55:33.318903 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:33.318876 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9j7xz_bff07fe8-5476-4202-bd92-39b4a2ab6d8c/ovnkube-controller/0.log" Jul 07 18:55:34.345481 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:34.345451 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-l5d2z_c7296533-de2d-44a3-8972-5e1b8baeaae2/network-check-target-container/0.log" Jul 07 18:55:35.219684 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:35.219653 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-2w8tr_5d65ba38-b541-4fe1-8933-4f6f8d366f4d/iptables-alerter/0.log" Jul 07 18:55:35.907192 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:35.907167 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-xlzmh_f379a0f4-5022-4bd6-ad64-ba9c7c7a335d/tuned/0.log" Jul 07 18:55:37.673346 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:37.673313 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-56fcf86bf-7zg9v_3dccf34c-d46e-4122-81a2-2794860331a8/cluster-samples-operator/0.log" Jul 07 18:55:37.690355 ip-10-0-140-120 kubenswrapper[2583]: I0707 18:55:37.690327 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-56fcf86bf-7zg9v_3dccf34c-d46e-4122-81a2-2794860331a8/cluster-samples-operator-watch/0.log"