Apr 16 08:39:11.506113 ip-10-0-139-84 systemd[1]: Starting Kubernetes Kubelet... Apr 16 08:39:11.980387 ip-10-0-139-84 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Apr 16 08:39:11.980387 ip-10-0-139-84 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Apr 16 08:39:11.980387 ip-10-0-139-84 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Apr 16 08:39:11.980387 ip-10-0-139-84 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Apr 16 08:39:11.980387 ip-10-0-139-84 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Apr 16 08:39:11.982208 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.982125 2578 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Apr 16 08:39:11.987474 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987451 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 16 08:39:11.987474 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987471 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 16 08:39:11.987474 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987476 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 16 08:39:11.987474 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987479 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 16 08:39:11.987636 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987482 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 16 08:39:11.987636 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987485 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 16 08:39:11.987636 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987488 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 16 08:39:11.987636 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987490 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 16 08:39:11.987636 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987493 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 16 08:39:11.987636 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987495 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 16 08:39:11.987636 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987498 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 16 08:39:11.987636 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987500 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 16 08:39:11.987636 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987503 2578 feature_gate.go:328] unrecognized feature gate: Example Apr 16 08:39:11.987636 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987506 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 16 08:39:11.987636 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987508 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 16 08:39:11.987636 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987512 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 16 08:39:11.987636 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987516 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 16 08:39:11.987636 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987519 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 16 08:39:11.987636 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987522 2578 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 16 08:39:11.987636 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987526 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 16 08:39:11.987636 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987530 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 16 08:39:11.987636 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987534 2578 feature_gate.go:328] unrecognized feature gate: Example2 Apr 16 08:39:11.987636 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987537 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 16 08:39:11.988087 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987539 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 16 08:39:11.988087 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987542 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 16 08:39:11.988087 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987544 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 16 08:39:11.988087 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987553 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 16 08:39:11.988087 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987556 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 16 08:39:11.988087 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987558 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 16 08:39:11.988087 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987562 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 16 08:39:11.988087 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987564 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 16 08:39:11.988087 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987568 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 16 08:39:11.988087 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987570 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 16 08:39:11.988087 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987573 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 16 08:39:11.988087 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987575 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 16 08:39:11.988087 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987578 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 16 08:39:11.988087 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987580 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 16 08:39:11.988087 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987583 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 16 08:39:11.988087 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987585 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 16 08:39:11.988087 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987588 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 16 08:39:11.988087 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987590 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 16 08:39:11.988087 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987593 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 16 08:39:11.988087 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987595 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 16 08:39:11.988574 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987598 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 16 08:39:11.988574 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987600 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 16 08:39:11.988574 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987603 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 16 08:39:11.988574 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987605 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 16 08:39:11.988574 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987608 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 16 08:39:11.988574 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987611 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 16 08:39:11.988574 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987614 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 16 08:39:11.988574 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987617 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 16 08:39:11.988574 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987620 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 16 08:39:11.988574 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987622 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 16 08:39:11.988574 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987624 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 16 08:39:11.988574 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987627 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 16 08:39:11.988574 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987629 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 16 08:39:11.988574 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987632 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 16 08:39:11.988574 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987634 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 16 08:39:11.988574 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987637 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 16 08:39:11.988574 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987639 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 16 08:39:11.988574 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987642 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 16 08:39:11.988574 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987645 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 16 08:39:11.989049 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987649 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 16 08:39:11.989049 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987652 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 16 08:39:11.989049 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987654 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 16 08:39:11.989049 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987656 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 16 08:39:11.989049 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987659 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 16 08:39:11.989049 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987661 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 16 08:39:11.989049 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987663 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 16 08:39:11.989049 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987666 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 16 08:39:11.989049 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987669 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 16 08:39:11.989049 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987671 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 16 08:39:11.989049 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987674 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 16 08:39:11.989049 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987676 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 16 08:39:11.989049 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987678 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 16 08:39:11.989049 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987681 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 16 08:39:11.989049 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987684 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 16 08:39:11.989049 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987686 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 16 08:39:11.989049 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987689 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 16 08:39:11.989049 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987692 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 16 08:39:11.989049 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987694 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 16 08:39:11.989049 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987697 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 16 08:39:11.989510 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987699 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 16 08:39:11.989510 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987702 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 16 08:39:11.989510 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987704 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 16 08:39:11.989510 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.987707 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 16 08:39:11.989510 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988093 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 16 08:39:11.989510 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988100 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 16 08:39:11.989510 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988102 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 16 08:39:11.989510 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988105 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 16 08:39:11.989510 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988107 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 16 08:39:11.989510 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988110 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 16 08:39:11.989510 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988112 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 16 08:39:11.989510 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988115 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 16 08:39:11.989510 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988118 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 16 08:39:11.989510 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988122 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 16 08:39:11.989510 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988125 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 16 08:39:11.989510 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988128 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 16 08:39:11.989510 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988130 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 16 08:39:11.989510 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988133 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 16 08:39:11.989510 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988135 2578 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 16 08:39:11.989510 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988138 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 16 08:39:11.990000 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988142 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 16 08:39:11.990000 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988146 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 16 08:39:11.990000 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988149 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 16 08:39:11.990000 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988152 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 16 08:39:11.990000 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988155 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 16 08:39:11.990000 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988157 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 16 08:39:11.990000 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988160 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 16 08:39:11.990000 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988163 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 16 08:39:11.990000 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988166 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 16 08:39:11.990000 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988168 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 16 08:39:11.990000 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988171 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 16 08:39:11.990000 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988173 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 16 08:39:11.990000 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988176 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 16 08:39:11.990000 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988178 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 16 08:39:11.990000 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988181 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 16 08:39:11.990000 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988183 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 16 08:39:11.990000 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988186 2578 feature_gate.go:328] unrecognized feature gate: Example2 Apr 16 08:39:11.990000 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988188 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 16 08:39:11.990000 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988190 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 16 08:39:11.990476 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988193 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 16 08:39:11.990476 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988195 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 16 08:39:11.990476 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988198 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 16 08:39:11.990476 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988201 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 16 08:39:11.990476 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988203 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 16 08:39:11.990476 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988206 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 16 08:39:11.990476 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988209 2578 feature_gate.go:328] unrecognized feature gate: Example Apr 16 08:39:11.990476 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988211 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 16 08:39:11.990476 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988214 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 16 08:39:11.990476 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988216 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 16 08:39:11.990476 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988219 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 16 08:39:11.990476 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988222 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 16 08:39:11.990476 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988226 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 16 08:39:11.990476 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988228 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 16 08:39:11.990476 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988230 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 16 08:39:11.990476 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988233 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 16 08:39:11.990476 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988235 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 16 08:39:11.990476 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988238 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 16 08:39:11.990476 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988240 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 16 08:39:11.990476 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988243 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 16 08:39:11.990962 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988245 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 16 08:39:11.990962 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988248 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 16 08:39:11.990962 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988251 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 16 08:39:11.990962 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988253 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 16 08:39:11.990962 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988256 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 16 08:39:11.990962 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988258 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 16 08:39:11.990962 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988260 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 16 08:39:11.990962 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988263 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 16 08:39:11.990962 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988265 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 16 08:39:11.990962 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988267 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 16 08:39:11.990962 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988270 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 16 08:39:11.990962 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988272 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 16 08:39:11.990962 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988274 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 16 08:39:11.990962 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988277 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 16 08:39:11.990962 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988279 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 16 08:39:11.990962 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988282 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 16 08:39:11.990962 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988285 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 16 08:39:11.990962 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988287 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 16 08:39:11.990962 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988290 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 16 08:39:11.990962 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988293 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 16 08:39:11.991466 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988295 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 16 08:39:11.991466 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988298 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 16 08:39:11.991466 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988300 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 16 08:39:11.991466 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988302 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 16 08:39:11.991466 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988305 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 16 08:39:11.991466 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988308 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 16 08:39:11.991466 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988310 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 16 08:39:11.991466 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988313 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 16 08:39:11.991466 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988315 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 16 08:39:11.991466 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988317 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 16 08:39:11.991466 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.988320 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 16 08:39:11.991466 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989089 2578 flags.go:64] FLAG: --address="0.0.0.0" Apr 16 08:39:11.991466 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989098 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Apr 16 08:39:11.991466 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989105 2578 flags.go:64] FLAG: --anonymous-auth="true" Apr 16 08:39:11.991466 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989109 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Apr 16 08:39:11.991466 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989114 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Apr 16 08:39:11.991466 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989118 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Apr 16 08:39:11.991466 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989122 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Apr 16 08:39:11.991466 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989126 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Apr 16 08:39:11.991466 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989129 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Apr 16 08:39:11.991466 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989132 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Apr 16 08:39:11.991976 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989136 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Apr 16 08:39:11.991976 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989139 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Apr 16 08:39:11.991976 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989142 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Apr 16 08:39:11.991976 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989145 2578 flags.go:64] FLAG: --cgroup-root="" Apr 16 08:39:11.991976 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989147 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Apr 16 08:39:11.991976 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989150 2578 flags.go:64] FLAG: --client-ca-file="" Apr 16 08:39:11.991976 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989153 2578 flags.go:64] FLAG: --cloud-config="" Apr 16 08:39:11.991976 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989156 2578 flags.go:64] FLAG: --cloud-provider="external" Apr 16 08:39:11.991976 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989159 2578 flags.go:64] FLAG: --cluster-dns="[]" Apr 16 08:39:11.991976 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989163 2578 flags.go:64] FLAG: --cluster-domain="" Apr 16 08:39:11.991976 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989165 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Apr 16 08:39:11.991976 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989168 2578 flags.go:64] FLAG: --config-dir="" Apr 16 08:39:11.991976 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989172 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Apr 16 08:39:11.991976 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989175 2578 flags.go:64] FLAG: --container-log-max-files="5" Apr 16 08:39:11.991976 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989179 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Apr 16 08:39:11.991976 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989182 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Apr 16 08:39:11.991976 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989185 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Apr 16 08:39:11.991976 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989189 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Apr 16 08:39:11.991976 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989192 2578 flags.go:64] FLAG: --contention-profiling="false" Apr 16 08:39:11.991976 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989194 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Apr 16 08:39:11.991976 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989197 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Apr 16 08:39:11.991976 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989200 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Apr 16 08:39:11.991976 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989203 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Apr 16 08:39:11.991976 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989207 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Apr 16 08:39:11.991976 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989210 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Apr 16 08:39:11.992542 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989212 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Apr 16 08:39:11.992542 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989215 2578 flags.go:64] FLAG: --enable-load-reader="false" Apr 16 08:39:11.992542 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989218 2578 flags.go:64] FLAG: --enable-server="true" Apr 16 08:39:11.992542 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989221 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Apr 16 08:39:11.992542 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989226 2578 flags.go:64] FLAG: --event-burst="100" Apr 16 08:39:11.992542 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989229 2578 flags.go:64] FLAG: --event-qps="50" Apr 16 08:39:11.992542 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989232 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Apr 16 08:39:11.992542 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989235 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Apr 16 08:39:11.992542 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989238 2578 flags.go:64] FLAG: --eviction-hard="" Apr 16 08:39:11.992542 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989242 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Apr 16 08:39:11.992542 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989245 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Apr 16 08:39:11.992542 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989247 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Apr 16 08:39:11.992542 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989250 2578 flags.go:64] FLAG: --eviction-soft="" Apr 16 08:39:11.992542 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989253 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Apr 16 08:39:11.992542 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989256 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Apr 16 08:39:11.992542 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989259 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Apr 16 08:39:11.992542 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989261 2578 flags.go:64] FLAG: --experimental-mounter-path="" Apr 16 08:39:11.992542 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989264 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Apr 16 08:39:11.992542 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989267 2578 flags.go:64] FLAG: --fail-swap-on="true" Apr 16 08:39:11.992542 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989270 2578 flags.go:64] FLAG: --feature-gates="" Apr 16 08:39:11.992542 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989273 2578 flags.go:64] FLAG: --file-check-frequency="20s" Apr 16 08:39:11.992542 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989276 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Apr 16 08:39:11.992542 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989279 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Apr 16 08:39:11.992542 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989282 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Apr 16 08:39:11.992542 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989285 2578 flags.go:64] FLAG: --healthz-port="10248" Apr 16 08:39:11.992542 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989288 2578 flags.go:64] FLAG: --help="false" Apr 16 08:39:11.993156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989291 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-139-84.ec2.internal" Apr 16 08:39:11.993156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989294 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Apr 16 08:39:11.993156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989296 2578 flags.go:64] FLAG: --http-check-frequency="20s" Apr 16 08:39:11.993156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989299 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Apr 16 08:39:11.993156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989302 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Apr 16 08:39:11.993156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989306 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Apr 16 08:39:11.993156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989308 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Apr 16 08:39:11.993156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989311 2578 flags.go:64] FLAG: --image-service-endpoint="" Apr 16 08:39:11.993156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989313 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Apr 16 08:39:11.993156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989317 2578 flags.go:64] FLAG: --kube-api-burst="100" Apr 16 08:39:11.993156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989319 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Apr 16 08:39:11.993156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989325 2578 flags.go:64] FLAG: --kube-api-qps="50" Apr 16 08:39:11.993156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989328 2578 flags.go:64] FLAG: --kube-reserved="" Apr 16 08:39:11.993156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989330 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Apr 16 08:39:11.993156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989333 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Apr 16 08:39:11.993156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989336 2578 flags.go:64] FLAG: --kubelet-cgroups="" Apr 16 08:39:11.993156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989339 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Apr 16 08:39:11.993156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989341 2578 flags.go:64] FLAG: --lock-file="" Apr 16 08:39:11.993156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989344 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Apr 16 08:39:11.993156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989347 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Apr 16 08:39:11.993156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989350 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Apr 16 08:39:11.993156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989355 2578 flags.go:64] FLAG: --log-json-split-stream="false" Apr 16 08:39:11.993156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989358 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Apr 16 08:39:11.993677 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989360 2578 flags.go:64] FLAG: --log-text-split-stream="false" Apr 16 08:39:11.993677 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989363 2578 flags.go:64] FLAG: --logging-format="text" Apr 16 08:39:11.993677 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989366 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Apr 16 08:39:11.993677 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989369 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Apr 16 08:39:11.993677 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989372 2578 flags.go:64] FLAG: --manifest-url="" Apr 16 08:39:11.993677 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989377 2578 flags.go:64] FLAG: --manifest-url-header="" Apr 16 08:39:11.993677 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989381 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Apr 16 08:39:11.993677 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989384 2578 flags.go:64] FLAG: --max-open-files="1000000" Apr 16 08:39:11.993677 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989388 2578 flags.go:64] FLAG: --max-pods="110" Apr 16 08:39:11.993677 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989390 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Apr 16 08:39:11.993677 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989393 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Apr 16 08:39:11.993677 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989396 2578 flags.go:64] FLAG: --memory-manager-policy="None" Apr 16 08:39:11.993677 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989399 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Apr 16 08:39:11.993677 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989401 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Apr 16 08:39:11.993677 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989404 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Apr 16 08:39:11.993677 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989407 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Apr 16 08:39:11.993677 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989418 2578 flags.go:64] FLAG: --node-status-max-images="50" Apr 16 08:39:11.993677 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989421 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Apr 16 08:39:11.993677 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989423 2578 flags.go:64] FLAG: --oom-score-adj="-999" Apr 16 08:39:11.993677 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989427 2578 flags.go:64] FLAG: --pod-cidr="" Apr 16 08:39:11.993677 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989431 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dc76bab72f320de3d4105c90d73c4fb139c09e20ce0fa8dcbc0cb59920d27dec" Apr 16 08:39:11.993677 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989436 2578 flags.go:64] FLAG: --pod-manifest-path="" Apr 16 08:39:11.993677 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989439 2578 flags.go:64] FLAG: --pod-max-pids="-1" Apr 16 08:39:11.993677 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989442 2578 flags.go:64] FLAG: --pods-per-core="0" Apr 16 08:39:11.994265 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989444 2578 flags.go:64] FLAG: --port="10250" Apr 16 08:39:11.994265 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989447 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Apr 16 08:39:11.994265 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989450 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-074e956d2616437e4" Apr 16 08:39:11.994265 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989453 2578 flags.go:64] FLAG: --qos-reserved="" Apr 16 08:39:11.994265 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989456 2578 flags.go:64] FLAG: --read-only-port="10255" Apr 16 08:39:11.994265 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989459 2578 flags.go:64] FLAG: --register-node="true" Apr 16 08:39:11.994265 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989461 2578 flags.go:64] FLAG: --register-schedulable="true" Apr 16 08:39:11.994265 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989464 2578 flags.go:64] FLAG: --register-with-taints="" Apr 16 08:39:11.994265 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989467 2578 flags.go:64] FLAG: --registry-burst="10" Apr 16 08:39:11.994265 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989470 2578 flags.go:64] FLAG: --registry-qps="5" Apr 16 08:39:11.994265 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989473 2578 flags.go:64] FLAG: --reserved-cpus="" Apr 16 08:39:11.994265 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989475 2578 flags.go:64] FLAG: --reserved-memory="" Apr 16 08:39:11.994265 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989479 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Apr 16 08:39:11.994265 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989484 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Apr 16 08:39:11.994265 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989487 2578 flags.go:64] FLAG: --rotate-certificates="false" Apr 16 08:39:11.994265 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989490 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Apr 16 08:39:11.994265 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989493 2578 flags.go:64] FLAG: --runonce="false" Apr 16 08:39:11.994265 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989496 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Apr 16 08:39:11.994265 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989499 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Apr 16 08:39:11.994265 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989502 2578 flags.go:64] FLAG: --seccomp-default="false" Apr 16 08:39:11.994265 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989504 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Apr 16 08:39:11.994265 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989507 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Apr 16 08:39:11.994265 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989510 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Apr 16 08:39:11.994265 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989513 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Apr 16 08:39:11.994265 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989516 2578 flags.go:64] FLAG: --storage-driver-password="root" Apr 16 08:39:11.994265 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989519 2578 flags.go:64] FLAG: --storage-driver-secure="false" Apr 16 08:39:11.994917 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989522 2578 flags.go:64] FLAG: --storage-driver-table="stats" Apr 16 08:39:11.994917 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989524 2578 flags.go:64] FLAG: --storage-driver-user="root" Apr 16 08:39:11.994917 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989531 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Apr 16 08:39:11.994917 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989535 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Apr 16 08:39:11.994917 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989538 2578 flags.go:64] FLAG: --system-cgroups="" Apr 16 08:39:11.994917 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989541 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Apr 16 08:39:11.994917 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989546 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Apr 16 08:39:11.994917 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989548 2578 flags.go:64] FLAG: --tls-cert-file="" Apr 16 08:39:11.994917 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989551 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Apr 16 08:39:11.994917 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989555 2578 flags.go:64] FLAG: --tls-min-version="" Apr 16 08:39:11.994917 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989557 2578 flags.go:64] FLAG: --tls-private-key-file="" Apr 16 08:39:11.994917 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989560 2578 flags.go:64] FLAG: --topology-manager-policy="none" Apr 16 08:39:11.994917 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989562 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Apr 16 08:39:11.994917 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989565 2578 flags.go:64] FLAG: --topology-manager-scope="container" Apr 16 08:39:11.994917 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989568 2578 flags.go:64] FLAG: --v="2" Apr 16 08:39:11.994917 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989572 2578 flags.go:64] FLAG: --version="false" Apr 16 08:39:11.994917 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989580 2578 flags.go:64] FLAG: --vmodule="" Apr 16 08:39:11.994917 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989584 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Apr 16 08:39:11.994917 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.989587 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Apr 16 08:39:11.994917 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989670 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 16 08:39:11.994917 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989673 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 16 08:39:11.994917 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989676 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 16 08:39:11.994917 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989679 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 16 08:39:11.994917 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989682 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 16 08:39:11.995468 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989685 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 16 08:39:11.995468 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989687 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 16 08:39:11.995468 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989690 2578 feature_gate.go:328] unrecognized feature gate: Example2 Apr 16 08:39:11.995468 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989693 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 16 08:39:11.995468 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989696 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 16 08:39:11.995468 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989700 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 16 08:39:11.995468 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989704 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 16 08:39:11.995468 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989708 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 16 08:39:11.995468 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989710 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 16 08:39:11.995468 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989713 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 16 08:39:11.995468 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989716 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 16 08:39:11.995468 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989719 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 16 08:39:11.995468 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989722 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 16 08:39:11.995468 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989724 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 16 08:39:11.995468 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989727 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 16 08:39:11.995468 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989729 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 16 08:39:11.995468 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989731 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 16 08:39:11.995468 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989734 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 16 08:39:11.995468 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989736 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 16 08:39:11.996004 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989739 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 16 08:39:11.996004 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989741 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 16 08:39:11.996004 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989743 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 16 08:39:11.996004 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989746 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 16 08:39:11.996004 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989748 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 16 08:39:11.996004 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989751 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 16 08:39:11.996004 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989753 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 16 08:39:11.996004 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989756 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 16 08:39:11.996004 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989758 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 16 08:39:11.996004 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989761 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 16 08:39:11.996004 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989777 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 16 08:39:11.996004 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989780 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 16 08:39:11.996004 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989783 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 16 08:39:11.996004 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989785 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 16 08:39:11.996004 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989787 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 16 08:39:11.996004 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989790 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 16 08:39:11.996004 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989792 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 16 08:39:11.996004 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989795 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 16 08:39:11.996004 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989797 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 16 08:39:11.996004 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989800 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 16 08:39:11.996477 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989802 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 16 08:39:11.996477 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989805 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 16 08:39:11.996477 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989808 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 16 08:39:11.996477 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989810 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 16 08:39:11.996477 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989813 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 16 08:39:11.996477 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989816 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 16 08:39:11.996477 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989818 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 16 08:39:11.996477 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989821 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 16 08:39:11.996477 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989823 2578 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 16 08:39:11.996477 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989826 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 16 08:39:11.996477 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989828 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 16 08:39:11.996477 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989831 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 16 08:39:11.996477 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989834 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 16 08:39:11.996477 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989836 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 16 08:39:11.996477 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989838 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 16 08:39:11.996477 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989841 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 16 08:39:11.996477 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989843 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 16 08:39:11.996477 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989846 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 16 08:39:11.996477 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989848 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 16 08:39:11.996940 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989850 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 16 08:39:11.996940 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989853 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 16 08:39:11.996940 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989855 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 16 08:39:11.996940 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989857 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 16 08:39:11.996940 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989860 2578 feature_gate.go:328] unrecognized feature gate: Example Apr 16 08:39:11.996940 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989862 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 16 08:39:11.996940 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989865 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 16 08:39:11.996940 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989867 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 16 08:39:11.996940 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989869 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 16 08:39:11.996940 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989872 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 16 08:39:11.996940 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989875 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 16 08:39:11.996940 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989877 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 16 08:39:11.996940 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989879 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 16 08:39:11.996940 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989882 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 16 08:39:11.996940 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989884 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 16 08:39:11.996940 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989886 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 16 08:39:11.996940 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989889 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 16 08:39:11.996940 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989891 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 16 08:39:11.996940 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989894 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 16 08:39:11.996940 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989896 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 16 08:39:11.997408 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989899 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 16 08:39:11.997408 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989901 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 16 08:39:11.997408 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.989903 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 16 08:39:11.997408 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.990570 2578 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Apr 16 08:39:11.997735 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.997715 2578 server.go:530] "Kubelet version" kubeletVersion="v1.33.9" Apr 16 08:39:11.997788 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.997737 2578 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Apr 16 08:39:11.997826 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997800 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 16 08:39:11.997826 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997806 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 16 08:39:11.997826 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997809 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 16 08:39:11.997826 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997812 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 16 08:39:11.997826 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997815 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 16 08:39:11.997826 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997818 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 16 08:39:11.997826 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997820 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 16 08:39:11.997826 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997823 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 16 08:39:11.997826 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997826 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 16 08:39:11.997826 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997829 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 16 08:39:11.997826 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997831 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 16 08:39:11.998082 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997834 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 16 08:39:11.998082 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997837 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 16 08:39:11.998082 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997839 2578 feature_gate.go:328] unrecognized feature gate: Example2 Apr 16 08:39:11.998082 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997842 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 16 08:39:11.998082 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997844 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 16 08:39:11.998082 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997847 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 16 08:39:11.998082 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997849 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 16 08:39:11.998082 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997852 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 16 08:39:11.998082 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997855 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 16 08:39:11.998082 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997857 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 16 08:39:11.998082 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997859 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 16 08:39:11.998082 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997862 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 16 08:39:11.998082 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997864 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 16 08:39:11.998082 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997866 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 16 08:39:11.998082 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997869 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 16 08:39:11.998082 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997871 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 16 08:39:11.998082 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997874 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 16 08:39:11.998082 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997876 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 16 08:39:11.998082 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997878 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 16 08:39:11.998082 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997881 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 16 08:39:11.998539 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997884 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 16 08:39:11.998539 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997886 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 16 08:39:11.998539 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997889 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 16 08:39:11.998539 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997891 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 16 08:39:11.998539 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997893 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 16 08:39:11.998539 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997896 2578 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 16 08:39:11.998539 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997898 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 16 08:39:11.998539 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997901 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 16 08:39:11.998539 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997903 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 16 08:39:11.998539 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997905 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 16 08:39:11.998539 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997908 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 16 08:39:11.998539 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997910 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 16 08:39:11.998539 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997912 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 16 08:39:11.998539 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997916 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 16 08:39:11.998539 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997918 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 16 08:39:11.998539 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997921 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 16 08:39:11.998539 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997923 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 16 08:39:11.998539 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997926 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 16 08:39:11.998539 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997928 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 16 08:39:11.998539 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997931 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 16 08:39:11.999062 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997933 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 16 08:39:11.999062 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997935 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 16 08:39:11.999062 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997938 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 16 08:39:11.999062 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997940 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 16 08:39:11.999062 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997943 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 16 08:39:11.999062 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997946 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 16 08:39:11.999062 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997951 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 16 08:39:11.999062 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997954 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 16 08:39:11.999062 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997956 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 16 08:39:11.999062 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997958 2578 feature_gate.go:328] unrecognized feature gate: Example Apr 16 08:39:11.999062 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997961 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 16 08:39:11.999062 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997963 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 16 08:39:11.999062 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997965 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 16 08:39:11.999062 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997969 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 16 08:39:11.999062 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997971 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 16 08:39:11.999062 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997973 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 16 08:39:11.999062 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997976 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 16 08:39:11.999062 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997978 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 16 08:39:11.999062 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997981 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 16 08:39:11.999514 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997983 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 16 08:39:11.999514 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997985 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 16 08:39:11.999514 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997988 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 16 08:39:11.999514 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997990 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 16 08:39:11.999514 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997993 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 16 08:39:11.999514 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997995 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 16 08:39:11.999514 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.997999 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 16 08:39:11.999514 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998001 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 16 08:39:11.999514 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998005 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 16 08:39:11.999514 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998008 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 16 08:39:11.999514 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998012 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 16 08:39:11.999514 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998014 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 16 08:39:11.999514 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998017 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 16 08:39:11.999514 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998020 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 16 08:39:11.999514 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998023 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 16 08:39:11.999514 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998026 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 16 08:39:11.999907 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.998031 2578 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Apr 16 08:39:11.999907 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998141 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 16 08:39:11.999907 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998146 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 16 08:39:11.999907 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998149 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 16 08:39:11.999907 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998152 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 16 08:39:11.999907 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998155 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 16 08:39:11.999907 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998158 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 16 08:39:11.999907 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998160 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 16 08:39:11.999907 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998163 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 16 08:39:11.999907 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998166 2578 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 16 08:39:11.999907 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998168 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 16 08:39:11.999907 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998171 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 16 08:39:11.999907 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998173 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 16 08:39:11.999907 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998176 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 16 08:39:11.999907 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998178 2578 feature_gate.go:328] unrecognized feature gate: Example Apr 16 08:39:11.999907 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998180 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 16 08:39:12.000318 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998183 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 16 08:39:12.000318 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998185 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 16 08:39:12.000318 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998188 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 16 08:39:12.000318 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998190 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 16 08:39:12.000318 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998193 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 16 08:39:12.000318 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998196 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 16 08:39:12.000318 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998198 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 16 08:39:12.000318 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998201 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 16 08:39:12.000318 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998204 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 16 08:39:12.000318 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998206 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 16 08:39:12.000318 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998208 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 16 08:39:12.000318 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998212 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 16 08:39:12.000318 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998215 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 16 08:39:12.000318 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998217 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 16 08:39:12.000318 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998220 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 16 08:39:12.000318 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998222 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 16 08:39:12.000318 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998224 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 16 08:39:12.000318 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998227 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 16 08:39:12.000318 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998229 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 16 08:39:12.000754 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998231 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 16 08:39:12.000754 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998234 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 16 08:39:12.000754 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998236 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 16 08:39:12.000754 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998238 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 16 08:39:12.000754 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998240 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 16 08:39:12.000754 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998243 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 16 08:39:12.000754 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998245 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 16 08:39:12.000754 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998247 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 16 08:39:12.000754 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998250 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 16 08:39:12.000754 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998253 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 16 08:39:12.000754 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998255 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 16 08:39:12.000754 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998258 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 16 08:39:12.000754 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998260 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 16 08:39:12.000754 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998262 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 16 08:39:12.000754 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998265 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 16 08:39:12.000754 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998267 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 16 08:39:12.000754 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998270 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 16 08:39:12.000754 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998272 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 16 08:39:12.000754 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998274 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 16 08:39:12.000754 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998277 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 16 08:39:12.001293 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998279 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 16 08:39:12.001293 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998282 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 16 08:39:12.001293 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998284 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 16 08:39:12.001293 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998286 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 16 08:39:12.001293 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998289 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 16 08:39:12.001293 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998291 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 16 08:39:12.001293 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998294 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 16 08:39:12.001293 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998296 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 16 08:39:12.001293 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998298 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 16 08:39:12.001293 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998300 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 16 08:39:12.001293 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998303 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 16 08:39:12.001293 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998305 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 16 08:39:12.001293 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998307 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 16 08:39:12.001293 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998310 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 16 08:39:12.001293 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998312 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 16 08:39:12.001293 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998314 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 16 08:39:12.001293 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998317 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 16 08:39:12.001293 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998319 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 16 08:39:12.001293 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998321 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 16 08:39:12.001293 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998324 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 16 08:39:12.001756 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998326 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 16 08:39:12.001756 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998328 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 16 08:39:12.001756 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998331 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 16 08:39:12.001756 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998333 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 16 08:39:12.001756 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998336 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 16 08:39:12.001756 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998339 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 16 08:39:12.001756 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998341 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 16 08:39:12.001756 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998344 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 16 08:39:12.001756 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998347 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 16 08:39:12.001756 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998350 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 16 08:39:12.001756 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998353 2578 feature_gate.go:328] unrecognized feature gate: Example2 Apr 16 08:39:12.001756 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:11.998355 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 16 08:39:12.001756 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.998360 2578 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Apr 16 08:39:12.001756 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:11.999058 2578 server.go:962] "Client rotation is on, will bootstrap in background" Apr 16 08:39:12.001756 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.000937 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Apr 16 08:39:12.002137 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.001758 2578 server.go:1019] "Starting client certificate rotation" Apr 16 08:39:12.002137 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.001867 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Apr 16 08:39:12.002137 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.001912 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Apr 16 08:39:12.027096 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.027079 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Apr 16 08:39:12.029409 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.029390 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Apr 16 08:39:12.043821 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.043799 2578 log.go:25] "Validated CRI v1 runtime API" Apr 16 08:39:12.050436 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.050410 2578 log.go:25] "Validated CRI v1 image API" Apr 16 08:39:12.051826 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.051810 2578 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Apr 16 08:39:12.054156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.054137 2578 fs.go:135] Filesystem UUIDs: map[3210ad0f-b954-428e-b942-425ecd122fec:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 d6987882-c9b8-46ad-9702-4f1ef5fac5ca:/dev/nvme0n1p3] Apr 16 08:39:12.054231 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.054156 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Apr 16 08:39:12.059619 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.059601 2578 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Apr 16 08:39:12.060197 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.060101 2578 manager.go:217] Machine: {Timestamp:2026-04-16 08:39:12.058052294 +0000 UTC m=+0.427036342 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3150639 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2094af0f3e94752d676c2bed10c3de SystemUUID:ec2094af-0f3e-9475-2d67-6c2bed10c3de BootID:a6085bbd-12df-40d6-86b2-de46fc2357ce Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6098944 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:8f:8c:ae:c7:c3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:8f:8c:ae:c7:c3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:aa:50:f2:a1:af:f4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Apr 16 08:39:12.060197 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.060193 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Apr 16 08:39:12.060286 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.060267 2578 manager.go:233] Version: {KernelVersion:5.14.0-570.104.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260401-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Apr 16 08:39:12.062742 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.062718 2578 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Apr 16 08:39:12.062877 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.062745 2578 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-84.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Apr 16 08:39:12.062918 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.062886 2578 topology_manager.go:138] "Creating topology manager with none policy" Apr 16 08:39:12.062918 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.062894 2578 container_manager_linux.go:306] "Creating device plugin manager" Apr 16 08:39:12.062918 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.062906 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Apr 16 08:39:12.064451 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.064439 2578 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Apr 16 08:39:12.066021 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.066012 2578 state_mem.go:36] "Initialized new in-memory state store" Apr 16 08:39:12.066127 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.066118 2578 server.go:1267] "Using root directory" path="/var/lib/kubelet" Apr 16 08:39:12.068653 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.068644 2578 kubelet.go:491] "Attempting to sync node with API server" Apr 16 08:39:12.068690 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.068663 2578 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Apr 16 08:39:12.068690 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.068677 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Apr 16 08:39:12.068690 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.068685 2578 kubelet.go:397] "Adding apiserver pod source" Apr 16 08:39:12.068791 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.068693 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Apr 16 08:39:12.069775 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.069755 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Apr 16 08:39:12.069807 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.069786 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Apr 16 08:39:12.073067 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.073040 2578 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.10-2.rhaos4.20.gita4d0894.el9" apiVersion="v1" Apr 16 08:39:12.075310 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.075298 2578 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Apr 16 08:39:12.076790 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.076780 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Apr 16 08:39:12.076837 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.076797 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Apr 16 08:39:12.076837 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.076803 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Apr 16 08:39:12.076837 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.076809 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Apr 16 08:39:12.076837 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.076815 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Apr 16 08:39:12.076837 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.076821 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Apr 16 08:39:12.076837 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.076829 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Apr 16 08:39:12.076837 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.076834 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Apr 16 08:39:12.077004 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.076840 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Apr 16 08:39:12.077004 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.076846 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Apr 16 08:39:12.077004 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.076860 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Apr 16 08:39:12.077004 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.076868 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Apr 16 08:39:12.077914 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.077904 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Apr 16 08:39:12.077914 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.077914 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Apr 16 08:39:12.078475 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.078459 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-f7wpf" Apr 16 08:39:12.079794 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:12.079760 2578 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Apr 16 08:39:12.079872 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:12.079850 2578 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-84.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Apr 16 08:39:12.081368 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.081356 2578 watchdog_linux.go:99] "Systemd watchdog is not enabled" Apr 16 08:39:12.081434 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.081389 2578 server.go:1295] "Started kubelet" Apr 16 08:39:12.081490 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.081464 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Apr 16 08:39:12.081608 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.081555 2578 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Apr 16 08:39:12.081655 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.081636 2578 server_v1.go:47] "podresources" method="list" useActivePods=true Apr 16 08:39:12.082132 ip-10-0-139-84 systemd[1]: Started Kubernetes Kubelet. Apr 16 08:39:12.082552 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.082535 2578 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-84.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Apr 16 08:39:12.082623 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.082610 2578 server.go:317] "Adding debug handlers to kubelet server" Apr 16 08:39:12.082891 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.082876 2578 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Apr 16 08:39:12.085620 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.085605 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-f7wpf" Apr 16 08:39:12.090015 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.089994 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Apr 16 08:39:12.090111 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:12.090077 2578 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Apr 16 08:39:12.090685 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.090673 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Apr 16 08:39:12.091301 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.091284 2578 factory.go:55] Registering systemd factory Apr 16 08:39:12.091301 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.091304 2578 factory.go:223] Registration of the systemd container factory successfully Apr 16 08:39:12.091423 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.091365 2578 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Apr 16 08:39:12.091423 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:12.091404 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-84.ec2.internal\" not found" Apr 16 08:39:12.091567 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.091466 2578 volume_manager.go:295] "The desired_state_of_world populator starts" Apr 16 08:39:12.091567 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.091478 2578 volume_manager.go:297] "Starting Kubelet Volume Manager" Apr 16 08:39:12.091567 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.091492 2578 factory.go:153] Registering CRI-O factory Apr 16 08:39:12.091567 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.091504 2578 factory.go:223] Registration of the crio container factory successfully Apr 16 08:39:12.091567 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.091557 2578 reconstruct.go:97] "Volume reconstruction finished" Apr 16 08:39:12.091567 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.091566 2578 reconciler.go:26] "Reconciler: start to sync state" Apr 16 08:39:12.091567 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.091583 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Apr 16 08:39:12.091567 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.091617 2578 factory.go:103] Registering Raw factory Apr 16 08:39:12.091567 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.091631 2578 manager.go:1196] Started watching for new ooms in manager Apr 16 08:39:12.092212 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.092085 2578 manager.go:319] Starting recovery of all containers Apr 16 08:39:12.093680 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.093657 2578 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Apr 16 08:39:12.097497 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:12.097477 2578 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-84.ec2.internal\" not found" node="ip-10-0-139-84.ec2.internal" Apr 16 08:39:12.101462 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.101447 2578 manager.go:324] Recovery completed Apr 16 08:39:12.102960 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:12.102933 2578 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Apr 16 08:39:12.105800 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.105785 2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 16 08:39:12.108566 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.108553 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-84.ec2.internal" event="NodeHasSufficientMemory" Apr 16 08:39:12.108626 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.108583 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-84.ec2.internal" event="NodeHasNoDiskPressure" Apr 16 08:39:12.108626 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.108600 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-84.ec2.internal" event="NodeHasSufficientPID" Apr 16 08:39:12.109033 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.109019 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Apr 16 08:39:12.109089 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.109035 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Apr 16 08:39:12.109089 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.109050 2578 state_mem.go:36] "Initialized new in-memory state store" Apr 16 08:39:12.111589 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.111578 2578 policy_none.go:49] "None policy: Start" Apr 16 08:39:12.111624 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.111593 2578 memory_manager.go:186] "Starting memorymanager" policy="None" Apr 16 08:39:12.111624 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.111602 2578 state_mem.go:35] "Initializing new in-memory state store" Apr 16 08:39:12.150412 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.150399 2578 manager.go:341] "Starting Device Plugin manager" Apr 16 08:39:12.150482 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:12.150427 2578 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Apr 16 08:39:12.150482 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.150437 2578 server.go:85] "Starting device plugin registration server" Apr 16 08:39:12.150656 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.150643 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Apr 16 08:39:12.150697 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.150657 2578 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Apr 16 08:39:12.151067 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.150811 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Apr 16 08:39:12.151067 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.150919 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Apr 16 08:39:12.151067 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.150928 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Apr 16 08:39:12.151354 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:12.151339 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Apr 16 08:39:12.151391 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:12.151378 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-84.ec2.internal\" not found" Apr 16 08:39:12.207883 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.207846 2578 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Apr 16 08:39:12.209149 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.209135 2578 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Apr 16 08:39:12.209201 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.209164 2578 status_manager.go:230] "Starting to sync pod status with apiserver" Apr 16 08:39:12.209201 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.209182 2578 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Apr 16 08:39:12.209201 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.209188 2578 kubelet.go:2451] "Starting kubelet main sync loop" Apr 16 08:39:12.209310 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:12.209253 2578 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Apr 16 08:39:12.213248 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.213230 2578 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Apr 16 08:39:12.251413 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.251365 2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 16 08:39:12.252610 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.252592 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-84.ec2.internal" event="NodeHasSufficientMemory" Apr 16 08:39:12.252678 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.252626 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-84.ec2.internal" event="NodeHasNoDiskPressure" Apr 16 08:39:12.252678 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.252640 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-84.ec2.internal" event="NodeHasSufficientPID" Apr 16 08:39:12.252678 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.252661 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-84.ec2.internal" Apr 16 08:39:12.261324 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.261306 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-84.ec2.internal" Apr 16 08:39:12.261324 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:12.261325 2578 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-84.ec2.internal\": node \"ip-10-0-139-84.ec2.internal\" not found" Apr 16 08:39:12.273176 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:12.273160 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-84.ec2.internal\" not found" Apr 16 08:39:12.309332 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.309301 2578 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-84.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-84.ec2.internal"] Apr 16 08:39:12.309415 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.309370 2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 16 08:39:12.310917 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.310904 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-84.ec2.internal" event="NodeHasSufficientMemory" Apr 16 08:39:12.310980 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.310928 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-84.ec2.internal" event="NodeHasNoDiskPressure" Apr 16 08:39:12.310980 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.310937 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-84.ec2.internal" event="NodeHasSufficientPID" Apr 16 08:39:12.312270 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.312259 2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 16 08:39:12.312401 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.312388 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-84.ec2.internal" Apr 16 08:39:12.312439 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.312413 2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 16 08:39:12.312958 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.312935 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-84.ec2.internal" event="NodeHasSufficientMemory" Apr 16 08:39:12.313038 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.312935 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-84.ec2.internal" event="NodeHasSufficientMemory" Apr 16 08:39:12.313038 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.312986 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-84.ec2.internal" event="NodeHasNoDiskPressure" Apr 16 08:39:12.313038 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.312997 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-84.ec2.internal" event="NodeHasSufficientPID" Apr 16 08:39:12.313038 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.312964 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-84.ec2.internal" event="NodeHasNoDiskPressure" Apr 16 08:39:12.313216 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.313055 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-84.ec2.internal" event="NodeHasSufficientPID" Apr 16 08:39:12.314862 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.314847 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-84.ec2.internal" Apr 16 08:39:12.314916 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.314874 2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 16 08:39:12.315567 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.315547 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-84.ec2.internal" event="NodeHasSufficientMemory" Apr 16 08:39:12.315646 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.315580 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-84.ec2.internal" event="NodeHasNoDiskPressure" Apr 16 08:39:12.315646 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.315595 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-84.ec2.internal" event="NodeHasSufficientPID" Apr 16 08:39:12.341224 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:12.341205 2578 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-84.ec2.internal\" not found" node="ip-10-0-139-84.ec2.internal" Apr 16 08:39:12.345300 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:12.345287 2578 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-84.ec2.internal\" not found" node="ip-10-0-139-84.ec2.internal" Apr 16 08:39:12.373664 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:12.373648 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-84.ec2.internal\" not found" Apr 16 08:39:12.393834 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.393820 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/267abad74bc5a64078f92c3f8060b88d-config\") pod \"kube-apiserver-proxy-ip-10-0-139-84.ec2.internal\" (UID: \"267abad74bc5a64078f92c3f8060b88d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-84.ec2.internal" Apr 16 08:39:12.393907 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.393842 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5a1f89a5e7a6831921722fd68212b6b3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-84.ec2.internal\" (UID: \"5a1f89a5e7a6831921722fd68212b6b3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-84.ec2.internal" Apr 16 08:39:12.393907 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.393859 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a1f89a5e7a6831921722fd68212b6b3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-84.ec2.internal\" (UID: \"5a1f89a5e7a6831921722fd68212b6b3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-84.ec2.internal" Apr 16 08:39:12.474529 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:12.474503 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-84.ec2.internal\" not found" Apr 16 08:39:12.494849 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.494830 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5a1f89a5e7a6831921722fd68212b6b3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-84.ec2.internal\" (UID: \"5a1f89a5e7a6831921722fd68212b6b3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-84.ec2.internal" Apr 16 08:39:12.494903 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.494890 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a1f89a5e7a6831921722fd68212b6b3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-84.ec2.internal\" (UID: \"5a1f89a5e7a6831921722fd68212b6b3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-84.ec2.internal" Apr 16 08:39:12.494940 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.494853 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a1f89a5e7a6831921722fd68212b6b3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-84.ec2.internal\" (UID: \"5a1f89a5e7a6831921722fd68212b6b3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-84.ec2.internal" Apr 16 08:39:12.494940 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.494914 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5a1f89a5e7a6831921722fd68212b6b3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-84.ec2.internal\" (UID: \"5a1f89a5e7a6831921722fd68212b6b3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-84.ec2.internal" Apr 16 08:39:12.495007 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.494942 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/267abad74bc5a64078f92c3f8060b88d-config\") pod \"kube-apiserver-proxy-ip-10-0-139-84.ec2.internal\" (UID: \"267abad74bc5a64078f92c3f8060b88d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-84.ec2.internal" Apr 16 08:39:12.495007 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.494963 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/267abad74bc5a64078f92c3f8060b88d-config\") pod \"kube-apiserver-proxy-ip-10-0-139-84.ec2.internal\" (UID: \"267abad74bc5a64078f92c3f8060b88d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-84.ec2.internal" Apr 16 08:39:12.575289 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:12.575226 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-84.ec2.internal\" not found" Apr 16 08:39:12.644779 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.644749 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-84.ec2.internal" Apr 16 08:39:12.648319 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.648290 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-84.ec2.internal" Apr 16 08:39:12.676387 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:12.676261 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-84.ec2.internal\" not found" Apr 16 08:39:12.776710 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:12.776689 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-84.ec2.internal\" not found" Apr 16 08:39:12.877184 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:12.877121 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-84.ec2.internal\" not found" Apr 16 08:39:12.977629 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:12.977606 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-84.ec2.internal\" not found" Apr 16 08:39:12.979678 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:12.979664 2578 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Apr 16 08:39:13.002145 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:13.002129 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Apr 16 08:39:13.002717 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:13.002242 2578 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Apr 16 08:39:13.002717 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:13.002283 2578 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Apr 16 08:39:13.002717 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:13.002295 2578 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Apr 16 08:39:13.077679 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:13.077658 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-84.ec2.internal\" not found" Apr 16 08:39:13.088058 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:13.088019 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-04-15 08:34:12 +0000 UTC" deadline="2027-10-23 18:12:49.43323587 +0000 UTC" Apr 16 08:39:13.088058 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:13.088055 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13329h33m36.3451842s" Apr 16 08:39:13.090134 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:13.090117 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Apr 16 08:39:13.099229 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:13.099209 2578 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Apr 16 08:39:13.119257 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:13.119235 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gsmrd" Apr 16 08:39:13.126226 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:13.126209 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gsmrd" Apr 16 08:39:13.130732 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:13.130681 2578 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Apr 16 08:39:13.165527 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:13.165501 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a1f89a5e7a6831921722fd68212b6b3.slice/crio-b2a77f139c83f35966c68ed5698def9f308c9f63326cf0d6cad650eb456caa2e WatchSource:0}: Error finding container b2a77f139c83f35966c68ed5698def9f308c9f63326cf0d6cad650eb456caa2e: Status 404 returned error can't find the container with id b2a77f139c83f35966c68ed5698def9f308c9f63326cf0d6cad650eb456caa2e Apr 16 08:39:13.165999 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:13.165986 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod267abad74bc5a64078f92c3f8060b88d.slice/crio-2aad081e5db973ed423e42c43d2aed10f364255ea718708d0cb32851a756dacc WatchSource:0}: Error finding container 2aad081e5db973ed423e42c43d2aed10f364255ea718708d0cb32851a756dacc: Status 404 returned error can't find the container with id 2aad081e5db973ed423e42c43d2aed10f364255ea718708d0cb32851a756dacc Apr 16 08:39:13.170644 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:13.170630 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 16 08:39:13.191056 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:13.191034 2578 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-84.ec2.internal" Apr 16 08:39:13.200991 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:13.200970 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Apr 16 08:39:13.203760 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:13.203745 2578 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-84.ec2.internal" Apr 16 08:39:13.210309 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:13.210278 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Apr 16 08:39:13.211703 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:13.211669 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-84.ec2.internal" event={"ID":"5a1f89a5e7a6831921722fd68212b6b3","Type":"ContainerStarted","Data":"b2a77f139c83f35966c68ed5698def9f308c9f63326cf0d6cad650eb456caa2e"} Apr 16 08:39:13.212646 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:13.212626 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-84.ec2.internal" event={"ID":"267abad74bc5a64078f92c3f8060b88d","Type":"ContainerStarted","Data":"2aad081e5db973ed423e42c43d2aed10f364255ea718708d0cb32851a756dacc"} Apr 16 08:39:14.069784 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.069736 2578 apiserver.go:52] "Watching apiserver" Apr 16 08:39:14.074949 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.074930 2578 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Apr 16 08:39:14.076040 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.076018 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-mnwk4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-84.ec2.internal","openshift-multus/multus-additional-cni-plugins-7kxqz","openshift-multus/multus-f9c2h","openshift-multus/network-metrics-daemon-fxtnv","openshift-network-diagnostics/network-check-target-5l2xc","openshift-network-operator/iptables-alerter-lvthh","kube-system/kube-apiserver-proxy-ip-10-0-139-84.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m","openshift-dns/node-resolver-qxxqp","openshift-image-registry/node-ca-7rgcb","openshift-ovn-kubernetes/ovnkube-node-6jwz6","kube-system/konnectivity-agent-6k8v5"] Apr 16 08:39:14.077302 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.077284 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lvthh" Apr 16 08:39:14.078451 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.078416 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7kxqz" Apr 16 08:39:14.080741 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.079894 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.080741 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.080266 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Apr 16 08:39:14.080741 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.080701 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zlvc6\"" Apr 16 08:39:14.080957 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.080940 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-96fgd\"" Apr 16 08:39:14.081031 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.081016 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Apr 16 08:39:14.081337 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.081320 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Apr 16 08:39:14.081677 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.081663 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Apr 16 08:39:14.082704 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.082088 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Apr 16 08:39:14.082704 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.082102 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Apr 16 08:39:14.082704 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.082441 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:14.082704 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:14.082547 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxtnv" podUID="57f278d8-41a1-4600-a089-04d841c7d446" Apr 16 08:39:14.083074 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.083057 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Apr 16 08:39:14.083807 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.083729 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Apr 16 08:39:14.083893 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.083872 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rpk6n\"" Apr 16 08:39:14.084897 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.084877 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Apr 16 08:39:14.085784 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.085705 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:39:14.085873 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:14.085791 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5l2xc" podUID="e285ae7e-c21c-46b9-b705-ce4365740bd9" Apr 16 08:39:14.086963 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.086877 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" Apr 16 08:39:14.088321 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.088303 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.088412 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.088397 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qxxqp" Apr 16 08:39:14.088472 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.088452 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Apr 16 08:39:14.088901 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.088876 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Apr 16 08:39:14.088991 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.088923 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gjpcq\"" Apr 16 08:39:14.088991 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.088923 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Apr 16 08:39:14.089718 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.089701 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7rgcb" Apr 16 08:39:14.090309 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.090290 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Apr 16 08:39:14.090393 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.090346 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-t5cfx\"" Apr 16 08:39:14.090452 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.090416 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Apr 16 08:39:14.090597 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.090579 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Apr 16 08:39:14.090715 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.090688 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8gc9p\"" Apr 16 08:39:14.091031 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.091016 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Apr 16 08:39:14.091796 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.091283 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.091796 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.091439 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Apr 16 08:39:14.091796 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.091538 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-szt97\"" Apr 16 08:39:14.091796 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.091757 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Apr 16 08:39:14.092054 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.091879 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Apr 16 08:39:14.092778 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.092750 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Apr 16 08:39:14.092935 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.092920 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6k8v5" Apr 16 08:39:14.093033 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.093019 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Apr 16 08:39:14.093487 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.093471 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Apr 16 08:39:14.093612 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.093597 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Apr 16 08:39:14.093674 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.093658 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Apr 16 08:39:14.093712 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.093697 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Apr 16 08:39:14.093851 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.093835 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zk9zs\"" Apr 16 08:39:14.094628 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.094515 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Apr 16 08:39:14.094795 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.094782 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fgwx2\"" Apr 16 08:39:14.094877 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.094795 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Apr 16 08:39:14.102711 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.102692 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-run-ovn\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.102822 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.102727 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d38f82dc-5ceb-4b6a-b918-e2b0085a9949-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7kxqz\" (UID: \"d38f82dc-5ceb-4b6a-b918-e2b0085a9949\") " pod="openshift-multus/multus-additional-cni-plugins-7kxqz" Apr 16 08:39:14.102822 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.102747 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-cni-binary-copy\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.102822 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.102793 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-run\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.102822 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.102822 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-var-lib-kubelet\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.103012 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.102873 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69ff05e4-5126-420e-ba65-9f88a7fec110-tmp-dir\") pod \"node-resolver-qxxqp\" (UID: \"69ff05e4-5126-420e-ba65-9f88a7fec110\") " pod="openshift-dns/node-resolver-qxxqp" Apr 16 08:39:14.103012 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.102894 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpcq9\" (UniqueName: \"kubernetes.io/projected/57f278d8-41a1-4600-a089-04d841c7d446-kube-api-access-cpcq9\") pod \"network-metrics-daemon-fxtnv\" (UID: \"57f278d8-41a1-4600-a089-04d841c7d446\") " pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:14.103012 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.102953 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-host-kubelet\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.103012 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103007 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-host-slash\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.103180 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103047 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.103180 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103074 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-ovnkube-config\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.103180 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103124 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-system-cni-dir\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.103180 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103153 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-host-var-lib-kubelet\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.103340 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103179 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d38f82dc-5ceb-4b6a-b918-e2b0085a9949-cnibin\") pod \"multus-additional-cni-plugins-7kxqz\" (UID: \"d38f82dc-5ceb-4b6a-b918-e2b0085a9949\") " pod="openshift-multus/multus-additional-cni-plugins-7kxqz" Apr 16 08:39:14.103340 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103201 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-os-release\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.103340 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103225 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-hostroot\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.103340 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103265 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-etc-sysctl-d\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.103340 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103296 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1a9eef0f-407a-4515-a994-782aae576f13-etc-tuned\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.103340 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103325 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f2a856ae-d19f-4c9d-bb77-e04127e50283-etc-selinux\") pod \"aws-ebs-csi-driver-node-4nl8m\" (UID: \"f2a856ae-d19f-4c9d-bb77-e04127e50283\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" Apr 16 08:39:14.103579 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103349 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-host-cni-bin\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.103579 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103396 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gk5xf\" (UniqueName: \"kubernetes.io/projected/d38f82dc-5ceb-4b6a-b918-e2b0085a9949-kube-api-access-gk5xf\") pod \"multus-additional-cni-plugins-7kxqz\" (UID: \"d38f82dc-5ceb-4b6a-b918-e2b0085a9949\") " pod="openshift-multus/multus-additional-cni-plugins-7kxqz" Apr 16 08:39:14.103579 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103413 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-multus-socket-dir-parent\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.103579 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103427 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/69ff05e4-5126-420e-ba65-9f88a7fec110-hosts-file\") pod \"node-resolver-qxxqp\" (UID: \"69ff05e4-5126-420e-ba65-9f88a7fec110\") " pod="openshift-dns/node-resolver-qxxqp" Apr 16 08:39:14.103579 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103469 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-host-run-ovn-kubernetes\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.103579 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103513 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-ovnkube-script-lib\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.103579 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103542 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d38f82dc-5ceb-4b6a-b918-e2b0085a9949-os-release\") pod \"multus-additional-cni-plugins-7kxqz\" (UID: \"d38f82dc-5ceb-4b6a-b918-e2b0085a9949\") " pod="openshift-multus/multus-additional-cni-plugins-7kxqz" Apr 16 08:39:14.103579 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103567 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d38f82dc-5ceb-4b6a-b918-e2b0085a9949-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7kxqz\" (UID: \"d38f82dc-5ceb-4b6a-b918-e2b0085a9949\") " pod="openshift-multus/multus-additional-cni-plugins-7kxqz" Apr 16 08:39:14.103916 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103597 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f2a856ae-d19f-4c9d-bb77-e04127e50283-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4nl8m\" (UID: \"f2a856ae-d19f-4c9d-bb77-e04127e50283\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" Apr 16 08:39:14.103916 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103626 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-multus-cni-dir\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.103916 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103649 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-etc-sysconfig\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.103916 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103697 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5q6q\" (UniqueName: \"kubernetes.io/projected/1a9eef0f-407a-4515-a994-782aae576f13-kube-api-access-s5q6q\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.103916 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103731 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17b0a66e-5c6b-4926-b67d-e916dd6009d3-host\") pod \"node-ca-7rgcb\" (UID: \"17b0a66e-5c6b-4926-b67d-e916dd6009d3\") " pod="openshift-image-registry/node-ca-7rgcb" Apr 16 08:39:14.103916 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103755 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57f278d8-41a1-4600-a089-04d841c7d446-metrics-certs\") pod \"network-metrics-daemon-fxtnv\" (UID: \"57f278d8-41a1-4600-a089-04d841c7d446\") " pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:14.103916 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103792 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-systemd-units\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.103916 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103813 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-host-run-netns\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.103916 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103845 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-var-lib-openvswitch\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.103916 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103865 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw5kf\" (UniqueName: \"kubernetes.io/projected/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-kube-api-access-kw5kf\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.103916 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103885 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-host-run-netns\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.103916 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103909 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-host-var-lib-cni-multus\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.104370 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103929 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f2a856ae-d19f-4c9d-bb77-e04127e50283-socket-dir\") pod \"aws-ebs-csi-driver-node-4nl8m\" (UID: \"f2a856ae-d19f-4c9d-bb77-e04127e50283\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" Apr 16 08:39:14.104370 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103949 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f2a856ae-d19f-4c9d-bb77-e04127e50283-sys-fs\") pod \"aws-ebs-csi-driver-node-4nl8m\" (UID: \"f2a856ae-d19f-4c9d-bb77-e04127e50283\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" Apr 16 08:39:14.104370 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.103969 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-host-var-lib-cni-bin\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.104370 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104006 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-sys\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.104370 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104035 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f2a856ae-d19f-4c9d-bb77-e04127e50283-device-dir\") pod \"aws-ebs-csi-driver-node-4nl8m\" (UID: \"f2a856ae-d19f-4c9d-bb77-e04127e50283\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" Apr 16 08:39:14.104370 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104064 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z867x\" (UniqueName: \"kubernetes.io/projected/f7b551d1-cdb9-462e-bf68-8b67b6304fff-kube-api-access-z867x\") pod \"iptables-alerter-lvthh\" (UID: \"f7b551d1-cdb9-462e-bf68-8b67b6304fff\") " pod="openshift-network-operator/iptables-alerter-lvthh" Apr 16 08:39:14.104370 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104098 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddmc5\" (UniqueName: \"kubernetes.io/projected/e285ae7e-c21c-46b9-b705-ce4365740bd9-kube-api-access-ddmc5\") pod \"network-check-target-5l2xc\" (UID: \"e285ae7e-c21c-46b9-b705-ce4365740bd9\") " pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:39:14.104370 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104133 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f7b551d1-cdb9-462e-bf68-8b67b6304fff-host-slash\") pod \"iptables-alerter-lvthh\" (UID: \"f7b551d1-cdb9-462e-bf68-8b67b6304fff\") " pod="openshift-network-operator/iptables-alerter-lvthh" Apr 16 08:39:14.104370 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104159 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d38f82dc-5ceb-4b6a-b918-e2b0085a9949-system-cni-dir\") pod \"multus-additional-cni-plugins-7kxqz\" (UID: \"d38f82dc-5ceb-4b6a-b918-e2b0085a9949\") " pod="openshift-multus/multus-additional-cni-plugins-7kxqz" Apr 16 08:39:14.104370 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104203 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-host-run-k8s-cni-cncf-io\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.104370 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104217 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-etc-modprobe-d\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.104370 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104240 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxk69\" (UniqueName: \"kubernetes.io/projected/17b0a66e-5c6b-4926-b67d-e916dd6009d3-kube-api-access-vxk69\") pod \"node-ca-7rgcb\" (UID: \"17b0a66e-5c6b-4926-b67d-e916dd6009d3\") " pod="openshift-image-registry/node-ca-7rgcb" Apr 16 08:39:14.104370 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104254 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-etc-openvswitch\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.104370 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104281 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-run-openvswitch\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.104370 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104300 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8e7b065b-9f25-4b3b-aa65-785e10aec00d-agent-certs\") pod \"konnectivity-agent-6k8v5\" (UID: \"8e7b065b-9f25-4b3b-aa65-785e10aec00d\") " pod="kube-system/konnectivity-agent-6k8v5" Apr 16 08:39:14.104370 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104322 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f7b551d1-cdb9-462e-bf68-8b67b6304fff-iptables-alerter-script\") pod \"iptables-alerter-lvthh\" (UID: \"f7b551d1-cdb9-462e-bf68-8b67b6304fff\") " pod="openshift-network-operator/iptables-alerter-lvthh" Apr 16 08:39:14.104979 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104358 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-log-socket\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.104979 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104385 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d38f82dc-5ceb-4b6a-b918-e2b0085a9949-cni-binary-copy\") pod \"multus-additional-cni-plugins-7kxqz\" (UID: \"d38f82dc-5ceb-4b6a-b918-e2b0085a9949\") " pod="openshift-multus/multus-additional-cni-plugins-7kxqz" Apr 16 08:39:14.104979 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104405 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-etc-kubernetes\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.104979 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104425 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-etc-kubernetes\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.104979 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104446 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-etc-sysctl-conf\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.104979 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104466 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-etc-systemd\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.104979 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104484 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-node-log\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.104979 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104526 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-multus-daemon-config\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.104979 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104546 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-lib-modules\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.104979 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104559 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a9eef0f-407a-4515-a994-782aae576f13-tmp\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.104979 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104581 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/17b0a66e-5c6b-4926-b67d-e916dd6009d3-serviceca\") pod \"node-ca-7rgcb\" (UID: \"17b0a66e-5c6b-4926-b67d-e916dd6009d3\") " pod="openshift-image-registry/node-ca-7rgcb" Apr 16 08:39:14.104979 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104602 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mzzf\" (UniqueName: \"kubernetes.io/projected/f2a856ae-d19f-4c9d-bb77-e04127e50283-kube-api-access-2mzzf\") pod \"aws-ebs-csi-driver-node-4nl8m\" (UID: \"f2a856ae-d19f-4c9d-bb77-e04127e50283\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" Apr 16 08:39:14.104979 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104629 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-run-systemd\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.104979 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104649 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-host-cni-netd\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.104979 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104671 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8e7b065b-9f25-4b3b-aa65-785e10aec00d-konnectivity-ca\") pod \"konnectivity-agent-6k8v5\" (UID: \"8e7b065b-9f25-4b3b-aa65-785e10aec00d\") " pod="kube-system/konnectivity-agent-6k8v5" Apr 16 08:39:14.104979 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104691 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-host-run-multus-certs\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.104979 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104709 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf9d6\" (UniqueName: \"kubernetes.io/projected/69ff05e4-5126-420e-ba65-9f88a7fec110-kube-api-access-zf9d6\") pod \"node-resolver-qxxqp\" (UID: \"69ff05e4-5126-420e-ba65-9f88a7fec110\") " pod="openshift-dns/node-resolver-qxxqp" Apr 16 08:39:14.105648 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104723 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-env-overrides\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.105648 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104736 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-ovn-node-metrics-cert\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.105648 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104752 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-cnibin\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.105648 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104800 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-multus-conf-dir\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.105648 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104822 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r47mj\" (UniqueName: \"kubernetes.io/projected/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-kube-api-access-r47mj\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.105648 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104850 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f2a856ae-d19f-4c9d-bb77-e04127e50283-registration-dir\") pod \"aws-ebs-csi-driver-node-4nl8m\" (UID: \"f2a856ae-d19f-4c9d-bb77-e04127e50283\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" Apr 16 08:39:14.105648 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104910 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d38f82dc-5ceb-4b6a-b918-e2b0085a9949-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7kxqz\" (UID: \"d38f82dc-5ceb-4b6a-b918-e2b0085a9949\") " pod="openshift-multus/multus-additional-cni-plugins-7kxqz" Apr 16 08:39:14.105648 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.104942 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-host\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.127168 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.127138 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-04-15 08:34:13 +0000 UTC" deadline="2027-11-03 16:27:16.268405 +0000 UTC" Apr 16 08:39:14.127168 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.127160 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13591h48m2.141248193s" Apr 16 08:39:14.192356 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.192335 2578 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Apr 16 08:39:14.205852 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.205827 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-multus-socket-dir-parent\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.205960 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.205861 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/69ff05e4-5126-420e-ba65-9f88a7fec110-hosts-file\") pod \"node-resolver-qxxqp\" (UID: \"69ff05e4-5126-420e-ba65-9f88a7fec110\") " pod="openshift-dns/node-resolver-qxxqp" Apr 16 08:39:14.205960 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.205885 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-host-run-ovn-kubernetes\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.205960 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.205931 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-host-run-ovn-kubernetes\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.205960 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.205949 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-multus-socket-dir-parent\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.206156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.205959 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/69ff05e4-5126-420e-ba65-9f88a7fec110-hosts-file\") pod \"node-resolver-qxxqp\" (UID: \"69ff05e4-5126-420e-ba65-9f88a7fec110\") " pod="openshift-dns/node-resolver-qxxqp" Apr 16 08:39:14.206156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.205951 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-ovnkube-script-lib\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.206156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206075 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d38f82dc-5ceb-4b6a-b918-e2b0085a9949-os-release\") pod \"multus-additional-cni-plugins-7kxqz\" (UID: \"d38f82dc-5ceb-4b6a-b918-e2b0085a9949\") " pod="openshift-multus/multus-additional-cni-plugins-7kxqz" Apr 16 08:39:14.206156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206105 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d38f82dc-5ceb-4b6a-b918-e2b0085a9949-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7kxqz\" (UID: \"d38f82dc-5ceb-4b6a-b918-e2b0085a9949\") " pod="openshift-multus/multus-additional-cni-plugins-7kxqz" Apr 16 08:39:14.206156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206134 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f2a856ae-d19f-4c9d-bb77-e04127e50283-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4nl8m\" (UID: \"f2a856ae-d19f-4c9d-bb77-e04127e50283\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" Apr 16 08:39:14.206372 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206160 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-multus-cni-dir\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.206372 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206161 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d38f82dc-5ceb-4b6a-b918-e2b0085a9949-os-release\") pod \"multus-additional-cni-plugins-7kxqz\" (UID: \"d38f82dc-5ceb-4b6a-b918-e2b0085a9949\") " pod="openshift-multus/multus-additional-cni-plugins-7kxqz" Apr 16 08:39:14.206372 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206184 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-etc-sysconfig\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.206372 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206222 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f2a856ae-d19f-4c9d-bb77-e04127e50283-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4nl8m\" (UID: \"f2a856ae-d19f-4c9d-bb77-e04127e50283\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" Apr 16 08:39:14.206372 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206223 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s5q6q\" (UniqueName: \"kubernetes.io/projected/1a9eef0f-407a-4515-a994-782aae576f13-kube-api-access-s5q6q\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.206372 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206269 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17b0a66e-5c6b-4926-b67d-e916dd6009d3-host\") pod \"node-ca-7rgcb\" (UID: \"17b0a66e-5c6b-4926-b67d-e916dd6009d3\") " pod="openshift-image-registry/node-ca-7rgcb" Apr 16 08:39:14.206372 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206270 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-multus-cni-dir\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.206372 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206335 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57f278d8-41a1-4600-a089-04d841c7d446-metrics-certs\") pod \"network-metrics-daemon-fxtnv\" (UID: \"57f278d8-41a1-4600-a089-04d841c7d446\") " pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:14.206372 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206364 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-systemd-units\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.206372 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206363 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17b0a66e-5c6b-4926-b67d-e916dd6009d3-host\") pod \"node-ca-7rgcb\" (UID: \"17b0a66e-5c6b-4926-b67d-e916dd6009d3\") " pod="openshift-image-registry/node-ca-7rgcb" Apr 16 08:39:14.206759 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206387 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-host-run-netns\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.206759 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206413 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-var-lib-openvswitch\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.206759 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206422 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-systemd-units\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.206759 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206441 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kw5kf\" (UniqueName: \"kubernetes.io/projected/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-kube-api-access-kw5kf\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.206759 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206464 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-host-run-netns\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.206759 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206468 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-host-run-netns\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.206759 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206491 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-ovnkube-script-lib\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.206759 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:14.206506 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:39:14.206759 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206511 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-var-lib-openvswitch\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.206759 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206515 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-host-run-netns\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.206759 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206536 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-host-var-lib-cni-multus\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.206759 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206580 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-host-var-lib-cni-multus\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.206759 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:14.206584 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57f278d8-41a1-4600-a089-04d841c7d446-metrics-certs podName:57f278d8-41a1-4600-a089-04d841c7d446 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:14.706552154 +0000 UTC m=+3.075536190 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57f278d8-41a1-4600-a089-04d841c7d446-metrics-certs") pod "network-metrics-daemon-fxtnv" (UID: "57f278d8-41a1-4600-a089-04d841c7d446") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:39:14.206759 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206610 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f2a856ae-d19f-4c9d-bb77-e04127e50283-socket-dir\") pod \"aws-ebs-csi-driver-node-4nl8m\" (UID: \"f2a856ae-d19f-4c9d-bb77-e04127e50283\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" Apr 16 08:39:14.206759 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206614 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d38f82dc-5ceb-4b6a-b918-e2b0085a9949-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7kxqz\" (UID: \"d38f82dc-5ceb-4b6a-b918-e2b0085a9949\") " pod="openshift-multus/multus-additional-cni-plugins-7kxqz" Apr 16 08:39:14.206759 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206634 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f2a856ae-d19f-4c9d-bb77-e04127e50283-sys-fs\") pod \"aws-ebs-csi-driver-node-4nl8m\" (UID: \"f2a856ae-d19f-4c9d-bb77-e04127e50283\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" Apr 16 08:39:14.206759 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206675 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-host-var-lib-cni-bin\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.207554 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206700 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-sys\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.207554 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206736 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f2a856ae-d19f-4c9d-bb77-e04127e50283-device-dir\") pod \"aws-ebs-csi-driver-node-4nl8m\" (UID: \"f2a856ae-d19f-4c9d-bb77-e04127e50283\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" Apr 16 08:39:14.207554 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206760 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z867x\" (UniqueName: \"kubernetes.io/projected/f7b551d1-cdb9-462e-bf68-8b67b6304fff-kube-api-access-z867x\") pod \"iptables-alerter-lvthh\" (UID: \"f7b551d1-cdb9-462e-bf68-8b67b6304fff\") " pod="openshift-network-operator/iptables-alerter-lvthh" Apr 16 08:39:14.207554 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206754 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f2a856ae-d19f-4c9d-bb77-e04127e50283-sys-fs\") pod \"aws-ebs-csi-driver-node-4nl8m\" (UID: \"f2a856ae-d19f-4c9d-bb77-e04127e50283\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" Apr 16 08:39:14.207554 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206758 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-host-var-lib-cni-bin\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.207554 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206819 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ddmc5\" (UniqueName: \"kubernetes.io/projected/e285ae7e-c21c-46b9-b705-ce4365740bd9-kube-api-access-ddmc5\") pod \"network-check-target-5l2xc\" (UID: \"e285ae7e-c21c-46b9-b705-ce4365740bd9\") " pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:39:14.207554 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206839 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f2a856ae-d19f-4c9d-bb77-e04127e50283-device-dir\") pod \"aws-ebs-csi-driver-node-4nl8m\" (UID: \"f2a856ae-d19f-4c9d-bb77-e04127e50283\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" Apr 16 08:39:14.207554 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206822 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-sys\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.207554 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206850 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f7b551d1-cdb9-462e-bf68-8b67b6304fff-host-slash\") pod \"iptables-alerter-lvthh\" (UID: \"f7b551d1-cdb9-462e-bf68-8b67b6304fff\") " pod="openshift-network-operator/iptables-alerter-lvthh" Apr 16 08:39:14.207554 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206867 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f2a856ae-d19f-4c9d-bb77-e04127e50283-socket-dir\") pod \"aws-ebs-csi-driver-node-4nl8m\" (UID: \"f2a856ae-d19f-4c9d-bb77-e04127e50283\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" Apr 16 08:39:14.207554 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206878 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d38f82dc-5ceb-4b6a-b918-e2b0085a9949-system-cni-dir\") pod \"multus-additional-cni-plugins-7kxqz\" (UID: \"d38f82dc-5ceb-4b6a-b918-e2b0085a9949\") " pod="openshift-multus/multus-additional-cni-plugins-7kxqz" Apr 16 08:39:14.207554 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206916 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-host-run-k8s-cni-cncf-io\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.207554 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206913 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d38f82dc-5ceb-4b6a-b918-e2b0085a9949-system-cni-dir\") pod \"multus-additional-cni-plugins-7kxqz\" (UID: \"d38f82dc-5ceb-4b6a-b918-e2b0085a9949\") " pod="openshift-multus/multus-additional-cni-plugins-7kxqz" Apr 16 08:39:14.207554 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206871 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f7b551d1-cdb9-462e-bf68-8b67b6304fff-host-slash\") pod \"iptables-alerter-lvthh\" (UID: \"f7b551d1-cdb9-462e-bf68-8b67b6304fff\") " pod="openshift-network-operator/iptables-alerter-lvthh" Apr 16 08:39:14.207554 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206946 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-etc-modprobe-d\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.207554 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206971 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vxk69\" (UniqueName: \"kubernetes.io/projected/17b0a66e-5c6b-4926-b67d-e916dd6009d3-kube-api-access-vxk69\") pod \"node-ca-7rgcb\" (UID: \"17b0a66e-5c6b-4926-b67d-e916dd6009d3\") " pod="openshift-image-registry/node-ca-7rgcb" Apr 16 08:39:14.207554 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206977 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-host-run-k8s-cni-cncf-io\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.208386 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206993 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-etc-openvswitch\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.208386 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207018 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-run-openvswitch\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.208386 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207042 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8e7b065b-9f25-4b3b-aa65-785e10aec00d-agent-certs\") pod \"konnectivity-agent-6k8v5\" (UID: \"8e7b065b-9f25-4b3b-aa65-785e10aec00d\") " pod="kube-system/konnectivity-agent-6k8v5" Apr 16 08:39:14.208386 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207049 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-etc-openvswitch\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.208386 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207066 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f7b551d1-cdb9-462e-bf68-8b67b6304fff-iptables-alerter-script\") pod \"iptables-alerter-lvthh\" (UID: \"f7b551d1-cdb9-462e-bf68-8b67b6304fff\") " pod="openshift-network-operator/iptables-alerter-lvthh" Apr 16 08:39:14.208386 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207084 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-etc-modprobe-d\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.208386 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207091 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-log-socket\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.208386 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207093 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-run-openvswitch\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.208386 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.206312 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-etc-sysconfig\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.208386 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207123 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d38f82dc-5ceb-4b6a-b918-e2b0085a9949-cni-binary-copy\") pod \"multus-additional-cni-plugins-7kxqz\" (UID: \"d38f82dc-5ceb-4b6a-b918-e2b0085a9949\") " pod="openshift-multus/multus-additional-cni-plugins-7kxqz" Apr 16 08:39:14.208386 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207144 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-log-socket\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.208386 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207159 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-etc-kubernetes\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.208386 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207178 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-etc-kubernetes\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.208386 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207201 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-etc-sysctl-conf\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.208386 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207211 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-etc-kubernetes\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.208386 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207225 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-etc-systemd\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.208386 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207245 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-node-log\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.208386 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207260 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-etc-kubernetes\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.209185 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207262 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-multus-daemon-config\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.209185 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207301 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-lib-modules\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.209185 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207326 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a9eef0f-407a-4515-a994-782aae576f13-tmp\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.209185 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207341 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-etc-sysctl-conf\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.209185 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207352 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/17b0a66e-5c6b-4926-b67d-e916dd6009d3-serviceca\") pod \"node-ca-7rgcb\" (UID: \"17b0a66e-5c6b-4926-b67d-e916dd6009d3\") " pod="openshift-image-registry/node-ca-7rgcb" Apr 16 08:39:14.209185 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207394 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-etc-systemd\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.209185 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207410 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2mzzf\" (UniqueName: \"kubernetes.io/projected/f2a856ae-d19f-4c9d-bb77-e04127e50283-kube-api-access-2mzzf\") pod \"aws-ebs-csi-driver-node-4nl8m\" (UID: \"f2a856ae-d19f-4c9d-bb77-e04127e50283\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" Apr 16 08:39:14.209185 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207429 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Apr 16 08:39:14.209185 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207457 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-node-log\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.209185 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207465 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-run-systemd\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.209185 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207436 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-run-systemd\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.209185 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207631 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-lib-modules\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.209185 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207641 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-host-cni-netd\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.209185 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207669 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8e7b065b-9f25-4b3b-aa65-785e10aec00d-konnectivity-ca\") pod \"konnectivity-agent-6k8v5\" (UID: \"8e7b065b-9f25-4b3b-aa65-785e10aec00d\") " pod="kube-system/konnectivity-agent-6k8v5" Apr 16 08:39:14.209185 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207678 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-host-cni-netd\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.209185 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207696 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-host-run-multus-certs\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.209185 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207721 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zf9d6\" (UniqueName: \"kubernetes.io/projected/69ff05e4-5126-420e-ba65-9f88a7fec110-kube-api-access-zf9d6\") pod \"node-resolver-qxxqp\" (UID: \"69ff05e4-5126-420e-ba65-9f88a7fec110\") " pod="openshift-dns/node-resolver-qxxqp" Apr 16 08:39:14.209185 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207744 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-env-overrides\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.209894 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207788 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-ovn-node-metrics-cert\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.209894 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207813 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-cnibin\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.209894 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207824 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-multus-daemon-config\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.209894 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207839 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-multus-conf-dir\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.209894 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207864 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r47mj\" (UniqueName: \"kubernetes.io/projected/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-kube-api-access-r47mj\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.209894 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207886 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f2a856ae-d19f-4c9d-bb77-e04127e50283-registration-dir\") pod \"aws-ebs-csi-driver-node-4nl8m\" (UID: \"f2a856ae-d19f-4c9d-bb77-e04127e50283\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" Apr 16 08:39:14.209894 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207872 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f7b551d1-cdb9-462e-bf68-8b67b6304fff-iptables-alerter-script\") pod \"iptables-alerter-lvthh\" (UID: \"f7b551d1-cdb9-462e-bf68-8b67b6304fff\") " pod="openshift-network-operator/iptables-alerter-lvthh" Apr 16 08:39:14.209894 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207913 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d38f82dc-5ceb-4b6a-b918-e2b0085a9949-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7kxqz\" (UID: \"d38f82dc-5ceb-4b6a-b918-e2b0085a9949\") " pod="openshift-multus/multus-additional-cni-plugins-7kxqz" Apr 16 08:39:14.209894 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207919 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/17b0a66e-5c6b-4926-b67d-e916dd6009d3-serviceca\") pod \"node-ca-7rgcb\" (UID: \"17b0a66e-5c6b-4926-b67d-e916dd6009d3\") " pod="openshift-image-registry/node-ca-7rgcb" Apr 16 08:39:14.209894 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207936 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-host\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.209894 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207960 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-run-ovn\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.209894 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207983 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-multus-conf-dir\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.209894 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207985 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d38f82dc-5ceb-4b6a-b918-e2b0085a9949-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7kxqz\" (UID: \"d38f82dc-5ceb-4b6a-b918-e2b0085a9949\") " pod="openshift-multus/multus-additional-cni-plugins-7kxqz" Apr 16 08:39:14.209894 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208027 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-cni-binary-copy\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.209894 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208051 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-run\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.209894 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208075 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-var-lib-kubelet\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.209894 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208100 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69ff05e4-5126-420e-ba65-9f88a7fec110-tmp-dir\") pod \"node-resolver-qxxqp\" (UID: \"69ff05e4-5126-420e-ba65-9f88a7fec110\") " pod="openshift-dns/node-resolver-qxxqp" Apr 16 08:39:14.209894 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208125 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cpcq9\" (UniqueName: \"kubernetes.io/projected/57f278d8-41a1-4600-a089-04d841c7d446-kube-api-access-cpcq9\") pod \"network-metrics-daemon-fxtnv\" (UID: \"57f278d8-41a1-4600-a089-04d841c7d446\") " pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:14.210520 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208150 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-host-kubelet\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.210520 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208174 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-host-slash\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.210520 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208175 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8e7b065b-9f25-4b3b-aa65-785e10aec00d-konnectivity-ca\") pod \"konnectivity-agent-6k8v5\" (UID: \"8e7b065b-9f25-4b3b-aa65-785e10aec00d\") " pod="kube-system/konnectivity-agent-6k8v5" Apr 16 08:39:14.210520 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208179 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-env-overrides\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.210520 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208198 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-run\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.210520 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208200 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.210520 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208246 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-cnibin\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.210520 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208263 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-var-lib-kubelet\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.210520 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208397 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d38f82dc-5ceb-4b6a-b918-e2b0085a9949-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7kxqz\" (UID: \"d38f82dc-5ceb-4b6a-b918-e2b0085a9949\") " pod="openshift-multus/multus-additional-cni-plugins-7kxqz" Apr 16 08:39:14.210520 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208448 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-host\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.210520 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208494 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-run-ovn\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.210520 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.207962 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-host-run-multus-certs\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.210520 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208509 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69ff05e4-5126-420e-ba65-9f88a7fec110-tmp-dir\") pod \"node-resolver-qxxqp\" (UID: \"69ff05e4-5126-420e-ba65-9f88a7fec110\") " pod="openshift-dns/node-resolver-qxxqp" Apr 16 08:39:14.210520 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208530 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d38f82dc-5ceb-4b6a-b918-e2b0085a9949-cni-binary-copy\") pod \"multus-additional-cni-plugins-7kxqz\" (UID: \"d38f82dc-5ceb-4b6a-b918-e2b0085a9949\") " pod="openshift-multus/multus-additional-cni-plugins-7kxqz" Apr 16 08:39:14.210520 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208551 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f2a856ae-d19f-4c9d-bb77-e04127e50283-registration-dir\") pod \"aws-ebs-csi-driver-node-4nl8m\" (UID: \"f2a856ae-d19f-4c9d-bb77-e04127e50283\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" Apr 16 08:39:14.210520 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208566 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.210520 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208581 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-host-kubelet\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.210520 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208593 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-host-slash\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.211233 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208627 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-ovnkube-config\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.211233 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208683 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-system-cni-dir\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.211233 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208707 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-cni-binary-copy\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.211233 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208712 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-host-var-lib-kubelet\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.211233 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208757 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-system-cni-dir\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.211233 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208788 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d38f82dc-5ceb-4b6a-b918-e2b0085a9949-cnibin\") pod \"multus-additional-cni-plugins-7kxqz\" (UID: \"d38f82dc-5ceb-4b6a-b918-e2b0085a9949\") " pod="openshift-multus/multus-additional-cni-plugins-7kxqz" Apr 16 08:39:14.211233 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208814 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-host-var-lib-kubelet\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.211233 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208869 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-os-release\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.211233 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208902 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-hostroot\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.211233 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208916 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-os-release\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.211233 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208910 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d38f82dc-5ceb-4b6a-b918-e2b0085a9949-cnibin\") pod \"multus-additional-cni-plugins-7kxqz\" (UID: \"d38f82dc-5ceb-4b6a-b918-e2b0085a9949\") " pod="openshift-multus/multus-additional-cni-plugins-7kxqz" Apr 16 08:39:14.211233 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208955 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-hostroot\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.211233 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.208956 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-etc-sysctl-d\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.211233 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.209032 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1a9eef0f-407a-4515-a994-782aae576f13-etc-tuned\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.211233 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.209064 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1a9eef0f-407a-4515-a994-782aae576f13-etc-sysctl-d\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.211233 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.209067 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f2a856ae-d19f-4c9d-bb77-e04127e50283-etc-selinux\") pod \"aws-ebs-csi-driver-node-4nl8m\" (UID: \"f2a856ae-d19f-4c9d-bb77-e04127e50283\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" Apr 16 08:39:14.211233 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.209125 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-host-cni-bin\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.211233 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.209141 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f2a856ae-d19f-4c9d-bb77-e04127e50283-etc-selinux\") pod \"aws-ebs-csi-driver-node-4nl8m\" (UID: \"f2a856ae-d19f-4c9d-bb77-e04127e50283\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" Apr 16 08:39:14.211855 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.209158 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gk5xf\" (UniqueName: \"kubernetes.io/projected/d38f82dc-5ceb-4b6a-b918-e2b0085a9949-kube-api-access-gk5xf\") pod \"multus-additional-cni-plugins-7kxqz\" (UID: \"d38f82dc-5ceb-4b6a-b918-e2b0085a9949\") " pod="openshift-multus/multus-additional-cni-plugins-7kxqz" Apr 16 08:39:14.211855 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.209189 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-ovnkube-config\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.211855 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.209219 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-host-cni-bin\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.211855 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.209334 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d38f82dc-5ceb-4b6a-b918-e2b0085a9949-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7kxqz\" (UID: \"d38f82dc-5ceb-4b6a-b918-e2b0085a9949\") " pod="openshift-multus/multus-additional-cni-plugins-7kxqz" Apr 16 08:39:14.211855 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.211198 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a9eef0f-407a-4515-a994-782aae576f13-tmp\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.211855 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.211402 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8e7b065b-9f25-4b3b-aa65-785e10aec00d-agent-certs\") pod \"konnectivity-agent-6k8v5\" (UID: \"8e7b065b-9f25-4b3b-aa65-785e10aec00d\") " pod="kube-system/konnectivity-agent-6k8v5" Apr 16 08:39:14.212787 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.212737 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-ovn-node-metrics-cert\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.214308 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.214286 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1a9eef0f-407a-4515-a994-782aae576f13-etc-tuned\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.216478 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.216458 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw5kf\" (UniqueName: \"kubernetes.io/projected/0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791-kube-api-access-kw5kf\") pod \"ovnkube-node-6jwz6\" (UID: \"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791\") " pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.216660 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:14.216644 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 16 08:39:14.216738 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:14.216664 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 16 08:39:14.216738 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:14.216685 2578 projected.go:194] Error preparing data for projected volume kube-api-access-ddmc5 for pod openshift-network-diagnostics/network-check-target-5l2xc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:39:14.217056 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:14.216930 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e285ae7e-c21c-46b9-b705-ce4365740bd9-kube-api-access-ddmc5 podName:e285ae7e-c21c-46b9-b705-ce4365740bd9 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:14.716907924 +0000 UTC m=+3.085891978 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ddmc5" (UniqueName: "kubernetes.io/projected/e285ae7e-c21c-46b9-b705-ce4365740bd9-kube-api-access-ddmc5") pod "network-check-target-5l2xc" (UID: "e285ae7e-c21c-46b9-b705-ce4365740bd9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:39:14.218245 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.218225 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpcq9\" (UniqueName: \"kubernetes.io/projected/57f278d8-41a1-4600-a089-04d841c7d446-kube-api-access-cpcq9\") pod \"network-metrics-daemon-fxtnv\" (UID: \"57f278d8-41a1-4600-a089-04d841c7d446\") " pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:14.219087 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.219061 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z867x\" (UniqueName: \"kubernetes.io/projected/f7b551d1-cdb9-462e-bf68-8b67b6304fff-kube-api-access-z867x\") pod \"iptables-alerter-lvthh\" (UID: \"f7b551d1-cdb9-462e-bf68-8b67b6304fff\") " pod="openshift-network-operator/iptables-alerter-lvthh" Apr 16 08:39:14.219800 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.219740 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxk69\" (UniqueName: \"kubernetes.io/projected/17b0a66e-5c6b-4926-b67d-e916dd6009d3-kube-api-access-vxk69\") pod \"node-ca-7rgcb\" (UID: \"17b0a66e-5c6b-4926-b67d-e916dd6009d3\") " pod="openshift-image-registry/node-ca-7rgcb" Apr 16 08:39:14.220081 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.220052 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mzzf\" (UniqueName: \"kubernetes.io/projected/f2a856ae-d19f-4c9d-bb77-e04127e50283-kube-api-access-2mzzf\") pod \"aws-ebs-csi-driver-node-4nl8m\" (UID: \"f2a856ae-d19f-4c9d-bb77-e04127e50283\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" Apr 16 08:39:14.220167 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.220140 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf9d6\" (UniqueName: \"kubernetes.io/projected/69ff05e4-5126-420e-ba65-9f88a7fec110-kube-api-access-zf9d6\") pod \"node-resolver-qxxqp\" (UID: \"69ff05e4-5126-420e-ba65-9f88a7fec110\") " pod="openshift-dns/node-resolver-qxxqp" Apr 16 08:39:14.221423 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.221406 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk5xf\" (UniqueName: \"kubernetes.io/projected/d38f82dc-5ceb-4b6a-b918-e2b0085a9949-kube-api-access-gk5xf\") pod \"multus-additional-cni-plugins-7kxqz\" (UID: \"d38f82dc-5ceb-4b6a-b918-e2b0085a9949\") " pod="openshift-multus/multus-additional-cni-plugins-7kxqz" Apr 16 08:39:14.222065 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.222047 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5q6q\" (UniqueName: \"kubernetes.io/projected/1a9eef0f-407a-4515-a994-782aae576f13-kube-api-access-s5q6q\") pod \"tuned-mnwk4\" (UID: \"1a9eef0f-407a-4515-a994-782aae576f13\") " pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.222065 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.222056 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r47mj\" (UniqueName: \"kubernetes.io/projected/f2f2b361-d096-4c34-8cc1-5cfe69cdfe03-kube-api-access-r47mj\") pod \"multus-f9c2h\" (UID: \"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03\") " pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.391176 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.391113 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lvthh" Apr 16 08:39:14.393409 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.393389 2578 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Apr 16 08:39:14.398632 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.398614 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7kxqz" Apr 16 08:39:14.407190 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.407175 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f9c2h" Apr 16 08:39:14.410876 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.410857 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" Apr 16 08:39:14.418416 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.418397 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" Apr 16 08:39:14.424967 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.424947 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qxxqp" Apr 16 08:39:14.432418 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.432399 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7rgcb" Apr 16 08:39:14.439042 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.439026 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:14.444597 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.444581 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6k8v5" Apr 16 08:39:14.566378 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.566351 2578 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Apr 16 08:39:14.701090 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:14.701030 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a9eef0f_407a_4515_a994_782aae576f13.slice/crio-f224fa8e19555ecce695a49ace20a98272d3132b79e886d244b1639e5c742341 WatchSource:0}: Error finding container f224fa8e19555ecce695a49ace20a98272d3132b79e886d244b1639e5c742341: Status 404 returned error can't find the container with id f224fa8e19555ecce695a49ace20a98272d3132b79e886d244b1639e5c742341 Apr 16 08:39:14.702429 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:14.702356 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd38f82dc_5ceb_4b6a_b918_e2b0085a9949.slice/crio-9890a500076f6daa330ec541fbeb70064b86a94116f03ee51a1599136a61bc70 WatchSource:0}: Error finding container 9890a500076f6daa330ec541fbeb70064b86a94116f03ee51a1599136a61bc70: Status 404 returned error can't find the container with id 9890a500076f6daa330ec541fbeb70064b86a94116f03ee51a1599136a61bc70 Apr 16 08:39:14.706100 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:14.706061 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e7b065b_9f25_4b3b_aa65_785e10aec00d.slice/crio-2021cecabe4a1640a939d650b11a3609a1424c8a72ae1e52ffe483770e2fe654 WatchSource:0}: Error finding container 2021cecabe4a1640a939d650b11a3609a1424c8a72ae1e52ffe483770e2fe654: Status 404 returned error can't find the container with id 2021cecabe4a1640a939d650b11a3609a1424c8a72ae1e52ffe483770e2fe654 Apr 16 08:39:14.706748 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:14.706724 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7b551d1_cdb9_462e_bf68_8b67b6304fff.slice/crio-6ac7e5a3d08bba0e6a2d1d952cbcc10965dd7d794ce7498593633a899238d29a WatchSource:0}: Error finding container 6ac7e5a3d08bba0e6a2d1d952cbcc10965dd7d794ce7498593633a899238d29a: Status 404 returned error can't find the container with id 6ac7e5a3d08bba0e6a2d1d952cbcc10965dd7d794ce7498593633a899238d29a Apr 16 08:39:14.707892 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:14.707616 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17b0a66e_5c6b_4926_b67d_e916dd6009d3.slice/crio-00c04376cc064cc6518b1aa3772249d84af1dc19be7fd531526b211f4a5155f8 WatchSource:0}: Error finding container 00c04376cc064cc6518b1aa3772249d84af1dc19be7fd531526b211f4a5155f8: Status 404 returned error can't find the container with id 00c04376cc064cc6518b1aa3772249d84af1dc19be7fd531526b211f4a5155f8 Apr 16 08:39:14.710086 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:39:14.709438 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2a856ae_d19f_4c9d_bb77_e04127e50283.slice/crio-12cd97d0bfe2ae1287cdb632ba92b2acb68988fb8fb2e32c63dbaf51e586f1ee WatchSource:0}: Error finding container 12cd97d0bfe2ae1287cdb632ba92b2acb68988fb8fb2e32c63dbaf51e586f1ee: Status 404 returned error can't find the container with id 12cd97d0bfe2ae1287cdb632ba92b2acb68988fb8fb2e32c63dbaf51e586f1ee Apr 16 08:39:14.712796 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.712730 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57f278d8-41a1-4600-a089-04d841c7d446-metrics-certs\") pod \"network-metrics-daemon-fxtnv\" (UID: \"57f278d8-41a1-4600-a089-04d841c7d446\") " pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:14.712899 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:14.712884 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:39:14.712962 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:14.712949 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57f278d8-41a1-4600-a089-04d841c7d446-metrics-certs podName:57f278d8-41a1-4600-a089-04d841c7d446 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:15.712930581 +0000 UTC m=+4.081914636 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57f278d8-41a1-4600-a089-04d841c7d446-metrics-certs") pod "network-metrics-daemon-fxtnv" (UID: "57f278d8-41a1-4600-a089-04d841c7d446") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:39:14.813269 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:14.813243 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ddmc5\" (UniqueName: \"kubernetes.io/projected/e285ae7e-c21c-46b9-b705-ce4365740bd9-kube-api-access-ddmc5\") pod \"network-check-target-5l2xc\" (UID: \"e285ae7e-c21c-46b9-b705-ce4365740bd9\") " pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:39:14.813371 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:14.813360 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 16 08:39:14.813411 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:14.813375 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 16 08:39:14.813411 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:14.813384 2578 projected.go:194] Error preparing data for projected volume kube-api-access-ddmc5 for pod openshift-network-diagnostics/network-check-target-5l2xc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:39:14.813467 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:14.813424 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e285ae7e-c21c-46b9-b705-ce4365740bd9-kube-api-access-ddmc5 podName:e285ae7e-c21c-46b9-b705-ce4365740bd9 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:15.813412822 +0000 UTC m=+4.182396861 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ddmc5" (UniqueName: "kubernetes.io/projected/e285ae7e-c21c-46b9-b705-ce4365740bd9-kube-api-access-ddmc5") pod "network-check-target-5l2xc" (UID: "e285ae7e-c21c-46b9-b705-ce4365740bd9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:39:15.127601 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:15.127325 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-04-15 08:34:13 +0000 UTC" deadline="2028-01-17 10:41:57.19909102 +0000 UTC" Apr 16 08:39:15.127601 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:15.127560 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15386h2m42.071536679s" Apr 16 08:39:15.222750 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:15.222698 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qxxqp" event={"ID":"69ff05e4-5126-420e-ba65-9f88a7fec110","Type":"ContainerStarted","Data":"4bfd2d229579b2015d39e64a1c685c2429fd5aa2913f590d744c4bb547d373d4"} Apr 16 08:39:15.230728 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:15.230698 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lvthh" event={"ID":"f7b551d1-cdb9-462e-bf68-8b67b6304fff","Type":"ContainerStarted","Data":"6ac7e5a3d08bba0e6a2d1d952cbcc10965dd7d794ce7498593633a899238d29a"} Apr 16 08:39:15.232923 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:15.232898 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" event={"ID":"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791","Type":"ContainerStarted","Data":"51cb6833019c175cfc2a7991955f2da36fdb36a6020042038f58ad4fd572bf8b"} Apr 16 08:39:15.234021 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:15.233999 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7rgcb" event={"ID":"17b0a66e-5c6b-4926-b67d-e916dd6009d3","Type":"ContainerStarted","Data":"00c04376cc064cc6518b1aa3772249d84af1dc19be7fd531526b211f4a5155f8"} Apr 16 08:39:15.241709 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:15.241682 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6k8v5" event={"ID":"8e7b065b-9f25-4b3b-aa65-785e10aec00d","Type":"ContainerStarted","Data":"2021cecabe4a1640a939d650b11a3609a1424c8a72ae1e52ffe483770e2fe654"} Apr 16 08:39:15.247017 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:15.246993 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-84.ec2.internal" event={"ID":"267abad74bc5a64078f92c3f8060b88d","Type":"ContainerStarted","Data":"680a7c90003d5b4acb77c42bb1c79d391fbe8943409557d0bf87cef276727094"} Apr 16 08:39:15.255957 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:15.255911 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f9c2h" event={"ID":"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03","Type":"ContainerStarted","Data":"fa428f8082f9d70512472118c12d6063e83e568585b684549c9d700e0d1a5b56"} Apr 16 08:39:15.264735 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:15.264706 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" event={"ID":"f2a856ae-d19f-4c9d-bb77-e04127e50283","Type":"ContainerStarted","Data":"12cd97d0bfe2ae1287cdb632ba92b2acb68988fb8fb2e32c63dbaf51e586f1ee"} Apr 16 08:39:15.266276 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:15.266250 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7kxqz" event={"ID":"d38f82dc-5ceb-4b6a-b918-e2b0085a9949","Type":"ContainerStarted","Data":"9890a500076f6daa330ec541fbeb70064b86a94116f03ee51a1599136a61bc70"} Apr 16 08:39:15.271452 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:15.271428 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" event={"ID":"1a9eef0f-407a-4515-a994-782aae576f13","Type":"ContainerStarted","Data":"f224fa8e19555ecce695a49ace20a98272d3132b79e886d244b1639e5c742341"} Apr 16 08:39:15.603042 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:15.603014 2578 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Apr 16 08:39:15.725103 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:15.724485 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57f278d8-41a1-4600-a089-04d841c7d446-metrics-certs\") pod \"network-metrics-daemon-fxtnv\" (UID: \"57f278d8-41a1-4600-a089-04d841c7d446\") " pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:15.725103 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:15.724652 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:39:15.725103 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:15.724742 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57f278d8-41a1-4600-a089-04d841c7d446-metrics-certs podName:57f278d8-41a1-4600-a089-04d841c7d446 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:17.724696886 +0000 UTC m=+6.093680937 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57f278d8-41a1-4600-a089-04d841c7d446-metrics-certs") pod "network-metrics-daemon-fxtnv" (UID: "57f278d8-41a1-4600-a089-04d841c7d446") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:39:15.825570 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:15.825539 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ddmc5\" (UniqueName: \"kubernetes.io/projected/e285ae7e-c21c-46b9-b705-ce4365740bd9-kube-api-access-ddmc5\") pod \"network-check-target-5l2xc\" (UID: \"e285ae7e-c21c-46b9-b705-ce4365740bd9\") " pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:39:15.825722 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:15.825696 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 16 08:39:15.825722 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:15.825716 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 16 08:39:15.826103 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:15.825978 2578 projected.go:194] Error preparing data for projected volume kube-api-access-ddmc5 for pod openshift-network-diagnostics/network-check-target-5l2xc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:39:15.826103 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:15.826068 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e285ae7e-c21c-46b9-b705-ce4365740bd9-kube-api-access-ddmc5 podName:e285ae7e-c21c-46b9-b705-ce4365740bd9 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:17.826035857 +0000 UTC m=+6.195019892 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ddmc5" (UniqueName: "kubernetes.io/projected/e285ae7e-c21c-46b9-b705-ce4365740bd9-kube-api-access-ddmc5") pod "network-check-target-5l2xc" (UID: "e285ae7e-c21c-46b9-b705-ce4365740bd9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:39:16.212234 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:16.212159 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:16.212660 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:16.212298 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxtnv" podUID="57f278d8-41a1-4600-a089-04d841c7d446" Apr 16 08:39:16.212870 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:16.212850 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:39:16.212957 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:16.212943 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5l2xc" podUID="e285ae7e-c21c-46b9-b705-ce4365740bd9" Apr 16 08:39:16.290155 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:16.290120 2578 generic.go:358] "Generic (PLEG): container finished" podID="5a1f89a5e7a6831921722fd68212b6b3" containerID="171adbbd965430467e8817b926db6900772d51159e0d7f4d15e7298d68d91e4f" exitCode=0 Apr 16 08:39:16.290309 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:16.290220 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-84.ec2.internal" event={"ID":"5a1f89a5e7a6831921722fd68212b6b3","Type":"ContainerDied","Data":"171adbbd965430467e8817b926db6900772d51159e0d7f4d15e7298d68d91e4f"} Apr 16 08:39:16.303725 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:16.302985 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-84.ec2.internal" podStartSLOduration=3.3029692920000002 podStartE2EDuration="3.302969292s" podCreationTimestamp="2026-04-16 08:39:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 08:39:15.258297449 +0000 UTC m=+3.627281507" watchObservedRunningTime="2026-04-16 08:39:16.302969292 +0000 UTC m=+4.671953349" Apr 16 08:39:17.294842 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:17.294673 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-84.ec2.internal" event={"ID":"5a1f89a5e7a6831921722fd68212b6b3","Type":"ContainerStarted","Data":"989b56e69c5104be3e5f1790d5249a9e4f68b6d093c68fc51e0ca8dfb27442f3"} Apr 16 08:39:17.307064 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:17.306591 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-84.ec2.internal" podStartSLOduration=4.306574263 podStartE2EDuration="4.306574263s" podCreationTimestamp="2026-04-16 08:39:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 08:39:17.305942682 +0000 UTC m=+5.674926739" watchObservedRunningTime="2026-04-16 08:39:17.306574263 +0000 UTC m=+5.675558320" Apr 16 08:39:17.743908 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:17.743875 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57f278d8-41a1-4600-a089-04d841c7d446-metrics-certs\") pod \"network-metrics-daemon-fxtnv\" (UID: \"57f278d8-41a1-4600-a089-04d841c7d446\") " pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:17.744075 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:17.744017 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:39:17.744075 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:17.744065 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57f278d8-41a1-4600-a089-04d841c7d446-metrics-certs podName:57f278d8-41a1-4600-a089-04d841c7d446 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:21.744051697 +0000 UTC m=+10.113035736 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57f278d8-41a1-4600-a089-04d841c7d446-metrics-certs") pod "network-metrics-daemon-fxtnv" (UID: "57f278d8-41a1-4600-a089-04d841c7d446") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:39:17.845005 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:17.844966 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ddmc5\" (UniqueName: \"kubernetes.io/projected/e285ae7e-c21c-46b9-b705-ce4365740bd9-kube-api-access-ddmc5\") pod \"network-check-target-5l2xc\" (UID: \"e285ae7e-c21c-46b9-b705-ce4365740bd9\") " pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:39:17.845255 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:17.845234 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 16 08:39:17.845321 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:17.845262 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 16 08:39:17.845321 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:17.845276 2578 projected.go:194] Error preparing data for projected volume kube-api-access-ddmc5 for pod openshift-network-diagnostics/network-check-target-5l2xc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:39:17.845413 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:17.845367 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e285ae7e-c21c-46b9-b705-ce4365740bd9-kube-api-access-ddmc5 podName:e285ae7e-c21c-46b9-b705-ce4365740bd9 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:21.845346591 +0000 UTC m=+10.214330638 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ddmc5" (UniqueName: "kubernetes.io/projected/e285ae7e-c21c-46b9-b705-ce4365740bd9-kube-api-access-ddmc5") pod "network-check-target-5l2xc" (UID: "e285ae7e-c21c-46b9-b705-ce4365740bd9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:39:18.213166 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:18.213084 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:39:18.213318 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:18.213206 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5l2xc" podUID="e285ae7e-c21c-46b9-b705-ce4365740bd9" Apr 16 08:39:18.213868 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:18.213585 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:18.213868 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:18.213688 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxtnv" podUID="57f278d8-41a1-4600-a089-04d841c7d446" Apr 16 08:39:20.209626 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:20.209594 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:39:20.210054 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:20.209714 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5l2xc" podUID="e285ae7e-c21c-46b9-b705-ce4365740bd9" Apr 16 08:39:20.210160 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:20.210142 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:20.210246 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:20.210231 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxtnv" podUID="57f278d8-41a1-4600-a089-04d841c7d446" Apr 16 08:39:21.780139 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:21.780102 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57f278d8-41a1-4600-a089-04d841c7d446-metrics-certs\") pod \"network-metrics-daemon-fxtnv\" (UID: \"57f278d8-41a1-4600-a089-04d841c7d446\") " pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:21.780504 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:21.780287 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:39:21.780504 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:21.780378 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57f278d8-41a1-4600-a089-04d841c7d446-metrics-certs podName:57f278d8-41a1-4600-a089-04d841c7d446 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:29.780358634 +0000 UTC m=+18.149342684 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57f278d8-41a1-4600-a089-04d841c7d446-metrics-certs") pod "network-metrics-daemon-fxtnv" (UID: "57f278d8-41a1-4600-a089-04d841c7d446") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:39:21.880692 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:21.880654 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ddmc5\" (UniqueName: \"kubernetes.io/projected/e285ae7e-c21c-46b9-b705-ce4365740bd9-kube-api-access-ddmc5\") pod \"network-check-target-5l2xc\" (UID: \"e285ae7e-c21c-46b9-b705-ce4365740bd9\") " pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:39:21.880871 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:21.880831 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 16 08:39:21.880871 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:21.880848 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 16 08:39:21.880871 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:21.880859 2578 projected.go:194] Error preparing data for projected volume kube-api-access-ddmc5 for pod openshift-network-diagnostics/network-check-target-5l2xc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:39:21.881008 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:21.880907 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e285ae7e-c21c-46b9-b705-ce4365740bd9-kube-api-access-ddmc5 podName:e285ae7e-c21c-46b9-b705-ce4365740bd9 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:29.880889315 +0000 UTC m=+18.249873354 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ddmc5" (UniqueName: "kubernetes.io/projected/e285ae7e-c21c-46b9-b705-ce4365740bd9-kube-api-access-ddmc5") pod "network-check-target-5l2xc" (UID: "e285ae7e-c21c-46b9-b705-ce4365740bd9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:39:22.210354 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:22.210277 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:22.210503 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:22.210398 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxtnv" podUID="57f278d8-41a1-4600-a089-04d841c7d446" Apr 16 08:39:22.210503 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:22.210453 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:39:22.210614 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:22.210566 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5l2xc" podUID="e285ae7e-c21c-46b9-b705-ce4365740bd9" Apr 16 08:39:24.212263 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:24.212227 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:39:24.212263 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:24.212245 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:24.212873 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:24.212330 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5l2xc" podUID="e285ae7e-c21c-46b9-b705-ce4365740bd9" Apr 16 08:39:24.212873 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:24.212408 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxtnv" podUID="57f278d8-41a1-4600-a089-04d841c7d446" Apr 16 08:39:26.212196 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:26.212170 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:39:26.212611 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:26.212170 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:26.212611 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:26.212282 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5l2xc" podUID="e285ae7e-c21c-46b9-b705-ce4365740bd9" Apr 16 08:39:26.212611 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:26.212390 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxtnv" podUID="57f278d8-41a1-4600-a089-04d841c7d446" Apr 16 08:39:28.209962 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:28.209887 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:28.210417 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:28.209899 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:39:28.210417 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:28.210043 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxtnv" podUID="57f278d8-41a1-4600-a089-04d841c7d446" Apr 16 08:39:28.210417 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:28.210115 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5l2xc" podUID="e285ae7e-c21c-46b9-b705-ce4365740bd9" Apr 16 08:39:29.841348 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:29.841308 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57f278d8-41a1-4600-a089-04d841c7d446-metrics-certs\") pod \"network-metrics-daemon-fxtnv\" (UID: \"57f278d8-41a1-4600-a089-04d841c7d446\") " pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:29.841906 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:29.841472 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:39:29.841906 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:29.841547 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57f278d8-41a1-4600-a089-04d841c7d446-metrics-certs podName:57f278d8-41a1-4600-a089-04d841c7d446 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:45.84152685 +0000 UTC m=+34.210510903 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57f278d8-41a1-4600-a089-04d841c7d446-metrics-certs") pod "network-metrics-daemon-fxtnv" (UID: "57f278d8-41a1-4600-a089-04d841c7d446") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:39:29.942697 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:29.942653 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ddmc5\" (UniqueName: \"kubernetes.io/projected/e285ae7e-c21c-46b9-b705-ce4365740bd9-kube-api-access-ddmc5\") pod \"network-check-target-5l2xc\" (UID: \"e285ae7e-c21c-46b9-b705-ce4365740bd9\") " pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:39:29.942890 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:29.942837 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 16 08:39:29.942890 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:29.942862 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 16 08:39:29.942890 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:29.942874 2578 projected.go:194] Error preparing data for projected volume kube-api-access-ddmc5 for pod openshift-network-diagnostics/network-check-target-5l2xc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:39:29.943035 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:29.942932 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e285ae7e-c21c-46b9-b705-ce4365740bd9-kube-api-access-ddmc5 podName:e285ae7e-c21c-46b9-b705-ce4365740bd9 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:45.94291493 +0000 UTC m=+34.311898987 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ddmc5" (UniqueName: "kubernetes.io/projected/e285ae7e-c21c-46b9-b705-ce4365740bd9-kube-api-access-ddmc5") pod "network-check-target-5l2xc" (UID: "e285ae7e-c21c-46b9-b705-ce4365740bd9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:39:30.210176 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:30.210092 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:30.210176 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:30.210123 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:39:30.210396 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:30.210215 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxtnv" podUID="57f278d8-41a1-4600-a089-04d841c7d446" Apr 16 08:39:30.210396 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:30.210348 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5l2xc" podUID="e285ae7e-c21c-46b9-b705-ce4365740bd9" Apr 16 08:39:32.210617 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:32.210221 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:32.211219 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:32.210309 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:39:32.211219 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:32.210706 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxtnv" podUID="57f278d8-41a1-4600-a089-04d841c7d446" Apr 16 08:39:32.211219 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:32.210834 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5l2xc" podUID="e285ae7e-c21c-46b9-b705-ce4365740bd9" Apr 16 08:39:32.322041 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:32.322002 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f9c2h" event={"ID":"f2f2b361-d096-4c34-8cc1-5cfe69cdfe03","Type":"ContainerStarted","Data":"d096562bbed9910e9f0521518ad4d401141d12347a86e7b2a004a35d187d1304"} Apr 16 08:39:32.323760 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:32.323725 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" event={"ID":"f2a856ae-d19f-4c9d-bb77-e04127e50283","Type":"ContainerStarted","Data":"4542156c18191496a92e2e67b43454ce550a39e3aa4c7ebade7dcb565478daaa"} Apr 16 08:39:32.325127 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:32.325100 2578 generic.go:358] "Generic (PLEG): container finished" podID="d38f82dc-5ceb-4b6a-b918-e2b0085a9949" containerID="d0620559268e778eb1d226d9dc426147a9e7d9a610aacf10974c2e3b9dcc1ce8" exitCode=0 Apr 16 08:39:32.325224 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:32.325176 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7kxqz" event={"ID":"d38f82dc-5ceb-4b6a-b918-e2b0085a9949","Type":"ContainerDied","Data":"d0620559268e778eb1d226d9dc426147a9e7d9a610aacf10974c2e3b9dcc1ce8"} Apr 16 08:39:32.326733 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:32.326498 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" event={"ID":"1a9eef0f-407a-4515-a994-782aae576f13","Type":"ContainerStarted","Data":"7f415b04932fdaf449695158d80fca7042e019343aa8fb5ac9c031f1a2cd3983"} Apr 16 08:39:32.328110 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:32.328088 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qxxqp" event={"ID":"69ff05e4-5126-420e-ba65-9f88a7fec110","Type":"ContainerStarted","Data":"f437fe54f95ffa14591deb4008087853b0276cf2e214093934a3aa593c97b20c"} Apr 16 08:39:32.332447 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:32.332423 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" event={"ID":"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791","Type":"ContainerStarted","Data":"27680b4ac5aff1c6114a23b96814c4fcd0cacdfc3dece8c05b64998530c3fe9f"} Apr 16 08:39:32.332535 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:32.332455 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" event={"ID":"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791","Type":"ContainerStarted","Data":"33e75fd658e0a26459372381ee0c12091e61172e69259047f3f4b5dfb95627e9"} Apr 16 08:39:32.332535 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:32.332469 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" event={"ID":"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791","Type":"ContainerStarted","Data":"28c998022afe5165d9c949707e6f83fe492f95dd6cba3dbbe8e5a67d8c43ec95"} Apr 16 08:39:32.332535 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:32.332481 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" event={"ID":"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791","Type":"ContainerStarted","Data":"924c0ef8b45333d2387ef26793656998e656ac9563fbd06bd4fcf3765b251d5a"} Apr 16 08:39:32.332535 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:32.332493 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" event={"ID":"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791","Type":"ContainerStarted","Data":"602c4c7d1fbaa7a5295bc1fa8869a13328f46cd7cf6cb5b7dd03eedb7a53e068"} Apr 16 08:39:32.332535 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:32.332507 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" event={"ID":"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791","Type":"ContainerStarted","Data":"fbbf639bbd8657782563c5b707dc17bff444ec70fa4aa98a10092475c4d9836c"} Apr 16 08:39:32.334300 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:32.334279 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7rgcb" event={"ID":"17b0a66e-5c6b-4926-b67d-e916dd6009d3","Type":"ContainerStarted","Data":"dc961c00b675b1a4937d72eb78e07e0f25fe02cce884f1e7a96673d84d6a18cc"} Apr 16 08:39:32.335528 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:32.335508 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6k8v5" event={"ID":"8e7b065b-9f25-4b3b-aa65-785e10aec00d","Type":"ContainerStarted","Data":"31e135030c423a2455ab1ac8d399fdad58e58807f32d671cd8f8a62ec6492968"} Apr 16 08:39:32.337188 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:32.337151 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-f9c2h" podStartSLOduration=3.699536045 podStartE2EDuration="20.337141876s" podCreationTimestamp="2026-04-16 08:39:12 +0000 UTC" firstStartedPulling="2026-04-16 08:39:14.712816566 +0000 UTC m=+3.081800615" lastFinishedPulling="2026-04-16 08:39:31.350422408 +0000 UTC m=+19.719406446" observedRunningTime="2026-04-16 08:39:32.33667481 +0000 UTC m=+20.705658868" watchObservedRunningTime="2026-04-16 08:39:32.337141876 +0000 UTC m=+20.706125931" Apr 16 08:39:32.350542 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:32.350512 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mnwk4" podStartSLOduration=3.739163028 podStartE2EDuration="20.350501938s" podCreationTimestamp="2026-04-16 08:39:12 +0000 UTC" firstStartedPulling="2026-04-16 08:39:14.703095003 +0000 UTC m=+3.072079039" lastFinishedPulling="2026-04-16 08:39:31.314433902 +0000 UTC m=+19.683417949" observedRunningTime="2026-04-16 08:39:32.350475521 +0000 UTC m=+20.719459577" watchObservedRunningTime="2026-04-16 08:39:32.350501938 +0000 UTC m=+20.719485995" Apr 16 08:39:32.361709 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:32.361675 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-qxxqp" podStartSLOduration=3.763056018 podStartE2EDuration="20.361665099s" podCreationTimestamp="2026-04-16 08:39:12 +0000 UTC" firstStartedPulling="2026-04-16 08:39:14.714097647 +0000 UTC m=+3.083081682" lastFinishedPulling="2026-04-16 08:39:31.31270671 +0000 UTC m=+19.681690763" observedRunningTime="2026-04-16 08:39:32.361508637 +0000 UTC m=+20.730492694" watchObservedRunningTime="2026-04-16 08:39:32.361665099 +0000 UTC m=+20.730649154" Apr 16 08:39:32.387430 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:32.387405 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Apr 16 08:39:32.393503 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:32.393459 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6k8v5" podStartSLOduration=8.151383896 podStartE2EDuration="20.393446186s" podCreationTimestamp="2026-04-16 08:39:12 +0000 UTC" firstStartedPulling="2026-04-16 08:39:14.707368486 +0000 UTC m=+3.076352521" lastFinishedPulling="2026-04-16 08:39:26.949430748 +0000 UTC m=+15.318414811" observedRunningTime="2026-04-16 08:39:32.374018019 +0000 UTC m=+20.743002075" watchObservedRunningTime="2026-04-16 08:39:32.393446186 +0000 UTC m=+20.762430246" Apr 16 08:39:32.406169 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:32.406129 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7rgcb" podStartSLOduration=3.804282304 podStartE2EDuration="20.406120012s" podCreationTimestamp="2026-04-16 08:39:12 +0000 UTC" firstStartedPulling="2026-04-16 08:39:14.711201204 +0000 UTC m=+3.080185253" lastFinishedPulling="2026-04-16 08:39:31.313038912 +0000 UTC m=+19.682022961" observedRunningTime="2026-04-16 08:39:32.406030994 +0000 UTC m=+20.775015061" watchObservedRunningTime="2026-04-16 08:39:32.406120012 +0000 UTC m=+20.775104068" Apr 16 08:39:32.843702 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:32.843666 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6k8v5" Apr 16 08:39:32.844397 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:32.844374 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6k8v5" Apr 16 08:39:33.163276 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:33.163066 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-04-16T08:39:32.387425764Z","UUID":"deea3afa-3239-4d0e-bef0-87ccf1822206","Handler":null,"Name":"","Endpoint":""} Apr 16 08:39:33.164900 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:33.164876 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Apr 16 08:39:33.165064 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:33.164911 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Apr 16 08:39:33.339487 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:33.339428 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" event={"ID":"f2a856ae-d19f-4c9d-bb77-e04127e50283","Type":"ContainerStarted","Data":"5394aacfb53faefcbfb6e98ab7598ff8964b300ebcc665881f059e64175fab5d"} Apr 16 08:39:33.339487 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:33.339468 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" event={"ID":"f2a856ae-d19f-4c9d-bb77-e04127e50283","Type":"ContainerStarted","Data":"0734d94b7b3408386db841c58228d3966a36bc260591a95279e9e0d99f00e1d1"} Apr 16 08:39:33.340862 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:33.340813 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lvthh" event={"ID":"f7b551d1-cdb9-462e-bf68-8b67b6304fff","Type":"ContainerStarted","Data":"2fa8cc1304092ce2a93f7b7f3d5db74eff64182bd2d162e2f54e102009457718"} Apr 16 08:39:33.341946 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:33.341470 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6k8v5" Apr 16 08:39:33.344888 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:33.342134 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6k8v5" Apr 16 08:39:33.354987 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:33.354943 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4nl8m" podStartSLOduration=2.8574339220000002 podStartE2EDuration="21.354932034s" podCreationTimestamp="2026-04-16 08:39:12 +0000 UTC" firstStartedPulling="2026-04-16 08:39:14.711662642 +0000 UTC m=+3.080646677" lastFinishedPulling="2026-04-16 08:39:33.209160739 +0000 UTC m=+21.578144789" observedRunningTime="2026-04-16 08:39:33.354433793 +0000 UTC m=+21.723417856" watchObservedRunningTime="2026-04-16 08:39:33.354932034 +0000 UTC m=+21.723916094" Apr 16 08:39:33.365659 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:33.365617 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lvthh" podStartSLOduration=4.762546735 podStartE2EDuration="21.365603707s" podCreationTimestamp="2026-04-16 08:39:12 +0000 UTC" firstStartedPulling="2026-04-16 08:39:14.709628902 +0000 UTC m=+3.078612938" lastFinishedPulling="2026-04-16 08:39:31.312685859 +0000 UTC m=+19.681669910" observedRunningTime="2026-04-16 08:39:33.365554626 +0000 UTC m=+21.734538685" watchObservedRunningTime="2026-04-16 08:39:33.365603707 +0000 UTC m=+21.734587766" Apr 16 08:39:34.210133 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:34.210099 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:34.210278 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:34.210251 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxtnv" podUID="57f278d8-41a1-4600-a089-04d841c7d446" Apr 16 08:39:34.210337 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:34.210306 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:39:34.210438 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:34.210417 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5l2xc" podUID="e285ae7e-c21c-46b9-b705-ce4365740bd9" Apr 16 08:39:34.346461 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:34.346369 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" event={"ID":"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791","Type":"ContainerStarted","Data":"8b092659942e8c0ca043a2d4ac36b421a9a73729b04223ffe1a06db85286b7f2"} Apr 16 08:39:36.210008 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:36.209837 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:36.210406 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:36.209902 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:39:36.210406 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:36.210132 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxtnv" podUID="57f278d8-41a1-4600-a089-04d841c7d446" Apr 16 08:39:36.210406 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:36.210197 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5l2xc" podUID="e285ae7e-c21c-46b9-b705-ce4365740bd9" Apr 16 08:39:36.354314 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:36.354284 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" event={"ID":"0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791","Type":"ContainerStarted","Data":"e9c0b0b4b9162679846fb466a3bfbe525fa11e2d0fb6b8d346c9b43dc86e1b7e"} Apr 16 08:39:36.354613 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:36.354593 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:36.355996 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:36.355973 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7kxqz" event={"ID":"d38f82dc-5ceb-4b6a-b918-e2b0085a9949","Type":"ContainerStarted","Data":"b289afab0b9eb8c71d16e0f1bd52c39217c55c34a0f365ce936b963d99cdf8d6"} Apr 16 08:39:36.370453 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:36.370393 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:36.385308 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:36.385260 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" podStartSLOduration=7.49742423 podStartE2EDuration="24.385247489s" podCreationTimestamp="2026-04-16 08:39:12 +0000 UTC" firstStartedPulling="2026-04-16 08:39:14.712926176 +0000 UTC m=+3.081910212" lastFinishedPulling="2026-04-16 08:39:31.600749434 +0000 UTC m=+19.969733471" observedRunningTime="2026-04-16 08:39:36.38446415 +0000 UTC m=+24.753448220" watchObservedRunningTime="2026-04-16 08:39:36.385247489 +0000 UTC m=+24.754231545" Apr 16 08:39:37.358799 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:37.358754 2578 generic.go:358] "Generic (PLEG): container finished" podID="d38f82dc-5ceb-4b6a-b918-e2b0085a9949" containerID="b289afab0b9eb8c71d16e0f1bd52c39217c55c34a0f365ce936b963d99cdf8d6" exitCode=0 Apr 16 08:39:37.359230 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:37.358840 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7kxqz" event={"ID":"d38f82dc-5ceb-4b6a-b918-e2b0085a9949","Type":"ContainerDied","Data":"b289afab0b9eb8c71d16e0f1bd52c39217c55c34a0f365ce936b963d99cdf8d6"} Apr 16 08:39:37.359230 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:37.358928 2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Apr 16 08:39:37.359329 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:37.359262 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:37.373449 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:37.373432 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:38.177727 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:38.177549 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fxtnv"] Apr 16 08:39:38.177951 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:38.177843 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:38.177951 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:38.177929 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxtnv" podUID="57f278d8-41a1-4600-a089-04d841c7d446" Apr 16 08:39:38.180368 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:38.180347 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5l2xc"] Apr 16 08:39:38.180484 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:38.180434 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:39:38.180528 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:38.180497 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5l2xc" podUID="e285ae7e-c21c-46b9-b705-ce4365740bd9" Apr 16 08:39:38.360402 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:38.360383 2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Apr 16 08:39:39.363492 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:39.363456 2578 generic.go:358] "Generic (PLEG): container finished" podID="d38f82dc-5ceb-4b6a-b918-e2b0085a9949" containerID="369ced9faf3793e06f4d1b64a42a5abe67e5922b7df44ad3518cb6802dee1c2a" exitCode=0 Apr 16 08:39:39.363973 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:39.363548 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7kxqz" event={"ID":"d38f82dc-5ceb-4b6a-b918-e2b0085a9949","Type":"ContainerDied","Data":"369ced9faf3793e06f4d1b64a42a5abe67e5922b7df44ad3518cb6802dee1c2a"} Apr 16 08:39:39.363973 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:39.363605 2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Apr 16 08:39:40.209776 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:40.209736 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:40.209776 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:40.209754 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:39:40.209948 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:40.209890 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxtnv" podUID="57f278d8-41a1-4600-a089-04d841c7d446" Apr 16 08:39:40.210034 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:40.210014 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5l2xc" podUID="e285ae7e-c21c-46b9-b705-ce4365740bd9" Apr 16 08:39:41.368392 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:41.368357 2578 generic.go:358] "Generic (PLEG): container finished" podID="d38f82dc-5ceb-4b6a-b918-e2b0085a9949" containerID="a056e4838b801172c02a0b6d6137e5a2a3a84f42e58ef2111d8fa878cf086d8e" exitCode=0 Apr 16 08:39:41.369079 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:41.368405 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7kxqz" event={"ID":"d38f82dc-5ceb-4b6a-b918-e2b0085a9949","Type":"ContainerDied","Data":"a056e4838b801172c02a0b6d6137e5a2a3a84f42e58ef2111d8fa878cf086d8e"} Apr 16 08:39:42.211366 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:42.211323 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:42.211553 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:42.211379 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:39:42.211553 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:42.211456 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxtnv" podUID="57f278d8-41a1-4600-a089-04d841c7d446" Apr 16 08:39:42.211669 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:42.211558 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5l2xc" podUID="e285ae7e-c21c-46b9-b705-ce4365740bd9" Apr 16 08:39:42.376826 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:42.376780 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:42.377239 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:42.377038 2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Apr 16 08:39:42.392905 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:42.392515 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6jwz6" Apr 16 08:39:44.210187 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.210156 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:39:44.210187 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.210180 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:44.210637 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:44.210275 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5l2xc" podUID="e285ae7e-c21c-46b9-b705-ce4365740bd9" Apr 16 08:39:44.210637 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:44.210421 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxtnv" podUID="57f278d8-41a1-4600-a089-04d841c7d446" Apr 16 08:39:44.466189 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.466161 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-84.ec2.internal" event="NodeReady" Apr 16 08:39:44.466358 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.466313 2578 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Apr 16 08:39:44.506916 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.506886 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rqjhz"] Apr 16 08:39:44.534189 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.534160 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nlp9v"] Apr 16 08:39:44.534316 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.534307 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rqjhz" Apr 16 08:39:44.536274 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.536255 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Apr 16 08:39:44.536383 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.536255 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9d2hv\"" Apr 16 08:39:44.536383 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.536254 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Apr 16 08:39:44.550112 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.549979 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rqjhz"] Apr 16 08:39:44.550188 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.550120 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nlp9v"] Apr 16 08:39:44.550188 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.550099 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nlp9v" Apr 16 08:39:44.553693 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.553674 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Apr 16 08:39:44.553810 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.553788 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Apr 16 08:39:44.553881 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.553869 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-q9sj7\"" Apr 16 08:39:44.554087 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.554071 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Apr 16 08:39:44.650570 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.650533 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-metrics-tls\") pod \"dns-default-rqjhz\" (UID: \"b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8\") " pod="openshift-dns/dns-default-rqjhz" Apr 16 08:39:44.650733 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.650575 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6136520a-182b-44c5-837e-1dd3dc6af3f7-cert\") pod \"ingress-canary-nlp9v\" (UID: \"6136520a-182b-44c5-837e-1dd3dc6af3f7\") " pod="openshift-ingress-canary/ingress-canary-nlp9v" Apr 16 08:39:44.650733 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.650608 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-config-volume\") pod \"dns-default-rqjhz\" (UID: \"b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8\") " pod="openshift-dns/dns-default-rqjhz" Apr 16 08:39:44.650733 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.650629 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdlxc\" (UniqueName: \"kubernetes.io/projected/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-kube-api-access-kdlxc\") pod \"dns-default-rqjhz\" (UID: \"b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8\") " pod="openshift-dns/dns-default-rqjhz" Apr 16 08:39:44.650733 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.650655 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s6xr\" (UniqueName: \"kubernetes.io/projected/6136520a-182b-44c5-837e-1dd3dc6af3f7-kube-api-access-5s6xr\") pod \"ingress-canary-nlp9v\" (UID: \"6136520a-182b-44c5-837e-1dd3dc6af3f7\") " pod="openshift-ingress-canary/ingress-canary-nlp9v" Apr 16 08:39:44.650733 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.650680 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-tmp-dir\") pod \"dns-default-rqjhz\" (UID: \"b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8\") " pod="openshift-dns/dns-default-rqjhz" Apr 16 08:39:44.751813 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.751715 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6136520a-182b-44c5-837e-1dd3dc6af3f7-cert\") pod \"ingress-canary-nlp9v\" (UID: \"6136520a-182b-44c5-837e-1dd3dc6af3f7\") " pod="openshift-ingress-canary/ingress-canary-nlp9v" Apr 16 08:39:44.751813 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.751755 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-config-volume\") pod \"dns-default-rqjhz\" (UID: \"b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8\") " pod="openshift-dns/dns-default-rqjhz" Apr 16 08:39:44.751813 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.751795 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kdlxc\" (UniqueName: \"kubernetes.io/projected/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-kube-api-access-kdlxc\") pod \"dns-default-rqjhz\" (UID: \"b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8\") " pod="openshift-dns/dns-default-rqjhz" Apr 16 08:39:44.752055 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.751833 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5s6xr\" (UniqueName: \"kubernetes.io/projected/6136520a-182b-44c5-837e-1dd3dc6af3f7-kube-api-access-5s6xr\") pod \"ingress-canary-nlp9v\" (UID: \"6136520a-182b-44c5-837e-1dd3dc6af3f7\") " pod="openshift-ingress-canary/ingress-canary-nlp9v" Apr 16 08:39:44.752055 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:44.751868 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 08:39:44.752055 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.751902 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-tmp-dir\") pod \"dns-default-rqjhz\" (UID: \"b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8\") " pod="openshift-dns/dns-default-rqjhz" Apr 16 08:39:44.752055 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:44.751931 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6136520a-182b-44c5-837e-1dd3dc6af3f7-cert podName:6136520a-182b-44c5-837e-1dd3dc6af3f7 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:45.251912769 +0000 UTC m=+33.620896804 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6136520a-182b-44c5-837e-1dd3dc6af3f7-cert") pod "ingress-canary-nlp9v" (UID: "6136520a-182b-44c5-837e-1dd3dc6af3f7") : secret "canary-serving-cert" not found Apr 16 08:39:44.752207 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.752070 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-metrics-tls\") pod \"dns-default-rqjhz\" (UID: \"b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8\") " pod="openshift-dns/dns-default-rqjhz" Apr 16 08:39:44.752207 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:44.752178 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 08:39:44.752293 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:44.752235 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-metrics-tls podName:b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:45.252217295 +0000 UTC m=+33.621201333 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-metrics-tls") pod "dns-default-rqjhz" (UID: "b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8") : secret "dns-default-metrics-tls" not found Apr 16 08:39:44.752342 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.752289 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-tmp-dir\") pod \"dns-default-rqjhz\" (UID: \"b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8\") " pod="openshift-dns/dns-default-rqjhz" Apr 16 08:39:44.752536 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.752514 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-config-volume\") pod \"dns-default-rqjhz\" (UID: \"b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8\") " pod="openshift-dns/dns-default-rqjhz" Apr 16 08:39:44.762995 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.762972 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdlxc\" (UniqueName: \"kubernetes.io/projected/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-kube-api-access-kdlxc\") pod \"dns-default-rqjhz\" (UID: \"b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8\") " pod="openshift-dns/dns-default-rqjhz" Apr 16 08:39:44.763135 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:44.763098 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s6xr\" (UniqueName: \"kubernetes.io/projected/6136520a-182b-44c5-837e-1dd3dc6af3f7-kube-api-access-5s6xr\") pod \"ingress-canary-nlp9v\" (UID: \"6136520a-182b-44c5-837e-1dd3dc6af3f7\") " pod="openshift-ingress-canary/ingress-canary-nlp9v" Apr 16 08:39:45.254656 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:45.254622 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-metrics-tls\") pod \"dns-default-rqjhz\" (UID: \"b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8\") " pod="openshift-dns/dns-default-rqjhz" Apr 16 08:39:45.254656 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:45.254662 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6136520a-182b-44c5-837e-1dd3dc6af3f7-cert\") pod \"ingress-canary-nlp9v\" (UID: \"6136520a-182b-44c5-837e-1dd3dc6af3f7\") " pod="openshift-ingress-canary/ingress-canary-nlp9v" Apr 16 08:39:45.255178 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:45.254790 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 08:39:45.255178 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:45.254802 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 08:39:45.255178 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:45.254855 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6136520a-182b-44c5-837e-1dd3dc6af3f7-cert podName:6136520a-182b-44c5-837e-1dd3dc6af3f7 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:46.25483798 +0000 UTC m=+34.623822031 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6136520a-182b-44c5-837e-1dd3dc6af3f7-cert") pod "ingress-canary-nlp9v" (UID: "6136520a-182b-44c5-837e-1dd3dc6af3f7") : secret "canary-serving-cert" not found Apr 16 08:39:45.255178 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:45.254869 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-metrics-tls podName:b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:46.254861903 +0000 UTC m=+34.623845938 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-metrics-tls") pod "dns-default-rqjhz" (UID: "b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8") : secret "dns-default-metrics-tls" not found Apr 16 08:39:45.859511 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:45.859468 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57f278d8-41a1-4600-a089-04d841c7d446-metrics-certs\") pod \"network-metrics-daemon-fxtnv\" (UID: \"57f278d8-41a1-4600-a089-04d841c7d446\") " pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:45.859702 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:45.859594 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:39:45.859702 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:45.859669 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57f278d8-41a1-4600-a089-04d841c7d446-metrics-certs podName:57f278d8-41a1-4600-a089-04d841c7d446 nodeName:}" failed. No retries permitted until 2026-04-16 08:40:17.859651279 +0000 UTC m=+66.228635329 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57f278d8-41a1-4600-a089-04d841c7d446-metrics-certs") pod "network-metrics-daemon-fxtnv" (UID: "57f278d8-41a1-4600-a089-04d841c7d446") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:39:45.960582 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:45.960540 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ddmc5\" (UniqueName: \"kubernetes.io/projected/e285ae7e-c21c-46b9-b705-ce4365740bd9-kube-api-access-ddmc5\") pod \"network-check-target-5l2xc\" (UID: \"e285ae7e-c21c-46b9-b705-ce4365740bd9\") " pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:39:45.960731 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:45.960703 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 16 08:39:45.960731 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:45.960722 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 16 08:39:45.960834 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:45.960732 2578 projected.go:194] Error preparing data for projected volume kube-api-access-ddmc5 for pod openshift-network-diagnostics/network-check-target-5l2xc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:39:45.960834 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:45.960817 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e285ae7e-c21c-46b9-b705-ce4365740bd9-kube-api-access-ddmc5 podName:e285ae7e-c21c-46b9-b705-ce4365740bd9 nodeName:}" failed. No retries permitted until 2026-04-16 08:40:17.960797206 +0000 UTC m=+66.329781248 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ddmc5" (UniqueName: "kubernetes.io/projected/e285ae7e-c21c-46b9-b705-ce4365740bd9-kube-api-access-ddmc5") pod "network-check-target-5l2xc" (UID: "e285ae7e-c21c-46b9-b705-ce4365740bd9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:39:46.213372 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:46.213300 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:39:46.213514 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:46.213319 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:39:46.216680 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:46.216022 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Apr 16 08:39:46.216680 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:46.216042 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9gvlt\"" Apr 16 08:39:46.216680 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:46.216058 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Apr 16 08:39:46.216680 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:46.216065 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zdwck\"" Apr 16 08:39:46.216680 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:46.216142 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Apr 16 08:39:46.262671 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:46.262642 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-metrics-tls\") pod \"dns-default-rqjhz\" (UID: \"b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8\") " pod="openshift-dns/dns-default-rqjhz" Apr 16 08:39:46.262671 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:46.262678 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6136520a-182b-44c5-837e-1dd3dc6af3f7-cert\") pod \"ingress-canary-nlp9v\" (UID: \"6136520a-182b-44c5-837e-1dd3dc6af3f7\") " pod="openshift-ingress-canary/ingress-canary-nlp9v" Apr 16 08:39:46.263117 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:46.262804 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 08:39:46.263117 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:46.262814 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 08:39:46.263117 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:46.262855 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6136520a-182b-44c5-837e-1dd3dc6af3f7-cert podName:6136520a-182b-44c5-837e-1dd3dc6af3f7 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:48.262839356 +0000 UTC m=+36.631823395 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6136520a-182b-44c5-837e-1dd3dc6af3f7-cert") pod "ingress-canary-nlp9v" (UID: "6136520a-182b-44c5-837e-1dd3dc6af3f7") : secret "canary-serving-cert" not found Apr 16 08:39:46.263117 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:46.262884 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-metrics-tls podName:b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:48.262866663 +0000 UTC m=+36.631850716 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-metrics-tls") pod "dns-default-rqjhz" (UID: "b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8") : secret "dns-default-metrics-tls" not found Apr 16 08:39:47.383836 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:47.383803 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7kxqz" event={"ID":"d38f82dc-5ceb-4b6a-b918-e2b0085a9949","Type":"ContainerStarted","Data":"83e80e8f838e3ee1ff9fb0f9e3b5f8261a8cf028c545cbd20f2309a2d8871f61"} Apr 16 08:39:48.277935 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:48.277901 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-metrics-tls\") pod \"dns-default-rqjhz\" (UID: \"b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8\") " pod="openshift-dns/dns-default-rqjhz" Apr 16 08:39:48.277935 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:48.277939 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6136520a-182b-44c5-837e-1dd3dc6af3f7-cert\") pod \"ingress-canary-nlp9v\" (UID: \"6136520a-182b-44c5-837e-1dd3dc6af3f7\") " pod="openshift-ingress-canary/ingress-canary-nlp9v" Apr 16 08:39:48.278132 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:48.278036 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 08:39:48.278132 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:48.278042 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 08:39:48.278132 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:48.278085 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6136520a-182b-44c5-837e-1dd3dc6af3f7-cert podName:6136520a-182b-44c5-837e-1dd3dc6af3f7 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:52.278071792 +0000 UTC m=+40.647055828 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6136520a-182b-44c5-837e-1dd3dc6af3f7-cert") pod "ingress-canary-nlp9v" (UID: "6136520a-182b-44c5-837e-1dd3dc6af3f7") : secret "canary-serving-cert" not found Apr 16 08:39:48.278132 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:48.278097 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-metrics-tls podName:b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:52.278091598 +0000 UTC m=+40.647075633 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-metrics-tls") pod "dns-default-rqjhz" (UID: "b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8") : secret "dns-default-metrics-tls" not found Apr 16 08:39:48.388114 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:48.388082 2578 generic.go:358] "Generic (PLEG): container finished" podID="d38f82dc-5ceb-4b6a-b918-e2b0085a9949" containerID="83e80e8f838e3ee1ff9fb0f9e3b5f8261a8cf028c545cbd20f2309a2d8871f61" exitCode=0 Apr 16 08:39:48.388488 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:48.388131 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7kxqz" event={"ID":"d38f82dc-5ceb-4b6a-b918-e2b0085a9949","Type":"ContainerDied","Data":"83e80e8f838e3ee1ff9fb0f9e3b5f8261a8cf028c545cbd20f2309a2d8871f61"} Apr 16 08:39:49.392927 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:49.392897 2578 generic.go:358] "Generic (PLEG): container finished" podID="d38f82dc-5ceb-4b6a-b918-e2b0085a9949" containerID="3d0c9031772a6388d153359316866f3876d8e048a4d2bb53ccb6e35a1e71b6d8" exitCode=0 Apr 16 08:39:49.393303 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:49.392945 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7kxqz" event={"ID":"d38f82dc-5ceb-4b6a-b918-e2b0085a9949","Type":"ContainerDied","Data":"3d0c9031772a6388d153359316866f3876d8e048a4d2bb53ccb6e35a1e71b6d8"} Apr 16 08:39:50.397231 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:50.397191 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7kxqz" event={"ID":"d38f82dc-5ceb-4b6a-b918-e2b0085a9949","Type":"ContainerStarted","Data":"f7c3e0f07be8fcbfb247b389445f22e13ddcfd44b60b8139697a33e3a0d703db"} Apr 16 08:39:50.417022 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:50.416979 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7kxqz" podStartSLOduration=6.013976845 podStartE2EDuration="38.416964174s" podCreationTimestamp="2026-04-16 08:39:12 +0000 UTC" firstStartedPulling="2026-04-16 08:39:14.704682399 +0000 UTC m=+3.073666434" lastFinishedPulling="2026-04-16 08:39:47.107669714 +0000 UTC m=+35.476653763" observedRunningTime="2026-04-16 08:39:50.416116435 +0000 UTC m=+38.785100504" watchObservedRunningTime="2026-04-16 08:39:50.416964174 +0000 UTC m=+38.785948229" Apr 16 08:39:52.308642 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:52.308588 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-metrics-tls\") pod \"dns-default-rqjhz\" (UID: \"b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8\") " pod="openshift-dns/dns-default-rqjhz" Apr 16 08:39:52.308642 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:39:52.308639 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6136520a-182b-44c5-837e-1dd3dc6af3f7-cert\") pod \"ingress-canary-nlp9v\" (UID: \"6136520a-182b-44c5-837e-1dd3dc6af3f7\") " pod="openshift-ingress-canary/ingress-canary-nlp9v" Apr 16 08:39:52.309079 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:52.308726 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 08:39:52.309079 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:52.308752 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 08:39:52.309079 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:52.308808 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-metrics-tls podName:b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8 nodeName:}" failed. No retries permitted until 2026-04-16 08:40:00.308791833 +0000 UTC m=+48.677775876 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-metrics-tls") pod "dns-default-rqjhz" (UID: "b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8") : secret "dns-default-metrics-tls" not found Apr 16 08:39:52.309079 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:39:52.308823 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6136520a-182b-44c5-837e-1dd3dc6af3f7-cert podName:6136520a-182b-44c5-837e-1dd3dc6af3f7 nodeName:}" failed. No retries permitted until 2026-04-16 08:40:00.30881683 +0000 UTC m=+48.677800866 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6136520a-182b-44c5-837e-1dd3dc6af3f7-cert") pod "ingress-canary-nlp9v" (UID: "6136520a-182b-44c5-837e-1dd3dc6af3f7") : secret "canary-serving-cert" not found Apr 16 08:40:00.358784 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:40:00.358740 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-metrics-tls\") pod \"dns-default-rqjhz\" (UID: \"b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8\") " pod="openshift-dns/dns-default-rqjhz" Apr 16 08:40:00.358784 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:40:00.358788 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6136520a-182b-44c5-837e-1dd3dc6af3f7-cert\") pod \"ingress-canary-nlp9v\" (UID: \"6136520a-182b-44c5-837e-1dd3dc6af3f7\") " pod="openshift-ingress-canary/ingress-canary-nlp9v" Apr 16 08:40:00.359295 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:40:00.358872 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 08:40:00.359295 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:40:00.358876 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 08:40:00.359295 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:40:00.358918 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6136520a-182b-44c5-837e-1dd3dc6af3f7-cert podName:6136520a-182b-44c5-837e-1dd3dc6af3f7 nodeName:}" failed. No retries permitted until 2026-04-16 08:40:16.358905934 +0000 UTC m=+64.727889973 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6136520a-182b-44c5-837e-1dd3dc6af3f7-cert") pod "ingress-canary-nlp9v" (UID: "6136520a-182b-44c5-837e-1dd3dc6af3f7") : secret "canary-serving-cert" not found Apr 16 08:40:00.359295 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:40:00.358932 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-metrics-tls podName:b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8 nodeName:}" failed. No retries permitted until 2026-04-16 08:40:16.358925061 +0000 UTC m=+64.727909096 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-metrics-tls") pod "dns-default-rqjhz" (UID: "b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8") : secret "dns-default-metrics-tls" not found Apr 16 08:40:16.365968 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:40:16.365936 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-metrics-tls\") pod \"dns-default-rqjhz\" (UID: \"b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8\") " pod="openshift-dns/dns-default-rqjhz" Apr 16 08:40:16.365968 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:40:16.365973 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6136520a-182b-44c5-837e-1dd3dc6af3f7-cert\") pod \"ingress-canary-nlp9v\" (UID: \"6136520a-182b-44c5-837e-1dd3dc6af3f7\") " pod="openshift-ingress-canary/ingress-canary-nlp9v" Apr 16 08:40:16.366510 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:40:16.366067 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 08:40:16.366510 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:40:16.366086 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 08:40:16.366510 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:40:16.366113 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6136520a-182b-44c5-837e-1dd3dc6af3f7-cert podName:6136520a-182b-44c5-837e-1dd3dc6af3f7 nodeName:}" failed. No retries permitted until 2026-04-16 08:40:48.366100534 +0000 UTC m=+96.735084574 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6136520a-182b-44c5-837e-1dd3dc6af3f7-cert") pod "ingress-canary-nlp9v" (UID: "6136520a-182b-44c5-837e-1dd3dc6af3f7") : secret "canary-serving-cert" not found Apr 16 08:40:16.366510 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:40:16.366145 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-metrics-tls podName:b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8 nodeName:}" failed. No retries permitted until 2026-04-16 08:40:48.366128002 +0000 UTC m=+96.735112053 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-metrics-tls") pod "dns-default-rqjhz" (UID: "b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8") : secret "dns-default-metrics-tls" not found Apr 16 08:40:17.874704 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:40:17.874676 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57f278d8-41a1-4600-a089-04d841c7d446-metrics-certs\") pod \"network-metrics-daemon-fxtnv\" (UID: \"57f278d8-41a1-4600-a089-04d841c7d446\") " pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:40:17.876689 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:40:17.876673 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Apr 16 08:40:17.885415 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:40:17.885401 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Apr 16 08:40:17.885494 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:40:17.885448 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57f278d8-41a1-4600-a089-04d841c7d446-metrics-certs podName:57f278d8-41a1-4600-a089-04d841c7d446 nodeName:}" failed. No retries permitted until 2026-04-16 08:41:21.885435135 +0000 UTC m=+130.254419175 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57f278d8-41a1-4600-a089-04d841c7d446-metrics-certs") pod "network-metrics-daemon-fxtnv" (UID: "57f278d8-41a1-4600-a089-04d841c7d446") : secret "metrics-daemon-secret" not found Apr 16 08:40:17.975784 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:40:17.975738 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ddmc5\" (UniqueName: \"kubernetes.io/projected/e285ae7e-c21c-46b9-b705-ce4365740bd9-kube-api-access-ddmc5\") pod \"network-check-target-5l2xc\" (UID: \"e285ae7e-c21c-46b9-b705-ce4365740bd9\") " pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:40:17.978243 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:40:17.978229 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Apr 16 08:40:17.988445 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:40:17.988425 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Apr 16 08:40:17.999547 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:40:17.999523 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddmc5\" (UniqueName: \"kubernetes.io/projected/e285ae7e-c21c-46b9-b705-ce4365740bd9-kube-api-access-ddmc5\") pod \"network-check-target-5l2xc\" (UID: \"e285ae7e-c21c-46b9-b705-ce4365740bd9\") " pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:40:18.026290 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:40:18.026270 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zdwck\"" Apr 16 08:40:18.034828 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:40:18.034809 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:40:18.217702 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:40:18.217679 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5l2xc"] Apr 16 08:40:18.221120 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:40:18.221097 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode285ae7e_c21c_46b9_b705_ce4365740bd9.slice/crio-de4f0694162bde5e5a2209af8070a3232a786ddb0ce6cfb104114d77e1bb810c WatchSource:0}: Error finding container de4f0694162bde5e5a2209af8070a3232a786ddb0ce6cfb104114d77e1bb810c: Status 404 returned error can't find the container with id de4f0694162bde5e5a2209af8070a3232a786ddb0ce6cfb104114d77e1bb810c Apr 16 08:40:18.449197 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:40:18.449136 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5l2xc" event={"ID":"e285ae7e-c21c-46b9-b705-ce4365740bd9","Type":"ContainerStarted","Data":"de4f0694162bde5e5a2209af8070a3232a786ddb0ce6cfb104114d77e1bb810c"} Apr 16 08:40:21.456310 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:40:21.456280 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5l2xc" event={"ID":"e285ae7e-c21c-46b9-b705-ce4365740bd9","Type":"ContainerStarted","Data":"39235a4a84118d98a8df906482469bea3407c528ad781c9aee2696267ccc8b99"} Apr 16 08:40:21.456684 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:40:21.456423 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:40:21.471049 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:40:21.471008 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5l2xc" podStartSLOduration=66.948379223 podStartE2EDuration="1m9.470995566s" podCreationTimestamp="2026-04-16 08:39:12 +0000 UTC" firstStartedPulling="2026-04-16 08:40:18.22275761 +0000 UTC m=+66.591741648" lastFinishedPulling="2026-04-16 08:40:20.745373941 +0000 UTC m=+69.114357991" observedRunningTime="2026-04-16 08:40:21.47006813 +0000 UTC m=+69.839052186" watchObservedRunningTime="2026-04-16 08:40:21.470995566 +0000 UTC m=+69.839979623" Apr 16 08:40:48.380487 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:40:48.380432 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-metrics-tls\") pod \"dns-default-rqjhz\" (UID: \"b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8\") " pod="openshift-dns/dns-default-rqjhz" Apr 16 08:40:48.380487 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:40:48.380487 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6136520a-182b-44c5-837e-1dd3dc6af3f7-cert\") pod \"ingress-canary-nlp9v\" (UID: \"6136520a-182b-44c5-837e-1dd3dc6af3f7\") " pod="openshift-ingress-canary/ingress-canary-nlp9v" Apr 16 08:40:48.380945 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:40:48.380576 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 08:40:48.380945 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:40:48.380586 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 08:40:48.380945 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:40:48.380629 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6136520a-182b-44c5-837e-1dd3dc6af3f7-cert podName:6136520a-182b-44c5-837e-1dd3dc6af3f7 nodeName:}" failed. No retries permitted until 2026-04-16 08:41:52.380614924 +0000 UTC m=+160.749598962 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6136520a-182b-44c5-837e-1dd3dc6af3f7-cert") pod "ingress-canary-nlp9v" (UID: "6136520a-182b-44c5-837e-1dd3dc6af3f7") : secret "canary-serving-cert" not found Apr 16 08:40:48.380945 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:40:48.380643 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-metrics-tls podName:b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8 nodeName:}" failed. No retries permitted until 2026-04-16 08:41:52.380636054 +0000 UTC m=+160.749620089 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-metrics-tls") pod "dns-default-rqjhz" (UID: "b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8") : secret "dns-default-metrics-tls" not found Apr 16 08:40:52.460814 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:40:52.460787 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5l2xc" Apr 16 08:41:00.530784 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.530743 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7d955d5dd4-bff4c"] Apr 16 08:41:00.533619 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.533600 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7d955d5dd4-bff4c" Apr 16 08:41:00.535608 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.535584 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Apr 16 08:41:00.535608 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.535602 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Apr 16 08:41:00.536203 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.536186 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-rrnfs\"" Apr 16 08:41:00.537177 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.537152 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6667474d89-qgz8h"] Apr 16 08:41:00.540232 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.540022 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6667474d89-qgz8h" Apr 16 08:41:00.540573 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.540554 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-54fdbc7b78-dxx97"] Apr 16 08:41:00.542071 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.542053 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-f4vlc\"" Apr 16 08:41:00.542792 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.542756 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Apr 16 08:41:00.543057 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.542942 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Apr 16 08:41:00.543057 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.542974 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Apr 16 08:41:00.543057 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.542984 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Apr 16 08:41:00.543371 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.543354 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:00.543761 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.543741 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7d955d5dd4-bff4c"] Apr 16 08:41:00.545717 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.545699 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Apr 16 08:41:00.545829 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.545734 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-ctdwt\"" Apr 16 08:41:00.545899 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.545844 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Apr 16 08:41:00.546108 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.546090 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Apr 16 08:41:00.546182 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.546123 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Apr 16 08:41:00.546238 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.546230 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Apr 16 08:41:00.546295 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.546281 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Apr 16 08:41:00.556235 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.556214 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-54fdbc7b78-dxx97"] Apr 16 08:41:00.559025 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.559004 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6667474d89-qgz8h"] Apr 16 08:41:00.640255 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.640220 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-69965bb79d-q8lbg"] Apr 16 08:41:00.643853 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.643828 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-q8lbg" Apr 16 08:41:00.645890 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.645870 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Apr 16 08:41:00.646070 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.646052 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Apr 16 08:41:00.646298 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.646279 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Apr 16 08:41:00.646404 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.646283 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Apr 16 08:41:00.646404 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.646307 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-kz77l\"" Apr 16 08:41:00.653162 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.653141 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-69965bb79d-q8lbg"] Apr 16 08:41:00.660985 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.660965 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b9v5\" (UniqueName: \"kubernetes.io/projected/28d97c1a-528d-4451-8bc1-c2f47b39d660-kube-api-access-2b9v5\") pod \"volume-data-source-validator-7d955d5dd4-bff4c\" (UID: \"28d97c1a-528d-4451-8bc1-c2f47b39d660\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7d955d5dd4-bff4c" Apr 16 08:41:00.661071 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.661010 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f45d6895-a6e2-4531-ac1e-6e8c46c51012-default-certificate\") pod \"router-default-54fdbc7b78-dxx97\" (UID: \"f45d6895-a6e2-4531-ac1e-6e8c46c51012\") " pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:00.661071 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.661037 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f45d6895-a6e2-4531-ac1e-6e8c46c51012-stats-auth\") pod \"router-default-54fdbc7b78-dxx97\" (UID: \"f45d6895-a6e2-4531-ac1e-6e8c46c51012\") " pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:00.661071 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.661056 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd224\" (UniqueName: \"kubernetes.io/projected/8af06da5-de2c-48a0-8894-9fde08d3085a-kube-api-access-jd224\") pod \"cluster-monitoring-operator-6667474d89-qgz8h\" (UID: \"8af06da5-de2c-48a0-8894-9fde08d3085a\") " pod="openshift-monitoring/cluster-monitoring-operator-6667474d89-qgz8h" Apr 16 08:41:00.661166 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.661073 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f45d6895-a6e2-4531-ac1e-6e8c46c51012-service-ca-bundle\") pod \"router-default-54fdbc7b78-dxx97\" (UID: \"f45d6895-a6e2-4531-ac1e-6e8c46c51012\") " pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:00.661166 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.661091 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8af06da5-de2c-48a0-8894-9fde08d3085a-telemetry-config\") pod \"cluster-monitoring-operator-6667474d89-qgz8h\" (UID: \"8af06da5-de2c-48a0-8894-9fde08d3085a\") " pod="openshift-monitoring/cluster-monitoring-operator-6667474d89-qgz8h" Apr 16 08:41:00.661166 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.661119 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8af06da5-de2c-48a0-8894-9fde08d3085a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6667474d89-qgz8h\" (UID: \"8af06da5-de2c-48a0-8894-9fde08d3085a\") " pod="openshift-monitoring/cluster-monitoring-operator-6667474d89-qgz8h" Apr 16 08:41:00.661166 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.661136 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9q68r\" (UniqueName: \"kubernetes.io/projected/f45d6895-a6e2-4531-ac1e-6e8c46c51012-kube-api-access-9q68r\") pod \"router-default-54fdbc7b78-dxx97\" (UID: \"f45d6895-a6e2-4531-ac1e-6e8c46c51012\") " pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:00.661166 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.661152 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f45d6895-a6e2-4531-ac1e-6e8c46c51012-metrics-certs\") pod \"router-default-54fdbc7b78-dxx97\" (UID: \"f45d6895-a6e2-4531-ac1e-6e8c46c51012\") " pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:00.762464 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.762421 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9q68r\" (UniqueName: \"kubernetes.io/projected/f45d6895-a6e2-4531-ac1e-6e8c46c51012-kube-api-access-9q68r\") pod \"router-default-54fdbc7b78-dxx97\" (UID: \"f45d6895-a6e2-4531-ac1e-6e8c46c51012\") " pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:00.762464 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.762463 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f45d6895-a6e2-4531-ac1e-6e8c46c51012-metrics-certs\") pod \"router-default-54fdbc7b78-dxx97\" (UID: \"f45d6895-a6e2-4531-ac1e-6e8c46c51012\") " pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:00.762670 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.762487 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7g5p\" (UniqueName: \"kubernetes.io/projected/47490d5e-7b28-4469-acf3-5cd7e9637c19-kube-api-access-k7g5p\") pod \"service-ca-operator-69965bb79d-q8lbg\" (UID: \"47490d5e-7b28-4469-acf3-5cd7e9637c19\") " pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-q8lbg" Apr 16 08:41:00.762670 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.762513 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2b9v5\" (UniqueName: \"kubernetes.io/projected/28d97c1a-528d-4451-8bc1-c2f47b39d660-kube-api-access-2b9v5\") pod \"volume-data-source-validator-7d955d5dd4-bff4c\" (UID: \"28d97c1a-528d-4451-8bc1-c2f47b39d660\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7d955d5dd4-bff4c" Apr 16 08:41:00.762670 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:00.762597 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Apr 16 08:41:00.762815 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:00.762684 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f45d6895-a6e2-4531-ac1e-6e8c46c51012-metrics-certs podName:f45d6895-a6e2-4531-ac1e-6e8c46c51012 nodeName:}" failed. No retries permitted until 2026-04-16 08:41:01.262664856 +0000 UTC m=+109.631648897 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f45d6895-a6e2-4531-ac1e-6e8c46c51012-metrics-certs") pod "router-default-54fdbc7b78-dxx97" (UID: "f45d6895-a6e2-4531-ac1e-6e8c46c51012") : secret "router-metrics-certs-default" not found Apr 16 08:41:00.762815 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.762599 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f45d6895-a6e2-4531-ac1e-6e8c46c51012-default-certificate\") pod \"router-default-54fdbc7b78-dxx97\" (UID: \"f45d6895-a6e2-4531-ac1e-6e8c46c51012\") " pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:00.762815 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.762740 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f45d6895-a6e2-4531-ac1e-6e8c46c51012-stats-auth\") pod \"router-default-54fdbc7b78-dxx97\" (UID: \"f45d6895-a6e2-4531-ac1e-6e8c46c51012\") " pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:00.762815 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.762761 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/47490d5e-7b28-4469-acf3-5cd7e9637c19-serving-cert\") pod \"service-ca-operator-69965bb79d-q8lbg\" (UID: \"47490d5e-7b28-4469-acf3-5cd7e9637c19\") " pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-q8lbg" Apr 16 08:41:00.762815 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.762811 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/47490d5e-7b28-4469-acf3-5cd7e9637c19-config\") pod \"service-ca-operator-69965bb79d-q8lbg\" (UID: \"47490d5e-7b28-4469-acf3-5cd7e9637c19\") " pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-q8lbg" Apr 16 08:41:00.763017 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.762866 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jd224\" (UniqueName: \"kubernetes.io/projected/8af06da5-de2c-48a0-8894-9fde08d3085a-kube-api-access-jd224\") pod \"cluster-monitoring-operator-6667474d89-qgz8h\" (UID: \"8af06da5-de2c-48a0-8894-9fde08d3085a\") " pod="openshift-monitoring/cluster-monitoring-operator-6667474d89-qgz8h" Apr 16 08:41:00.763017 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.762896 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f45d6895-a6e2-4531-ac1e-6e8c46c51012-service-ca-bundle\") pod \"router-default-54fdbc7b78-dxx97\" (UID: \"f45d6895-a6e2-4531-ac1e-6e8c46c51012\") " pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:00.763017 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.762929 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8af06da5-de2c-48a0-8894-9fde08d3085a-telemetry-config\") pod \"cluster-monitoring-operator-6667474d89-qgz8h\" (UID: \"8af06da5-de2c-48a0-8894-9fde08d3085a\") " pod="openshift-monitoring/cluster-monitoring-operator-6667474d89-qgz8h" Apr 16 08:41:00.763017 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.762956 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8af06da5-de2c-48a0-8894-9fde08d3085a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6667474d89-qgz8h\" (UID: \"8af06da5-de2c-48a0-8894-9fde08d3085a\") " pod="openshift-monitoring/cluster-monitoring-operator-6667474d89-qgz8h" Apr 16 08:41:00.763198 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:00.763065 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f45d6895-a6e2-4531-ac1e-6e8c46c51012-service-ca-bundle podName:f45d6895-a6e2-4531-ac1e-6e8c46c51012 nodeName:}" failed. No retries permitted until 2026-04-16 08:41:01.263047652 +0000 UTC m=+109.632031710 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f45d6895-a6e2-4531-ac1e-6e8c46c51012-service-ca-bundle") pod "router-default-54fdbc7b78-dxx97" (UID: "f45d6895-a6e2-4531-ac1e-6e8c46c51012") : configmap references non-existent config key: service-ca.crt Apr 16 08:41:00.763198 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:00.763072 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Apr 16 08:41:00.763198 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:00.763118 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8af06da5-de2c-48a0-8894-9fde08d3085a-cluster-monitoring-operator-tls podName:8af06da5-de2c-48a0-8894-9fde08d3085a nodeName:}" failed. No retries permitted until 2026-04-16 08:41:01.263106476 +0000 UTC m=+109.632090511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8af06da5-de2c-48a0-8894-9fde08d3085a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6667474d89-qgz8h" (UID: "8af06da5-de2c-48a0-8894-9fde08d3085a") : secret "cluster-monitoring-operator-tls" not found Apr 16 08:41:00.764134 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.764109 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8af06da5-de2c-48a0-8894-9fde08d3085a-telemetry-config\") pod \"cluster-monitoring-operator-6667474d89-qgz8h\" (UID: \"8af06da5-de2c-48a0-8894-9fde08d3085a\") " pod="openshift-monitoring/cluster-monitoring-operator-6667474d89-qgz8h" Apr 16 08:41:00.765189 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.765170 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f45d6895-a6e2-4531-ac1e-6e8c46c51012-stats-auth\") pod \"router-default-54fdbc7b78-dxx97\" (UID: \"f45d6895-a6e2-4531-ac1e-6e8c46c51012\") " pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:00.765314 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.765297 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f45d6895-a6e2-4531-ac1e-6e8c46c51012-default-certificate\") pod \"router-default-54fdbc7b78-dxx97\" (UID: \"f45d6895-a6e2-4531-ac1e-6e8c46c51012\") " pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:00.771264 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.771238 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9q68r\" (UniqueName: \"kubernetes.io/projected/f45d6895-a6e2-4531-ac1e-6e8c46c51012-kube-api-access-9q68r\") pod \"router-default-54fdbc7b78-dxx97\" (UID: \"f45d6895-a6e2-4531-ac1e-6e8c46c51012\") " pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:00.771372 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.771308 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b9v5\" (UniqueName: \"kubernetes.io/projected/28d97c1a-528d-4451-8bc1-c2f47b39d660-kube-api-access-2b9v5\") pod \"volume-data-source-validator-7d955d5dd4-bff4c\" (UID: \"28d97c1a-528d-4451-8bc1-c2f47b39d660\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7d955d5dd4-bff4c" Apr 16 08:41:00.771431 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.771373 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd224\" (UniqueName: \"kubernetes.io/projected/8af06da5-de2c-48a0-8894-9fde08d3085a-kube-api-access-jd224\") pod \"cluster-monitoring-operator-6667474d89-qgz8h\" (UID: \"8af06da5-de2c-48a0-8894-9fde08d3085a\") " pod="openshift-monitoring/cluster-monitoring-operator-6667474d89-qgz8h" Apr 16 08:41:00.844206 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.844112 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7d955d5dd4-bff4c" Apr 16 08:41:00.863972 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.863943 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/47490d5e-7b28-4469-acf3-5cd7e9637c19-serving-cert\") pod \"service-ca-operator-69965bb79d-q8lbg\" (UID: \"47490d5e-7b28-4469-acf3-5cd7e9637c19\") " pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-q8lbg" Apr 16 08:41:00.863972 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.863979 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/47490d5e-7b28-4469-acf3-5cd7e9637c19-config\") pod \"service-ca-operator-69965bb79d-q8lbg\" (UID: \"47490d5e-7b28-4469-acf3-5cd7e9637c19\") " pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-q8lbg" Apr 16 08:41:00.864361 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.864049 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k7g5p\" (UniqueName: \"kubernetes.io/projected/47490d5e-7b28-4469-acf3-5cd7e9637c19-kube-api-access-k7g5p\") pod \"service-ca-operator-69965bb79d-q8lbg\" (UID: \"47490d5e-7b28-4469-acf3-5cd7e9637c19\") " pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-q8lbg" Apr 16 08:41:00.864569 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.864544 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/47490d5e-7b28-4469-acf3-5cd7e9637c19-config\") pod \"service-ca-operator-69965bb79d-q8lbg\" (UID: \"47490d5e-7b28-4469-acf3-5cd7e9637c19\") " pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-q8lbg" Apr 16 08:41:00.866675 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.866645 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/47490d5e-7b28-4469-acf3-5cd7e9637c19-serving-cert\") pod \"service-ca-operator-69965bb79d-q8lbg\" (UID: \"47490d5e-7b28-4469-acf3-5cd7e9637c19\") " pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-q8lbg" Apr 16 08:41:00.870950 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.870926 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7g5p\" (UniqueName: \"kubernetes.io/projected/47490d5e-7b28-4469-acf3-5cd7e9637c19-kube-api-access-k7g5p\") pod \"service-ca-operator-69965bb79d-q8lbg\" (UID: \"47490d5e-7b28-4469-acf3-5cd7e9637c19\") " pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-q8lbg" Apr 16 08:41:00.952358 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.952322 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-q8lbg" Apr 16 08:41:00.957458 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:00.957374 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7d955d5dd4-bff4c"] Apr 16 08:41:00.961888 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:41:00.961851 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28d97c1a_528d_4451_8bc1_c2f47b39d660.slice/crio-9048dc31c2d95d8a5dd692d3b8cd2c10c95186523d0b70a6b870b295633c830d WatchSource:0}: Error finding container 9048dc31c2d95d8a5dd692d3b8cd2c10c95186523d0b70a6b870b295633c830d: Status 404 returned error can't find the container with id 9048dc31c2d95d8a5dd692d3b8cd2c10c95186523d0b70a6b870b295633c830d Apr 16 08:41:01.064884 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:01.064831 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-69965bb79d-q8lbg"] Apr 16 08:41:01.069105 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:41:01.069077 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47490d5e_7b28_4469_acf3_5cd7e9637c19.slice/crio-b12e6dbc8796988be1be6bf338c9ade6a1859373e3e856f56dcde825a897188e WatchSource:0}: Error finding container b12e6dbc8796988be1be6bf338c9ade6a1859373e3e856f56dcde825a897188e: Status 404 returned error can't find the container with id b12e6dbc8796988be1be6bf338c9ade6a1859373e3e856f56dcde825a897188e Apr 16 08:41:01.266038 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:01.266006 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f45d6895-a6e2-4531-ac1e-6e8c46c51012-service-ca-bundle\") pod \"router-default-54fdbc7b78-dxx97\" (UID: \"f45d6895-a6e2-4531-ac1e-6e8c46c51012\") " pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:01.266250 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:01.266061 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8af06da5-de2c-48a0-8894-9fde08d3085a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6667474d89-qgz8h\" (UID: \"8af06da5-de2c-48a0-8894-9fde08d3085a\") " pod="openshift-monitoring/cluster-monitoring-operator-6667474d89-qgz8h" Apr 16 08:41:01.266250 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:01.266094 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f45d6895-a6e2-4531-ac1e-6e8c46c51012-metrics-certs\") pod \"router-default-54fdbc7b78-dxx97\" (UID: \"f45d6895-a6e2-4531-ac1e-6e8c46c51012\") " pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:01.266250 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:01.266189 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f45d6895-a6e2-4531-ac1e-6e8c46c51012-service-ca-bundle podName:f45d6895-a6e2-4531-ac1e-6e8c46c51012 nodeName:}" failed. No retries permitted until 2026-04-16 08:41:02.266169513 +0000 UTC m=+110.635153553 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f45d6895-a6e2-4531-ac1e-6e8c46c51012-service-ca-bundle") pod "router-default-54fdbc7b78-dxx97" (UID: "f45d6895-a6e2-4531-ac1e-6e8c46c51012") : configmap references non-existent config key: service-ca.crt Apr 16 08:41:01.266250 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:01.266204 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Apr 16 08:41:01.266250 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:01.266209 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Apr 16 08:41:01.266250 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:01.266255 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f45d6895-a6e2-4531-ac1e-6e8c46c51012-metrics-certs podName:f45d6895-a6e2-4531-ac1e-6e8c46c51012 nodeName:}" failed. No retries permitted until 2026-04-16 08:41:02.266238508 +0000 UTC m=+110.635222553 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f45d6895-a6e2-4531-ac1e-6e8c46c51012-metrics-certs") pod "router-default-54fdbc7b78-dxx97" (UID: "f45d6895-a6e2-4531-ac1e-6e8c46c51012") : secret "router-metrics-certs-default" not found Apr 16 08:41:01.266458 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:01.266272 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8af06da5-de2c-48a0-8894-9fde08d3085a-cluster-monitoring-operator-tls podName:8af06da5-de2c-48a0-8894-9fde08d3085a nodeName:}" failed. No retries permitted until 2026-04-16 08:41:02.266263175 +0000 UTC m=+110.635247217 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8af06da5-de2c-48a0-8894-9fde08d3085a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6667474d89-qgz8h" (UID: "8af06da5-de2c-48a0-8894-9fde08d3085a") : secret "cluster-monitoring-operator-tls" not found Apr 16 08:41:01.524390 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:01.524307 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-q8lbg" event={"ID":"47490d5e-7b28-4469-acf3-5cd7e9637c19","Type":"ContainerStarted","Data":"b12e6dbc8796988be1be6bf338c9ade6a1859373e3e856f56dcde825a897188e"} Apr 16 08:41:01.525337 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:01.525306 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7d955d5dd4-bff4c" event={"ID":"28d97c1a-528d-4451-8bc1-c2f47b39d660","Type":"ContainerStarted","Data":"9048dc31c2d95d8a5dd692d3b8cd2c10c95186523d0b70a6b870b295633c830d"} Apr 16 08:41:02.275083 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:02.275042 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f45d6895-a6e2-4531-ac1e-6e8c46c51012-service-ca-bundle\") pod \"router-default-54fdbc7b78-dxx97\" (UID: \"f45d6895-a6e2-4531-ac1e-6e8c46c51012\") " pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:02.275580 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:02.275114 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8af06da5-de2c-48a0-8894-9fde08d3085a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6667474d89-qgz8h\" (UID: \"8af06da5-de2c-48a0-8894-9fde08d3085a\") " pod="openshift-monitoring/cluster-monitoring-operator-6667474d89-qgz8h" Apr 16 08:41:02.275580 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:02.275156 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f45d6895-a6e2-4531-ac1e-6e8c46c51012-metrics-certs\") pod \"router-default-54fdbc7b78-dxx97\" (UID: \"f45d6895-a6e2-4531-ac1e-6e8c46c51012\") " pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:02.275580 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:02.275228 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f45d6895-a6e2-4531-ac1e-6e8c46c51012-service-ca-bundle podName:f45d6895-a6e2-4531-ac1e-6e8c46c51012 nodeName:}" failed. No retries permitted until 2026-04-16 08:41:04.275205343 +0000 UTC m=+112.644189394 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f45d6895-a6e2-4531-ac1e-6e8c46c51012-service-ca-bundle") pod "router-default-54fdbc7b78-dxx97" (UID: "f45d6895-a6e2-4531-ac1e-6e8c46c51012") : configmap references non-existent config key: service-ca.crt Apr 16 08:41:02.275580 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:02.275264 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Apr 16 08:41:02.275580 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:02.275286 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Apr 16 08:41:02.275580 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:02.275305 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8af06da5-de2c-48a0-8894-9fde08d3085a-cluster-monitoring-operator-tls podName:8af06da5-de2c-48a0-8894-9fde08d3085a nodeName:}" failed. No retries permitted until 2026-04-16 08:41:04.275294857 +0000 UTC m=+112.644278893 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8af06da5-de2c-48a0-8894-9fde08d3085a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6667474d89-qgz8h" (UID: "8af06da5-de2c-48a0-8894-9fde08d3085a") : secret "cluster-monitoring-operator-tls" not found Apr 16 08:41:02.275580 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:02.275332 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f45d6895-a6e2-4531-ac1e-6e8c46c51012-metrics-certs podName:f45d6895-a6e2-4531-ac1e-6e8c46c51012 nodeName:}" failed. No retries permitted until 2026-04-16 08:41:04.275318522 +0000 UTC m=+112.644302562 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f45d6895-a6e2-4531-ac1e-6e8c46c51012-metrics-certs") pod "router-default-54fdbc7b78-dxx97" (UID: "f45d6895-a6e2-4531-ac1e-6e8c46c51012") : secret "router-metrics-certs-default" not found Apr 16 08:41:02.528475 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:02.528420 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7d955d5dd4-bff4c" event={"ID":"28d97c1a-528d-4451-8bc1-c2f47b39d660","Type":"ContainerStarted","Data":"474305eb9a49ddc8484d0e45bc6cd953cb75186af62a0cbe788453ab52c637a5"} Apr 16 08:41:02.541538 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:02.541484 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-7d955d5dd4-bff4c" podStartSLOduration=1.437347368 podStartE2EDuration="2.541464986s" podCreationTimestamp="2026-04-16 08:41:00 +0000 UTC" firstStartedPulling="2026-04-16 08:41:00.963881255 +0000 UTC m=+109.332865290" lastFinishedPulling="2026-04-16 08:41:02.067998867 +0000 UTC m=+110.436982908" observedRunningTime="2026-04-16 08:41:02.540829847 +0000 UTC m=+110.909813904" watchObservedRunningTime="2026-04-16 08:41:02.541464986 +0000 UTC m=+110.910449045" Apr 16 08:41:03.531674 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:03.531634 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-q8lbg" event={"ID":"47490d5e-7b28-4469-acf3-5cd7e9637c19","Type":"ContainerStarted","Data":"5e352fc0369bd143a6ab660dda71fe9ca640dc9428bc64556c3f3c063e4a8c6f"} Apr 16 08:41:03.545501 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:03.545459 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-q8lbg" podStartSLOduration=1.6288925509999999 podStartE2EDuration="3.54544554s" podCreationTimestamp="2026-04-16 08:41:00 +0000 UTC" firstStartedPulling="2026-04-16 08:41:01.070735635 +0000 UTC m=+109.439719670" lastFinishedPulling="2026-04-16 08:41:02.987288621 +0000 UTC m=+111.356272659" observedRunningTime="2026-04-16 08:41:03.544473972 +0000 UTC m=+111.913458030" watchObservedRunningTime="2026-04-16 08:41:03.54544554 +0000 UTC m=+111.914429596" Apr 16 08:41:04.291104 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:04.291064 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f45d6895-a6e2-4531-ac1e-6e8c46c51012-service-ca-bundle\") pod \"router-default-54fdbc7b78-dxx97\" (UID: \"f45d6895-a6e2-4531-ac1e-6e8c46c51012\") " pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:04.291104 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:04.291117 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8af06da5-de2c-48a0-8894-9fde08d3085a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6667474d89-qgz8h\" (UID: \"8af06da5-de2c-48a0-8894-9fde08d3085a\") " pod="openshift-monitoring/cluster-monitoring-operator-6667474d89-qgz8h" Apr 16 08:41:04.291362 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:04.291138 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f45d6895-a6e2-4531-ac1e-6e8c46c51012-metrics-certs\") pod \"router-default-54fdbc7b78-dxx97\" (UID: \"f45d6895-a6e2-4531-ac1e-6e8c46c51012\") " pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:04.291362 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:04.291231 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Apr 16 08:41:04.291362 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:04.291243 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f45d6895-a6e2-4531-ac1e-6e8c46c51012-service-ca-bundle podName:f45d6895-a6e2-4531-ac1e-6e8c46c51012 nodeName:}" failed. No retries permitted until 2026-04-16 08:41:08.291225884 +0000 UTC m=+116.660209924 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f45d6895-a6e2-4531-ac1e-6e8c46c51012-service-ca-bundle") pod "router-default-54fdbc7b78-dxx97" (UID: "f45d6895-a6e2-4531-ac1e-6e8c46c51012") : configmap references non-existent config key: service-ca.crt Apr 16 08:41:04.291362 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:04.291246 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Apr 16 08:41:04.291362 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:04.291270 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f45d6895-a6e2-4531-ac1e-6e8c46c51012-metrics-certs podName:f45d6895-a6e2-4531-ac1e-6e8c46c51012 nodeName:}" failed. No retries permitted until 2026-04-16 08:41:08.29125884 +0000 UTC m=+116.660242879 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f45d6895-a6e2-4531-ac1e-6e8c46c51012-metrics-certs") pod "router-default-54fdbc7b78-dxx97" (UID: "f45d6895-a6e2-4531-ac1e-6e8c46c51012") : secret "router-metrics-certs-default" not found Apr 16 08:41:04.291362 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:04.291307 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8af06da5-de2c-48a0-8894-9fde08d3085a-cluster-monitoring-operator-tls podName:8af06da5-de2c-48a0-8894-9fde08d3085a nodeName:}" failed. No retries permitted until 2026-04-16 08:41:08.291292213 +0000 UTC m=+116.660276281 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8af06da5-de2c-48a0-8894-9fde08d3085a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6667474d89-qgz8h" (UID: "8af06da5-de2c-48a0-8894-9fde08d3085a") : secret "cluster-monitoring-operator-tls" not found Apr 16 08:41:05.211523 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:05.211493 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-64d4d94569-zr5wc"] Apr 16 08:41:05.214419 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:05.214404 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-64d4d94569-zr5wc" Apr 16 08:41:05.216266 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:05.216240 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Apr 16 08:41:05.216403 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:05.216386 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Apr 16 08:41:05.216804 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:05.216788 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-vdvff\"" Apr 16 08:41:05.223266 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:05.223246 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-64d4d94569-zr5wc"] Apr 16 08:41:05.297466 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:05.297442 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgjvl\" (UniqueName: \"kubernetes.io/projected/b86396a6-050a-4fc8-87a7-3185d62b983c-kube-api-access-sgjvl\") pod \"migrator-64d4d94569-zr5wc\" (UID: \"b86396a6-050a-4fc8-87a7-3185d62b983c\") " pod="openshift-kube-storage-version-migrator/migrator-64d4d94569-zr5wc" Apr 16 08:41:05.398060 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:05.398020 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sgjvl\" (UniqueName: \"kubernetes.io/projected/b86396a6-050a-4fc8-87a7-3185d62b983c-kube-api-access-sgjvl\") pod \"migrator-64d4d94569-zr5wc\" (UID: \"b86396a6-050a-4fc8-87a7-3185d62b983c\") " pod="openshift-kube-storage-version-migrator/migrator-64d4d94569-zr5wc" Apr 16 08:41:05.405445 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:05.405421 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgjvl\" (UniqueName: \"kubernetes.io/projected/b86396a6-050a-4fc8-87a7-3185d62b983c-kube-api-access-sgjvl\") pod \"migrator-64d4d94569-zr5wc\" (UID: \"b86396a6-050a-4fc8-87a7-3185d62b983c\") " pod="openshift-kube-storage-version-migrator/migrator-64d4d94569-zr5wc" Apr 16 08:41:05.523668 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:05.523626 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-64d4d94569-zr5wc" Apr 16 08:41:05.632939 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:05.632912 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-64d4d94569-zr5wc"] Apr 16 08:41:05.636548 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:41:05.636523 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb86396a6_050a_4fc8_87a7_3185d62b983c.slice/crio-4897a2a6af634febb9bcdec37446fd8a3a8eca68c68b21b374395d21f57d2bd2 WatchSource:0}: Error finding container 4897a2a6af634febb9bcdec37446fd8a3a8eca68c68b21b374395d21f57d2bd2: Status 404 returned error can't find the container with id 4897a2a6af634febb9bcdec37446fd8a3a8eca68c68b21b374395d21f57d2bd2 Apr 16 08:41:06.538017 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:06.537983 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-64d4d94569-zr5wc" event={"ID":"b86396a6-050a-4fc8-87a7-3185d62b983c","Type":"ContainerStarted","Data":"4897a2a6af634febb9bcdec37446fd8a3a8eca68c68b21b374395d21f57d2bd2"} Apr 16 08:41:06.888124 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:06.888048 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qxxqp_69ff05e4-5126-420e-ba65-9f88a7fec110/dns-node-resolver/0.log" Apr 16 08:41:07.486175 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:07.486154 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7rgcb_17b0a66e-5c6b-4926-b67d-e916dd6009d3/node-ca/0.log" Apr 16 08:41:07.541537 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:07.541505 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-64d4d94569-zr5wc" event={"ID":"b86396a6-050a-4fc8-87a7-3185d62b983c","Type":"ContainerStarted","Data":"117b83dbe5c3c72c0bbb6ebddae03bfb721ea52c3dedc0b28a065e1408877846"} Apr 16 08:41:07.541537 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:07.541543 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-64d4d94569-zr5wc" event={"ID":"b86396a6-050a-4fc8-87a7-3185d62b983c","Type":"ContainerStarted","Data":"de5ccfd6104b4fc17bee8f1ac5a21e1d73a9638770b4969103f4222f9120516f"} Apr 16 08:41:07.555748 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:07.555698 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-64d4d94569-zr5wc" podStartSLOduration=1.6203214369999999 podStartE2EDuration="2.555681366s" podCreationTimestamp="2026-04-16 08:41:05 +0000 UTC" firstStartedPulling="2026-04-16 08:41:05.638266293 +0000 UTC m=+114.007250328" lastFinishedPulling="2026-04-16 08:41:06.57362622 +0000 UTC m=+114.942610257" observedRunningTime="2026-04-16 08:41:07.55532292 +0000 UTC m=+115.924306977" watchObservedRunningTime="2026-04-16 08:41:07.555681366 +0000 UTC m=+115.924665424" Apr 16 08:41:08.318165 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:08.318136 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8af06da5-de2c-48a0-8894-9fde08d3085a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6667474d89-qgz8h\" (UID: \"8af06da5-de2c-48a0-8894-9fde08d3085a\") " pod="openshift-monitoring/cluster-monitoring-operator-6667474d89-qgz8h" Apr 16 08:41:08.318360 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:08.318181 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f45d6895-a6e2-4531-ac1e-6e8c46c51012-metrics-certs\") pod \"router-default-54fdbc7b78-dxx97\" (UID: \"f45d6895-a6e2-4531-ac1e-6e8c46c51012\") " pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:08.318360 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:08.318254 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f45d6895-a6e2-4531-ac1e-6e8c46c51012-service-ca-bundle\") pod \"router-default-54fdbc7b78-dxx97\" (UID: \"f45d6895-a6e2-4531-ac1e-6e8c46c51012\") " pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:08.318360 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:08.318282 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Apr 16 08:41:08.318360 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:08.318328 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Apr 16 08:41:08.318360 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:08.318348 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8af06da5-de2c-48a0-8894-9fde08d3085a-cluster-monitoring-operator-tls podName:8af06da5-de2c-48a0-8894-9fde08d3085a nodeName:}" failed. No retries permitted until 2026-04-16 08:41:16.318332254 +0000 UTC m=+124.687316312 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8af06da5-de2c-48a0-8894-9fde08d3085a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6667474d89-qgz8h" (UID: "8af06da5-de2c-48a0-8894-9fde08d3085a") : secret "cluster-monitoring-operator-tls" not found Apr 16 08:41:08.318360 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:08.318361 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f45d6895-a6e2-4531-ac1e-6e8c46c51012-service-ca-bundle podName:f45d6895-a6e2-4531-ac1e-6e8c46c51012 nodeName:}" failed. No retries permitted until 2026-04-16 08:41:16.318356023 +0000 UTC m=+124.687340058 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f45d6895-a6e2-4531-ac1e-6e8c46c51012-service-ca-bundle") pod "router-default-54fdbc7b78-dxx97" (UID: "f45d6895-a6e2-4531-ac1e-6e8c46c51012") : configmap references non-existent config key: service-ca.crt Apr 16 08:41:08.318591 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:08.318377 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f45d6895-a6e2-4531-ac1e-6e8c46c51012-metrics-certs podName:f45d6895-a6e2-4531-ac1e-6e8c46c51012 nodeName:}" failed. No retries permitted until 2026-04-16 08:41:16.318366346 +0000 UTC m=+124.687350387 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f45d6895-a6e2-4531-ac1e-6e8c46c51012-metrics-certs") pod "router-default-54fdbc7b78-dxx97" (UID: "f45d6895-a6e2-4531-ac1e-6e8c46c51012") : secret "router-metrics-certs-default" not found Apr 16 08:41:16.378116 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:16.378082 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f45d6895-a6e2-4531-ac1e-6e8c46c51012-service-ca-bundle\") pod \"router-default-54fdbc7b78-dxx97\" (UID: \"f45d6895-a6e2-4531-ac1e-6e8c46c51012\") " pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:16.378574 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:16.378138 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8af06da5-de2c-48a0-8894-9fde08d3085a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6667474d89-qgz8h\" (UID: \"8af06da5-de2c-48a0-8894-9fde08d3085a\") " pod="openshift-monitoring/cluster-monitoring-operator-6667474d89-qgz8h" Apr 16 08:41:16.378574 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:16.378172 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f45d6895-a6e2-4531-ac1e-6e8c46c51012-metrics-certs\") pod \"router-default-54fdbc7b78-dxx97\" (UID: \"f45d6895-a6e2-4531-ac1e-6e8c46c51012\") " pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:16.378574 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:16.378302 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Apr 16 08:41:16.378574 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:16.378364 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8af06da5-de2c-48a0-8894-9fde08d3085a-cluster-monitoring-operator-tls podName:8af06da5-de2c-48a0-8894-9fde08d3085a nodeName:}" failed. No retries permitted until 2026-04-16 08:41:32.378349011 +0000 UTC m=+140.747333051 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8af06da5-de2c-48a0-8894-9fde08d3085a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6667474d89-qgz8h" (UID: "8af06da5-de2c-48a0-8894-9fde08d3085a") : secret "cluster-monitoring-operator-tls" not found Apr 16 08:41:16.378749 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:16.378660 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f45d6895-a6e2-4531-ac1e-6e8c46c51012-service-ca-bundle\") pod \"router-default-54fdbc7b78-dxx97\" (UID: \"f45d6895-a6e2-4531-ac1e-6e8c46c51012\") " pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:16.380493 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:16.380478 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f45d6895-a6e2-4531-ac1e-6e8c46c51012-metrics-certs\") pod \"router-default-54fdbc7b78-dxx97\" (UID: \"f45d6895-a6e2-4531-ac1e-6e8c46c51012\") " pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:16.460722 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:16.460698 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-ctdwt\"" Apr 16 08:41:16.469130 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:16.469113 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:16.587691 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:16.587664 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-54fdbc7b78-dxx97"] Apr 16 08:41:16.590661 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:41:16.590629 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf45d6895_a6e2_4531_ac1e_6e8c46c51012.slice/crio-e4399157a75137f592995a9977e3640be46724e02fceee76bd7361ccc4d7e8ab WatchSource:0}: Error finding container e4399157a75137f592995a9977e3640be46724e02fceee76bd7361ccc4d7e8ab: Status 404 returned error can't find the container with id e4399157a75137f592995a9977e3640be46724e02fceee76bd7361ccc4d7e8ab Apr 16 08:41:17.563411 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:17.563383 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-54fdbc7b78-dxx97" event={"ID":"f45d6895-a6e2-4531-ac1e-6e8c46c51012","Type":"ContainerStarted","Data":"908e1d945299de4b0f57a69cfd264b02267062f227e5ccfc0dd9bca4489d72a9"} Apr 16 08:41:17.563411 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:17.563416 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-54fdbc7b78-dxx97" event={"ID":"f45d6895-a6e2-4531-ac1e-6e8c46c51012","Type":"ContainerStarted","Data":"e4399157a75137f592995a9977e3640be46724e02fceee76bd7361ccc4d7e8ab"} Apr 16 08:41:17.578812 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:17.578749 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/router-default-54fdbc7b78-dxx97" podStartSLOduration=17.578733391 podStartE2EDuration="17.578733391s" podCreationTimestamp="2026-04-16 08:41:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 08:41:17.57840954 +0000 UTC m=+125.947393596" watchObservedRunningTime="2026-04-16 08:41:17.578733391 +0000 UTC m=+125.947717448" Apr 16 08:41:18.469835 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:18.469796 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:18.472230 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:18.472205 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:18.567818 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:18.567786 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:18.569006 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:18.568989 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-54fdbc7b78-dxx97" Apr 16 08:41:21.920820 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:21.920786 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57f278d8-41a1-4600-a089-04d841c7d446-metrics-certs\") pod \"network-metrics-daemon-fxtnv\" (UID: \"57f278d8-41a1-4600-a089-04d841c7d446\") " pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:41:21.923085 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:21.923065 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57f278d8-41a1-4600-a089-04d841c7d446-metrics-certs\") pod \"network-metrics-daemon-fxtnv\" (UID: \"57f278d8-41a1-4600-a089-04d841c7d446\") " pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:41:21.931761 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:21.931741 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9gvlt\"" Apr 16 08:41:21.940881 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:21.940867 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxtnv" Apr 16 08:41:22.048641 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:22.048611 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fxtnv"] Apr 16 08:41:22.052100 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:41:22.052070 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57f278d8_41a1_4600_a089_04d841c7d446.slice/crio-d2e0b125487faf4b25bdd7d9123d0985a2d89191d5a545cf5fd1b3aec845e78a WatchSource:0}: Error finding container d2e0b125487faf4b25bdd7d9123d0985a2d89191d5a545cf5fd1b3aec845e78a: Status 404 returned error can't find the container with id d2e0b125487faf4b25bdd7d9123d0985a2d89191d5a545cf5fd1b3aec845e78a Apr 16 08:41:22.578233 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:22.578199 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fxtnv" event={"ID":"57f278d8-41a1-4600-a089-04d841c7d446","Type":"ContainerStarted","Data":"d2e0b125487faf4b25bdd7d9123d0985a2d89191d5a545cf5fd1b3aec845e78a"} Apr 16 08:41:23.581675 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:23.581649 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fxtnv" event={"ID":"57f278d8-41a1-4600-a089-04d841c7d446","Type":"ContainerStarted","Data":"c8ceb270372e75ea887981a70eaeeca1a816be6db6fa44c056d80f15d38629b7"} Apr 16 08:41:23.582030 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:23.581682 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fxtnv" event={"ID":"57f278d8-41a1-4600-a089-04d841c7d446","Type":"ContainerStarted","Data":"b36a996ebf2894270b0cc043ce67cc6d8a81a272b0e983f29b3cb0d215a34602"} Apr 16 08:41:23.597583 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:23.597537 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fxtnv" podStartSLOduration=130.669335638 podStartE2EDuration="2m11.597522234s" podCreationTimestamp="2026-04-16 08:39:12 +0000 UTC" firstStartedPulling="2026-04-16 08:41:22.054062904 +0000 UTC m=+130.423046954" lastFinishedPulling="2026-04-16 08:41:22.982249507 +0000 UTC m=+131.351233550" observedRunningTime="2026-04-16 08:41:23.596583012 +0000 UTC m=+131.965567068" watchObservedRunningTime="2026-04-16 08:41:23.597522234 +0000 UTC m=+131.966506301" Apr 16 08:41:25.964341 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:25.964314 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rcsp2"] Apr 16 08:41:25.967406 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:25.967387 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rcsp2" Apr 16 08:41:25.970089 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:25.970071 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Apr 16 08:41:25.970199 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:25.970096 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Apr 16 08:41:25.970698 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:25.970676 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Apr 16 08:41:25.970819 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:25.970685 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Apr 16 08:41:25.970819 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:25.970795 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jjxz9\"" Apr 16 08:41:25.981572 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:25.981551 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rcsp2"] Apr 16 08:41:26.045860 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.045838 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f2fa2681-bd99-4a1a-b082-7c82172d5428-crio-socket\") pod \"insights-runtime-extractor-rcsp2\" (UID: \"f2fa2681-bd99-4a1a-b082-7c82172d5428\") " pod="openshift-insights/insights-runtime-extractor-rcsp2" Apr 16 08:41:26.045987 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.045889 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m25s\" (UniqueName: \"kubernetes.io/projected/f2fa2681-bd99-4a1a-b082-7c82172d5428-kube-api-access-9m25s\") pod \"insights-runtime-extractor-rcsp2\" (UID: \"f2fa2681-bd99-4a1a-b082-7c82172d5428\") " pod="openshift-insights/insights-runtime-extractor-rcsp2" Apr 16 08:41:26.045987 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.045922 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f2fa2681-bd99-4a1a-b082-7c82172d5428-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rcsp2\" (UID: \"f2fa2681-bd99-4a1a-b082-7c82172d5428\") " pod="openshift-insights/insights-runtime-extractor-rcsp2" Apr 16 08:41:26.045987 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.045945 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f2fa2681-bd99-4a1a-b082-7c82172d5428-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rcsp2\" (UID: \"f2fa2681-bd99-4a1a-b082-7c82172d5428\") " pod="openshift-insights/insights-runtime-extractor-rcsp2" Apr 16 08:41:26.046117 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.046006 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2fa2681-bd99-4a1a-b082-7c82172d5428-data-volume\") pod \"insights-runtime-extractor-rcsp2\" (UID: \"f2fa2681-bd99-4a1a-b082-7c82172d5428\") " pod="openshift-insights/insights-runtime-extractor-rcsp2" Apr 16 08:41:26.071506 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.071482 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5cb6cf4cb4-s5sxt"] Apr 16 08:41:26.074461 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.074445 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5cb6cf4cb4-s5sxt" Apr 16 08:41:26.076681 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.076663 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Apr 16 08:41:26.077117 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.077098 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-sn2fw\"" Apr 16 08:41:26.077221 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.077184 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Apr 16 08:41:26.087658 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.087637 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5cb6cf4cb4-s5sxt"] Apr 16 08:41:26.147306 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.147279 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9m25s\" (UniqueName: \"kubernetes.io/projected/f2fa2681-bd99-4a1a-b082-7c82172d5428-kube-api-access-9m25s\") pod \"insights-runtime-extractor-rcsp2\" (UID: \"f2fa2681-bd99-4a1a-b082-7c82172d5428\") " pod="openshift-insights/insights-runtime-extractor-rcsp2" Apr 16 08:41:26.147420 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.147309 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f2fa2681-bd99-4a1a-b082-7c82172d5428-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rcsp2\" (UID: \"f2fa2681-bd99-4a1a-b082-7c82172d5428\") " pod="openshift-insights/insights-runtime-extractor-rcsp2" Apr 16 08:41:26.147420 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.147326 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f2fa2681-bd99-4a1a-b082-7c82172d5428-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rcsp2\" (UID: \"f2fa2681-bd99-4a1a-b082-7c82172d5428\") " pod="openshift-insights/insights-runtime-extractor-rcsp2" Apr 16 08:41:26.147524 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.147473 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2fa2681-bd99-4a1a-b082-7c82172d5428-data-volume\") pod \"insights-runtime-extractor-rcsp2\" (UID: \"f2fa2681-bd99-4a1a-b082-7c82172d5428\") " pod="openshift-insights/insights-runtime-extractor-rcsp2" Apr 16 08:41:26.147572 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.147534 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f2fa2681-bd99-4a1a-b082-7c82172d5428-crio-socket\") pod \"insights-runtime-extractor-rcsp2\" (UID: \"f2fa2681-bd99-4a1a-b082-7c82172d5428\") " pod="openshift-insights/insights-runtime-extractor-rcsp2" Apr 16 08:41:26.147618 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.147611 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f2fa2681-bd99-4a1a-b082-7c82172d5428-crio-socket\") pod \"insights-runtime-extractor-rcsp2\" (UID: \"f2fa2681-bd99-4a1a-b082-7c82172d5428\") " pod="openshift-insights/insights-runtime-extractor-rcsp2" Apr 16 08:41:26.147799 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.147782 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f2fa2681-bd99-4a1a-b082-7c82172d5428-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rcsp2\" (UID: \"f2fa2681-bd99-4a1a-b082-7c82172d5428\") " pod="openshift-insights/insights-runtime-extractor-rcsp2" Apr 16 08:41:26.147898 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.147883 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2fa2681-bd99-4a1a-b082-7c82172d5428-data-volume\") pod \"insights-runtime-extractor-rcsp2\" (UID: \"f2fa2681-bd99-4a1a-b082-7c82172d5428\") " pod="openshift-insights/insights-runtime-extractor-rcsp2" Apr 16 08:41:26.149759 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.149737 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f2fa2681-bd99-4a1a-b082-7c82172d5428-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rcsp2\" (UID: \"f2fa2681-bd99-4a1a-b082-7c82172d5428\") " pod="openshift-insights/insights-runtime-extractor-rcsp2" Apr 16 08:41:26.158980 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.158962 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m25s\" (UniqueName: \"kubernetes.io/projected/f2fa2681-bd99-4a1a-b082-7c82172d5428-kube-api-access-9m25s\") pod \"insights-runtime-extractor-rcsp2\" (UID: \"f2fa2681-bd99-4a1a-b082-7c82172d5428\") " pod="openshift-insights/insights-runtime-extractor-rcsp2" Apr 16 08:41:26.248213 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.248159 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/66b6340c-4dc1-4115-8563-ad602e61e806-networking-console-plugin-cert\") pod \"networking-console-plugin-5cb6cf4cb4-s5sxt\" (UID: \"66b6340c-4dc1-4115-8563-ad602e61e806\") " pod="openshift-network-console/networking-console-plugin-5cb6cf4cb4-s5sxt" Apr 16 08:41:26.248213 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.248197 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/66b6340c-4dc1-4115-8563-ad602e61e806-nginx-conf\") pod \"networking-console-plugin-5cb6cf4cb4-s5sxt\" (UID: \"66b6340c-4dc1-4115-8563-ad602e61e806\") " pod="openshift-network-console/networking-console-plugin-5cb6cf4cb4-s5sxt" Apr 16 08:41:26.275828 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.275811 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rcsp2" Apr 16 08:41:26.348726 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.348701 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/66b6340c-4dc1-4115-8563-ad602e61e806-networking-console-plugin-cert\") pod \"networking-console-plugin-5cb6cf4cb4-s5sxt\" (UID: \"66b6340c-4dc1-4115-8563-ad602e61e806\") " pod="openshift-network-console/networking-console-plugin-5cb6cf4cb4-s5sxt" Apr 16 08:41:26.348868 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.348746 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/66b6340c-4dc1-4115-8563-ad602e61e806-nginx-conf\") pod \"networking-console-plugin-5cb6cf4cb4-s5sxt\" (UID: \"66b6340c-4dc1-4115-8563-ad602e61e806\") " pod="openshift-network-console/networking-console-plugin-5cb6cf4cb4-s5sxt" Apr 16 08:41:26.351006 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.350987 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/66b6340c-4dc1-4115-8563-ad602e61e806-nginx-conf\") pod \"networking-console-plugin-5cb6cf4cb4-s5sxt\" (UID: \"66b6340c-4dc1-4115-8563-ad602e61e806\") " pod="openshift-network-console/networking-console-plugin-5cb6cf4cb4-s5sxt" Apr 16 08:41:26.352654 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.352620 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/66b6340c-4dc1-4115-8563-ad602e61e806-networking-console-plugin-cert\") pod \"networking-console-plugin-5cb6cf4cb4-s5sxt\" (UID: \"66b6340c-4dc1-4115-8563-ad602e61e806\") " pod="openshift-network-console/networking-console-plugin-5cb6cf4cb4-s5sxt" Apr 16 08:41:26.384931 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.384907 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5cb6cf4cb4-s5sxt" Apr 16 08:41:26.397001 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.396976 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rcsp2"] Apr 16 08:41:26.401460 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:41:26.401432 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2fa2681_bd99_4a1a_b082_7c82172d5428.slice/crio-de7570e4c2ff566a1ec248c62318bc50593e4c90f8b6de7ffa5287a90ebc5338 WatchSource:0}: Error finding container de7570e4c2ff566a1ec248c62318bc50593e4c90f8b6de7ffa5287a90ebc5338: Status 404 returned error can't find the container with id de7570e4c2ff566a1ec248c62318bc50593e4c90f8b6de7ffa5287a90ebc5338 Apr 16 08:41:26.504946 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.504744 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5cb6cf4cb4-s5sxt"] Apr 16 08:41:26.507038 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:41:26.507011 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66b6340c_4dc1_4115_8563_ad602e61e806.slice/crio-29a3b1cfb2767faf138c451346fa8abaebe81025be9ff3568136a3c7da4811aa WatchSource:0}: Error finding container 29a3b1cfb2767faf138c451346fa8abaebe81025be9ff3568136a3c7da4811aa: Status 404 returned error can't find the container with id 29a3b1cfb2767faf138c451346fa8abaebe81025be9ff3568136a3c7da4811aa Apr 16 08:41:26.589908 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.589877 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5cb6cf4cb4-s5sxt" event={"ID":"66b6340c-4dc1-4115-8563-ad602e61e806","Type":"ContainerStarted","Data":"29a3b1cfb2767faf138c451346fa8abaebe81025be9ff3568136a3c7da4811aa"} Apr 16 08:41:26.591136 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.591117 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rcsp2" event={"ID":"f2fa2681-bd99-4a1a-b082-7c82172d5428","Type":"ContainerStarted","Data":"63b6774aa0f90e24358ecd0b486f05279d6c051163784905fd3380e109421e49"} Apr 16 08:41:26.591210 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:26.591141 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rcsp2" event={"ID":"f2fa2681-bd99-4a1a-b082-7c82172d5428","Type":"ContainerStarted","Data":"de7570e4c2ff566a1ec248c62318bc50593e4c90f8b6de7ffa5287a90ebc5338"} Apr 16 08:41:27.595432 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:27.595396 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rcsp2" event={"ID":"f2fa2681-bd99-4a1a-b082-7c82172d5428","Type":"ContainerStarted","Data":"137397eb96b77e651d6e17f045a88662272df870b617aed39bf3edc3e3e0ef88"} Apr 16 08:41:28.601597 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:28.601568 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5cb6cf4cb4-s5sxt" event={"ID":"66b6340c-4dc1-4115-8563-ad602e61e806","Type":"ContainerStarted","Data":"3ee3d7a98ed339f13223f1e45ad7819d9214fc64b631543680633a8f42e8b17d"} Apr 16 08:41:28.603327 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:28.603307 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rcsp2" event={"ID":"f2fa2681-bd99-4a1a-b082-7c82172d5428","Type":"ContainerStarted","Data":"b3e60165fc3fea1455c76d89f1f19b5df145c08c233c1342f48af745d9e34921"} Apr 16 08:41:28.615227 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:28.615165 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5cb6cf4cb4-s5sxt" podStartSLOduration=1.519733588 podStartE2EDuration="2.615150528s" podCreationTimestamp="2026-04-16 08:41:26 +0000 UTC" firstStartedPulling="2026-04-16 08:41:26.508736425 +0000 UTC m=+134.877720464" lastFinishedPulling="2026-04-16 08:41:27.604153368 +0000 UTC m=+135.973137404" observedRunningTime="2026-04-16 08:41:28.613841216 +0000 UTC m=+136.982825273" watchObservedRunningTime="2026-04-16 08:41:28.615150528 +0000 UTC m=+136.984134740" Apr 16 08:41:28.631143 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:28.631104 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rcsp2" podStartSLOduration=1.5986862290000001 podStartE2EDuration="3.631091942s" podCreationTimestamp="2026-04-16 08:41:25 +0000 UTC" firstStartedPulling="2026-04-16 08:41:26.463403596 +0000 UTC m=+134.832387631" lastFinishedPulling="2026-04-16 08:41:28.495809309 +0000 UTC m=+136.864793344" observedRunningTime="2026-04-16 08:41:28.630601322 +0000 UTC m=+136.999585378" watchObservedRunningTime="2026-04-16 08:41:28.631091942 +0000 UTC m=+137.000075998" Apr 16 08:41:32.392707 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:32.392666 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8af06da5-de2c-48a0-8894-9fde08d3085a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6667474d89-qgz8h\" (UID: \"8af06da5-de2c-48a0-8894-9fde08d3085a\") " pod="openshift-monitoring/cluster-monitoring-operator-6667474d89-qgz8h" Apr 16 08:41:32.395148 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:32.395127 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8af06da5-de2c-48a0-8894-9fde08d3085a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6667474d89-qgz8h\" (UID: \"8af06da5-de2c-48a0-8894-9fde08d3085a\") " pod="openshift-monitoring/cluster-monitoring-operator-6667474d89-qgz8h" Apr 16 08:41:32.654142 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:32.654062 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-f4vlc\"" Apr 16 08:41:32.662642 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:32.662625 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6667474d89-qgz8h" Apr 16 08:41:32.776353 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:32.776326 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6667474d89-qgz8h"] Apr 16 08:41:32.779739 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:41:32.779708 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8af06da5_de2c_48a0_8894_9fde08d3085a.slice/crio-70a09a3622b5c5f89fed1035b4ede68b9f557e3ff0a9d11ebac48384a6d589b6 WatchSource:0}: Error finding container 70a09a3622b5c5f89fed1035b4ede68b9f557e3ff0a9d11ebac48384a6d589b6: Status 404 returned error can't find the container with id 70a09a3622b5c5f89fed1035b4ede68b9f557e3ff0a9d11ebac48384a6d589b6 Apr 16 08:41:33.617965 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:33.617924 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6667474d89-qgz8h" event={"ID":"8af06da5-de2c-48a0-8894-9fde08d3085a","Type":"ContainerStarted","Data":"70a09a3622b5c5f89fed1035b4ede68b9f557e3ff0a9d11ebac48384a6d589b6"} Apr 16 08:41:34.621713 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:34.621630 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6667474d89-qgz8h" event={"ID":"8af06da5-de2c-48a0-8894-9fde08d3085a","Type":"ContainerStarted","Data":"394511589c326add604e91e3a8520b3d2623663dab3028f7e4f3fabba2c5dedb"} Apr 16 08:41:34.638304 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:34.638249 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6667474d89-qgz8h" podStartSLOduration=33.142456154 podStartE2EDuration="34.638231854s" podCreationTimestamp="2026-04-16 08:41:00 +0000 UTC" firstStartedPulling="2026-04-16 08:41:32.781906564 +0000 UTC m=+141.150890599" lastFinishedPulling="2026-04-16 08:41:34.277682264 +0000 UTC m=+142.646666299" observedRunningTime="2026-04-16 08:41:34.637458063 +0000 UTC m=+143.006442119" watchObservedRunningTime="2026-04-16 08:41:34.638231854 +0000 UTC m=+143.007215913" Apr 16 08:41:34.762646 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:34.762614 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-9cb97cd87-jldxt"] Apr 16 08:41:34.765566 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:34.765552 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-9cb97cd87-jldxt" Apr 16 08:41:34.767461 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:34.767440 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Apr 16 08:41:34.767548 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:34.767467 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-2m2mb\"" Apr 16 08:41:34.773125 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:34.773091 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-9cb97cd87-jldxt"] Apr 16 08:41:34.912207 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:34.912147 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5b0fe24d-1c68-4a6a-ae1a-ae2f6da5b957-tls-certificates\") pod \"prometheus-operator-admission-webhook-9cb97cd87-jldxt\" (UID: \"5b0fe24d-1c68-4a6a-ae1a-ae2f6da5b957\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-9cb97cd87-jldxt" Apr 16 08:41:35.012649 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:35.012620 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5b0fe24d-1c68-4a6a-ae1a-ae2f6da5b957-tls-certificates\") pod \"prometheus-operator-admission-webhook-9cb97cd87-jldxt\" (UID: \"5b0fe24d-1c68-4a6a-ae1a-ae2f6da5b957\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-9cb97cd87-jldxt" Apr 16 08:41:35.014944 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:35.014925 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5b0fe24d-1c68-4a6a-ae1a-ae2f6da5b957-tls-certificates\") pod \"prometheus-operator-admission-webhook-9cb97cd87-jldxt\" (UID: \"5b0fe24d-1c68-4a6a-ae1a-ae2f6da5b957\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-9cb97cd87-jldxt" Apr 16 08:41:35.074430 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:35.074410 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-9cb97cd87-jldxt" Apr 16 08:41:35.205964 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:35.205938 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-9cb97cd87-jldxt"] Apr 16 08:41:35.209351 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:41:35.209322 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b0fe24d_1c68_4a6a_ae1a_ae2f6da5b957.slice/crio-ee0fa606e092ffd891a68acb648dce64594c8aec6180af51870f70afee1a6be2 WatchSource:0}: Error finding container ee0fa606e092ffd891a68acb648dce64594c8aec6180af51870f70afee1a6be2: Status 404 returned error can't find the container with id ee0fa606e092ffd891a68acb648dce64594c8aec6180af51870f70afee1a6be2 Apr 16 08:41:35.625026 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:35.624994 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-9cb97cd87-jldxt" event={"ID":"5b0fe24d-1c68-4a6a-ae1a-ae2f6da5b957","Type":"ContainerStarted","Data":"ee0fa606e092ffd891a68acb648dce64594c8aec6180af51870f70afee1a6be2"} Apr 16 08:41:35.825539 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:35.825506 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-79c5cdd649-xthk8"] Apr 16 08:41:35.831079 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:35.831049 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79c5cdd649-xthk8" Apr 16 08:41:35.833854 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:35.833827 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-w94jt\"" Apr 16 08:41:35.833977 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:35.833862 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Apr 16 08:41:35.833977 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:35.833828 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Apr 16 08:41:35.833977 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:35.833828 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Apr 16 08:41:35.834132 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:35.834109 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Apr 16 08:41:35.834182 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:35.834159 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Apr 16 08:41:35.834182 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:35.834112 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Apr 16 08:41:35.834559 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:35.834541 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Apr 16 08:41:35.837020 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:35.836997 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79c5cdd649-xthk8"] Apr 16 08:41:35.840025 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:35.840007 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Apr 16 08:41:35.920957 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:35.920895 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5rl7\" (UniqueName: \"kubernetes.io/projected/19866fa8-b487-41d7-9eba-73fe93fcb783-kube-api-access-z5rl7\") pod \"console-79c5cdd649-xthk8\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " pod="openshift-console/console-79c5cdd649-xthk8" Apr 16 08:41:35.920957 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:35.920940 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19866fa8-b487-41d7-9eba-73fe93fcb783-oauth-serving-cert\") pod \"console-79c5cdd649-xthk8\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " pod="openshift-console/console-79c5cdd649-xthk8" Apr 16 08:41:35.921156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:35.920976 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19866fa8-b487-41d7-9eba-73fe93fcb783-console-config\") pod \"console-79c5cdd649-xthk8\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " pod="openshift-console/console-79c5cdd649-xthk8" Apr 16 08:41:35.921156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:35.921080 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19866fa8-b487-41d7-9eba-73fe93fcb783-service-ca\") pod \"console-79c5cdd649-xthk8\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " pod="openshift-console/console-79c5cdd649-xthk8" Apr 16 08:41:35.921156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:35.921117 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19866fa8-b487-41d7-9eba-73fe93fcb783-trusted-ca-bundle\") pod \"console-79c5cdd649-xthk8\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " pod="openshift-console/console-79c5cdd649-xthk8" Apr 16 08:41:35.921156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:35.921153 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19866fa8-b487-41d7-9eba-73fe93fcb783-console-oauth-config\") pod \"console-79c5cdd649-xthk8\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " pod="openshift-console/console-79c5cdd649-xthk8" Apr 16 08:41:35.921314 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:35.921183 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19866fa8-b487-41d7-9eba-73fe93fcb783-console-serving-cert\") pod \"console-79c5cdd649-xthk8\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " pod="openshift-console/console-79c5cdd649-xthk8" Apr 16 08:41:36.022249 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.022219 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19866fa8-b487-41d7-9eba-73fe93fcb783-console-oauth-config\") pod \"console-79c5cdd649-xthk8\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " pod="openshift-console/console-79c5cdd649-xthk8" Apr 16 08:41:36.022394 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.022280 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19866fa8-b487-41d7-9eba-73fe93fcb783-console-serving-cert\") pod \"console-79c5cdd649-xthk8\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " pod="openshift-console/console-79c5cdd649-xthk8" Apr 16 08:41:36.022394 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.022327 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z5rl7\" (UniqueName: \"kubernetes.io/projected/19866fa8-b487-41d7-9eba-73fe93fcb783-kube-api-access-z5rl7\") pod \"console-79c5cdd649-xthk8\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " pod="openshift-console/console-79c5cdd649-xthk8" Apr 16 08:41:36.022394 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.022353 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19866fa8-b487-41d7-9eba-73fe93fcb783-oauth-serving-cert\") pod \"console-79c5cdd649-xthk8\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " pod="openshift-console/console-79c5cdd649-xthk8" Apr 16 08:41:36.022777 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.022659 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19866fa8-b487-41d7-9eba-73fe93fcb783-console-config\") pod \"console-79c5cdd649-xthk8\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " pod="openshift-console/console-79c5cdd649-xthk8" Apr 16 08:41:36.022777 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.022738 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19866fa8-b487-41d7-9eba-73fe93fcb783-service-ca\") pod \"console-79c5cdd649-xthk8\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " pod="openshift-console/console-79c5cdd649-xthk8" Apr 16 08:41:36.022950 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.022813 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19866fa8-b487-41d7-9eba-73fe93fcb783-trusted-ca-bundle\") pod \"console-79c5cdd649-xthk8\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " pod="openshift-console/console-79c5cdd649-xthk8" Apr 16 08:41:36.023136 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.023113 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19866fa8-b487-41d7-9eba-73fe93fcb783-oauth-serving-cert\") pod \"console-79c5cdd649-xthk8\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " pod="openshift-console/console-79c5cdd649-xthk8" Apr 16 08:41:36.023301 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.023284 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19866fa8-b487-41d7-9eba-73fe93fcb783-console-config\") pod \"console-79c5cdd649-xthk8\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " pod="openshift-console/console-79c5cdd649-xthk8" Apr 16 08:41:36.023585 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.023563 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19866fa8-b487-41d7-9eba-73fe93fcb783-service-ca\") pod \"console-79c5cdd649-xthk8\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " pod="openshift-console/console-79c5cdd649-xthk8" Apr 16 08:41:36.023694 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.023644 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19866fa8-b487-41d7-9eba-73fe93fcb783-trusted-ca-bundle\") pod \"console-79c5cdd649-xthk8\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " pod="openshift-console/console-79c5cdd649-xthk8" Apr 16 08:41:36.025263 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.025240 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19866fa8-b487-41d7-9eba-73fe93fcb783-console-serving-cert\") pod \"console-79c5cdd649-xthk8\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " pod="openshift-console/console-79c5cdd649-xthk8" Apr 16 08:41:36.025340 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.025303 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19866fa8-b487-41d7-9eba-73fe93fcb783-console-oauth-config\") pod \"console-79c5cdd649-xthk8\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " pod="openshift-console/console-79c5cdd649-xthk8" Apr 16 08:41:36.030393 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.030373 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5rl7\" (UniqueName: \"kubernetes.io/projected/19866fa8-b487-41d7-9eba-73fe93fcb783-kube-api-access-z5rl7\") pod \"console-79c5cdd649-xthk8\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " pod="openshift-console/console-79c5cdd649-xthk8" Apr 16 08:41:36.142644 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.142615 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79c5cdd649-xthk8" Apr 16 08:41:36.281711 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.281679 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79c5cdd649-xthk8"] Apr 16 08:41:36.285229 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:41:36.285203 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19866fa8_b487_41d7_9eba_73fe93fcb783.slice/crio-eabe3e9341b6c62416316bd1b27ee8c073437f2a9859ea8c0f2ec419722a915b WatchSource:0}: Error finding container eabe3e9341b6c62416316bd1b27ee8c073437f2a9859ea8c0f2ec419722a915b: Status 404 returned error can't find the container with id eabe3e9341b6c62416316bd1b27ee8c073437f2a9859ea8c0f2ec419722a915b Apr 16 08:41:36.629062 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.629036 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-9cb97cd87-jldxt" event={"ID":"5b0fe24d-1c68-4a6a-ae1a-ae2f6da5b957","Type":"ContainerStarted","Data":"c410dd24040c72b74101b4fb8443f7069e22666f922921f1ffc62f2053348db1"} Apr 16 08:41:36.629424 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.629253 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-9cb97cd87-jldxt" Apr 16 08:41:36.630202 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.630180 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79c5cdd649-xthk8" event={"ID":"19866fa8-b487-41d7-9eba-73fe93fcb783","Type":"ContainerStarted","Data":"eabe3e9341b6c62416316bd1b27ee8c073437f2a9859ea8c0f2ec419722a915b"} Apr 16 08:41:36.633715 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.633696 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-9cb97cd87-jldxt" Apr 16 08:41:36.643926 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.643886 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-9cb97cd87-jldxt" podStartSLOduration=1.6868472890000001 podStartE2EDuration="2.643870709s" podCreationTimestamp="2026-04-16 08:41:34 +0000 UTC" firstStartedPulling="2026-04-16 08:41:35.211670827 +0000 UTC m=+143.580654869" lastFinishedPulling="2026-04-16 08:41:36.168694232 +0000 UTC m=+144.537678289" observedRunningTime="2026-04-16 08:41:36.642995508 +0000 UTC m=+145.011979567" watchObservedRunningTime="2026-04-16 08:41:36.643870709 +0000 UTC m=+145.012854769" Apr 16 08:41:36.825532 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.825507 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-78f957474d-n59rr"] Apr 16 08:41:36.828574 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.828550 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-78f957474d-n59rr" Apr 16 08:41:36.830863 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.830840 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-mbgts\"" Apr 16 08:41:36.830987 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.830839 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Apr 16 08:41:36.831045 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.830859 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Apr 16 08:41:36.831045 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.830880 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Apr 16 08:41:36.838811 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.838668 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-78f957474d-n59rr"] Apr 16 08:41:36.928862 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.928788 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8516f8a5-573f-41fb-bc6c-7e4880f69b09-prometheus-operator-tls\") pod \"prometheus-operator-78f957474d-n59rr\" (UID: \"8516f8a5-573f-41fb-bc6c-7e4880f69b09\") " pod="openshift-monitoring/prometheus-operator-78f957474d-n59rr" Apr 16 08:41:36.929002 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.928861 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8516f8a5-573f-41fb-bc6c-7e4880f69b09-metrics-client-ca\") pod \"prometheus-operator-78f957474d-n59rr\" (UID: \"8516f8a5-573f-41fb-bc6c-7e4880f69b09\") " pod="openshift-monitoring/prometheus-operator-78f957474d-n59rr" Apr 16 08:41:36.929002 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.928898 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8516f8a5-573f-41fb-bc6c-7e4880f69b09-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-78f957474d-n59rr\" (UID: \"8516f8a5-573f-41fb-bc6c-7e4880f69b09\") " pod="openshift-monitoring/prometheus-operator-78f957474d-n59rr" Apr 16 08:41:36.929002 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:36.928992 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxnls\" (UniqueName: \"kubernetes.io/projected/8516f8a5-573f-41fb-bc6c-7e4880f69b09-kube-api-access-mxnls\") pod \"prometheus-operator-78f957474d-n59rr\" (UID: \"8516f8a5-573f-41fb-bc6c-7e4880f69b09\") " pod="openshift-monitoring/prometheus-operator-78f957474d-n59rr" Apr 16 08:41:37.030261 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:37.030228 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mxnls\" (UniqueName: \"kubernetes.io/projected/8516f8a5-573f-41fb-bc6c-7e4880f69b09-kube-api-access-mxnls\") pod \"prometheus-operator-78f957474d-n59rr\" (UID: \"8516f8a5-573f-41fb-bc6c-7e4880f69b09\") " pod="openshift-monitoring/prometheus-operator-78f957474d-n59rr" Apr 16 08:41:37.030420 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:37.030295 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8516f8a5-573f-41fb-bc6c-7e4880f69b09-prometheus-operator-tls\") pod \"prometheus-operator-78f957474d-n59rr\" (UID: \"8516f8a5-573f-41fb-bc6c-7e4880f69b09\") " pod="openshift-monitoring/prometheus-operator-78f957474d-n59rr" Apr 16 08:41:37.030420 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:37.030349 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8516f8a5-573f-41fb-bc6c-7e4880f69b09-metrics-client-ca\") pod \"prometheus-operator-78f957474d-n59rr\" (UID: \"8516f8a5-573f-41fb-bc6c-7e4880f69b09\") " pod="openshift-monitoring/prometheus-operator-78f957474d-n59rr" Apr 16 08:41:37.030420 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:37.030380 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8516f8a5-573f-41fb-bc6c-7e4880f69b09-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-78f957474d-n59rr\" (UID: \"8516f8a5-573f-41fb-bc6c-7e4880f69b09\") " pod="openshift-monitoring/prometheus-operator-78f957474d-n59rr" Apr 16 08:41:37.031218 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:37.031164 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8516f8a5-573f-41fb-bc6c-7e4880f69b09-metrics-client-ca\") pod \"prometheus-operator-78f957474d-n59rr\" (UID: \"8516f8a5-573f-41fb-bc6c-7e4880f69b09\") " pod="openshift-monitoring/prometheus-operator-78f957474d-n59rr" Apr 16 08:41:37.033182 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:37.033155 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8516f8a5-573f-41fb-bc6c-7e4880f69b09-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-78f957474d-n59rr\" (UID: \"8516f8a5-573f-41fb-bc6c-7e4880f69b09\") " pod="openshift-monitoring/prometheus-operator-78f957474d-n59rr" Apr 16 08:41:37.033590 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:37.033572 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8516f8a5-573f-41fb-bc6c-7e4880f69b09-prometheus-operator-tls\") pod \"prometheus-operator-78f957474d-n59rr\" (UID: \"8516f8a5-573f-41fb-bc6c-7e4880f69b09\") " pod="openshift-monitoring/prometheus-operator-78f957474d-n59rr" Apr 16 08:41:37.037881 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:37.037857 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxnls\" (UniqueName: \"kubernetes.io/projected/8516f8a5-573f-41fb-bc6c-7e4880f69b09-kube-api-access-mxnls\") pod \"prometheus-operator-78f957474d-n59rr\" (UID: \"8516f8a5-573f-41fb-bc6c-7e4880f69b09\") " pod="openshift-monitoring/prometheus-operator-78f957474d-n59rr" Apr 16 08:41:37.139799 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:37.139747 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-78f957474d-n59rr" Apr 16 08:41:37.274790 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:37.274721 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-78f957474d-n59rr"] Apr 16 08:41:37.278061 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:41:37.278032 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8516f8a5_573f_41fb_bc6c_7e4880f69b09.slice/crio-e01368220d667f092c0063a1e360fef086c7c0c1d14524b1f0ea9f315d16da34 WatchSource:0}: Error finding container e01368220d667f092c0063a1e360fef086c7c0c1d14524b1f0ea9f315d16da34: Status 404 returned error can't find the container with id e01368220d667f092c0063a1e360fef086c7c0c1d14524b1f0ea9f315d16da34 Apr 16 08:41:37.634019 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:37.633980 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-78f957474d-n59rr" event={"ID":"8516f8a5-573f-41fb-bc6c-7e4880f69b09","Type":"ContainerStarted","Data":"e01368220d667f092c0063a1e360fef086c7c0c1d14524b1f0ea9f315d16da34"} Apr 16 08:41:39.640544 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:39.640518 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79c5cdd649-xthk8" event={"ID":"19866fa8-b487-41d7-9eba-73fe93fcb783","Type":"ContainerStarted","Data":"301e891f71f668942ad556a3a9df4f9d5cab75e59fa8f84554425dab02712fce"} Apr 16 08:41:39.657676 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:39.657621 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-79c5cdd649-xthk8" podStartSLOduration=2.149974108 podStartE2EDuration="4.657602843s" podCreationTimestamp="2026-04-16 08:41:35 +0000 UTC" firstStartedPulling="2026-04-16 08:41:36.287092638 +0000 UTC m=+144.656076693" lastFinishedPulling="2026-04-16 08:41:38.79472139 +0000 UTC m=+147.163705428" observedRunningTime="2026-04-16 08:41:39.656836071 +0000 UTC m=+148.025820128" watchObservedRunningTime="2026-04-16 08:41:39.657602843 +0000 UTC m=+148.026586901" Apr 16 08:41:40.644685 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:40.644651 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-78f957474d-n59rr" event={"ID":"8516f8a5-573f-41fb-bc6c-7e4880f69b09","Type":"ContainerStarted","Data":"7aef9a859cc99e4d49808baef0628cea5b77000a5e1cf644ef76ddfe64481ed6"} Apr 16 08:41:40.644685 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:40.644685 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-78f957474d-n59rr" event={"ID":"8516f8a5-573f-41fb-bc6c-7e4880f69b09","Type":"ContainerStarted","Data":"a0c84d18aec16be408f1854b96e7cbb977197d5f5116702fcb279e58f77ded5c"} Apr 16 08:41:40.662584 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:40.662541 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-78f957474d-n59rr" podStartSLOduration=2.301486052 podStartE2EDuration="4.662529781s" podCreationTimestamp="2026-04-16 08:41:36 +0000 UTC" firstStartedPulling="2026-04-16 08:41:37.280029544 +0000 UTC m=+145.649013579" lastFinishedPulling="2026-04-16 08:41:39.641073273 +0000 UTC m=+148.010057308" observedRunningTime="2026-04-16 08:41:40.661340788 +0000 UTC m=+149.030324845" watchObservedRunningTime="2026-04-16 08:41:40.662529781 +0000 UTC m=+149.031513837" Apr 16 08:41:42.151113 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.151082 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-5669946b84-tgwn7"] Apr 16 08:41:42.154363 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.154349 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-5669946b84-tgwn7" Apr 16 08:41:42.156376 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.156356 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Apr 16 08:41:42.156490 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.156399 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Apr 16 08:41:42.156678 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.156661 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-hc2bv\"" Apr 16 08:41:42.164751 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.164732 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-5669946b84-tgwn7"] Apr 16 08:41:42.179285 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.179262 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7479c89684-sbzrt"] Apr 16 08:41:42.182519 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.182494 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7479c89684-sbzrt" Apr 16 08:41:42.184436 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.184421 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-n8mbc\"" Apr 16 08:41:42.184681 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.184661 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Apr 16 08:41:42.184826 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.184805 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Apr 16 08:41:42.184903 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.184884 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Apr 16 08:41:42.186878 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.186858 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-z79cw"] Apr 16 08:41:42.189849 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.189822 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.191718 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.191698 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-lc2hb\"" Apr 16 08:41:42.191830 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.191699 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Apr 16 08:41:42.191830 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.191723 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Apr 16 08:41:42.191922 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.191881 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Apr 16 08:41:42.193706 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.193685 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7479c89684-sbzrt"] Apr 16 08:41:42.268508 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.268480 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q886l\" (UniqueName: \"kubernetes.io/projected/170f730a-ba17-492c-94d6-cc2f6c0d9177-kube-api-access-q886l\") pod \"openshift-state-metrics-5669946b84-tgwn7\" (UID: \"170f730a-ba17-492c-94d6-cc2f6c0d9177\") " pod="openshift-monitoring/openshift-state-metrics-5669946b84-tgwn7" Apr 16 08:41:42.268646 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.268520 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/170f730a-ba17-492c-94d6-cc2f6c0d9177-metrics-client-ca\") pod \"openshift-state-metrics-5669946b84-tgwn7\" (UID: \"170f730a-ba17-492c-94d6-cc2f6c0d9177\") " pod="openshift-monitoring/openshift-state-metrics-5669946b84-tgwn7" Apr 16 08:41:42.268646 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.268586 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/170f730a-ba17-492c-94d6-cc2f6c0d9177-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-5669946b84-tgwn7\" (UID: \"170f730a-ba17-492c-94d6-cc2f6c0d9177\") " pod="openshift-monitoring/openshift-state-metrics-5669946b84-tgwn7" Apr 16 08:41:42.268646 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.268620 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/170f730a-ba17-492c-94d6-cc2f6c0d9177-openshift-state-metrics-tls\") pod \"openshift-state-metrics-5669946b84-tgwn7\" (UID: \"170f730a-ba17-492c-94d6-cc2f6c0d9177\") " pod="openshift-monitoring/openshift-state-metrics-5669946b84-tgwn7" Apr 16 08:41:42.368970 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.368940 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q886l\" (UniqueName: \"kubernetes.io/projected/170f730a-ba17-492c-94d6-cc2f6c0d9177-kube-api-access-q886l\") pod \"openshift-state-metrics-5669946b84-tgwn7\" (UID: \"170f730a-ba17-492c-94d6-cc2f6c0d9177\") " pod="openshift-monitoring/openshift-state-metrics-5669946b84-tgwn7" Apr 16 08:41:42.369123 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.368982 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/170f730a-ba17-492c-94d6-cc2f6c0d9177-metrics-client-ca\") pod \"openshift-state-metrics-5669946b84-tgwn7\" (UID: \"170f730a-ba17-492c-94d6-cc2f6c0d9177\") " pod="openshift-monitoring/openshift-state-metrics-5669946b84-tgwn7" Apr 16 08:41:42.369123 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.369029 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/170f730a-ba17-492c-94d6-cc2f6c0d9177-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-5669946b84-tgwn7\" (UID: \"170f730a-ba17-492c-94d6-cc2f6c0d9177\") " pod="openshift-monitoring/openshift-state-metrics-5669946b84-tgwn7" Apr 16 08:41:42.369123 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.369055 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/170f730a-ba17-492c-94d6-cc2f6c0d9177-openshift-state-metrics-tls\") pod \"openshift-state-metrics-5669946b84-tgwn7\" (UID: \"170f730a-ba17-492c-94d6-cc2f6c0d9177\") " pod="openshift-monitoring/openshift-state-metrics-5669946b84-tgwn7" Apr 16 08:41:42.369123 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.369092 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-sys\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.369279 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.369122 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-root\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.369279 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:42.369176 2578 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Apr 16 08:41:42.369279 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.369181 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-node-exporter-wtmp\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.369279 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.369226 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/04cf4d4f-3397-4603-bb37-6ce19013885c-volume-directive-shadow\") pod \"kube-state-metrics-7479c89684-sbzrt\" (UID: \"04cf4d4f-3397-4603-bb37-6ce19013885c\") " pod="openshift-monitoring/kube-state-metrics-7479c89684-sbzrt" Apr 16 08:41:42.369279 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:42.369242 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/170f730a-ba17-492c-94d6-cc2f6c0d9177-openshift-state-metrics-tls podName:170f730a-ba17-492c-94d6-cc2f6c0d9177 nodeName:}" failed. No retries permitted until 2026-04-16 08:41:42.869222618 +0000 UTC m=+151.238206654 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/170f730a-ba17-492c-94d6-cc2f6c0d9177-openshift-state-metrics-tls") pod "openshift-state-metrics-5669946b84-tgwn7" (UID: "170f730a-ba17-492c-94d6-cc2f6c0d9177") : secret "openshift-state-metrics-tls" not found Apr 16 08:41:42.369456 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.369286 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-node-exporter-textfile\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.369456 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.369347 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8fz7\" (UniqueName: \"kubernetes.io/projected/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-kube-api-access-j8fz7\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.369456 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.369381 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/04cf4d4f-3397-4603-bb37-6ce19013885c-kube-state-metrics-tls\") pod \"kube-state-metrics-7479c89684-sbzrt\" (UID: \"04cf4d4f-3397-4603-bb37-6ce19013885c\") " pod="openshift-monitoring/kube-state-metrics-7479c89684-sbzrt" Apr 16 08:41:42.369456 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.369407 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04cf4d4f-3397-4603-bb37-6ce19013885c-metrics-client-ca\") pod \"kube-state-metrics-7479c89684-sbzrt\" (UID: \"04cf4d4f-3397-4603-bb37-6ce19013885c\") " pod="openshift-monitoring/kube-state-metrics-7479c89684-sbzrt" Apr 16 08:41:42.369456 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.369425 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04cf4d4f-3397-4603-bb37-6ce19013885c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7479c89684-sbzrt\" (UID: \"04cf4d4f-3397-4603-bb37-6ce19013885c\") " pod="openshift-monitoring/kube-state-metrics-7479c89684-sbzrt" Apr 16 08:41:42.369456 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.369448 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/04cf4d4f-3397-4603-bb37-6ce19013885c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7479c89684-sbzrt\" (UID: \"04cf4d4f-3397-4603-bb37-6ce19013885c\") " pod="openshift-monitoring/kube-state-metrics-7479c89684-sbzrt" Apr 16 08:41:42.369681 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.369466 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-node-exporter-tls\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.369681 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.369513 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-metrics-client-ca\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.369681 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.369536 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-node-exporter-accelerators-collector-config\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.369681 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.369555 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frn7k\" (UniqueName: \"kubernetes.io/projected/04cf4d4f-3397-4603-bb37-6ce19013885c-kube-api-access-frn7k\") pod \"kube-state-metrics-7479c89684-sbzrt\" (UID: \"04cf4d4f-3397-4603-bb37-6ce19013885c\") " pod="openshift-monitoring/kube-state-metrics-7479c89684-sbzrt" Apr 16 08:41:42.369681 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.369571 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.370738 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.370718 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/170f730a-ba17-492c-94d6-cc2f6c0d9177-metrics-client-ca\") pod \"openshift-state-metrics-5669946b84-tgwn7\" (UID: \"170f730a-ba17-492c-94d6-cc2f6c0d9177\") " pod="openshift-monitoring/openshift-state-metrics-5669946b84-tgwn7" Apr 16 08:41:42.371594 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.371577 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/170f730a-ba17-492c-94d6-cc2f6c0d9177-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-5669946b84-tgwn7\" (UID: \"170f730a-ba17-492c-94d6-cc2f6c0d9177\") " pod="openshift-monitoring/openshift-state-metrics-5669946b84-tgwn7" Apr 16 08:41:42.378864 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.378843 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q886l\" (UniqueName: \"kubernetes.io/projected/170f730a-ba17-492c-94d6-cc2f6c0d9177-kube-api-access-q886l\") pod \"openshift-state-metrics-5669946b84-tgwn7\" (UID: \"170f730a-ba17-492c-94d6-cc2f6c0d9177\") " pod="openshift-monitoring/openshift-state-metrics-5669946b84-tgwn7" Apr 16 08:41:42.470144 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.470115 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-sys\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.470144 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.470147 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-root\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.470306 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.470166 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-node-exporter-wtmp\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.470306 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.470193 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/04cf4d4f-3397-4603-bb37-6ce19013885c-volume-directive-shadow\") pod \"kube-state-metrics-7479c89684-sbzrt\" (UID: \"04cf4d4f-3397-4603-bb37-6ce19013885c\") " pod="openshift-monitoring/kube-state-metrics-7479c89684-sbzrt" Apr 16 08:41:42.470306 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.470222 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-node-exporter-textfile\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.470306 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.470228 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-sys\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.470306 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.470237 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-root\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.470306 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.470268 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j8fz7\" (UniqueName: \"kubernetes.io/projected/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-kube-api-access-j8fz7\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.470306 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.470295 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/04cf4d4f-3397-4603-bb37-6ce19013885c-kube-state-metrics-tls\") pod \"kube-state-metrics-7479c89684-sbzrt\" (UID: \"04cf4d4f-3397-4603-bb37-6ce19013885c\") " pod="openshift-monitoring/kube-state-metrics-7479c89684-sbzrt" Apr 16 08:41:42.470596 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.470317 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-node-exporter-wtmp\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.470596 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.470320 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04cf4d4f-3397-4603-bb37-6ce19013885c-metrics-client-ca\") pod \"kube-state-metrics-7479c89684-sbzrt\" (UID: \"04cf4d4f-3397-4603-bb37-6ce19013885c\") " pod="openshift-monitoring/kube-state-metrics-7479c89684-sbzrt" Apr 16 08:41:42.470596 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.470364 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04cf4d4f-3397-4603-bb37-6ce19013885c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7479c89684-sbzrt\" (UID: \"04cf4d4f-3397-4603-bb37-6ce19013885c\") " pod="openshift-monitoring/kube-state-metrics-7479c89684-sbzrt" Apr 16 08:41:42.470596 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.470391 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/04cf4d4f-3397-4603-bb37-6ce19013885c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7479c89684-sbzrt\" (UID: \"04cf4d4f-3397-4603-bb37-6ce19013885c\") " pod="openshift-monitoring/kube-state-metrics-7479c89684-sbzrt" Apr 16 08:41:42.470596 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.470415 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-node-exporter-tls\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.470596 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.470448 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-metrics-client-ca\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.470596 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.470475 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-node-exporter-accelerators-collector-config\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.470596 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.470502 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-frn7k\" (UniqueName: \"kubernetes.io/projected/04cf4d4f-3397-4603-bb37-6ce19013885c-kube-api-access-frn7k\") pod \"kube-state-metrics-7479c89684-sbzrt\" (UID: \"04cf4d4f-3397-4603-bb37-6ce19013885c\") " pod="openshift-monitoring/kube-state-metrics-7479c89684-sbzrt" Apr 16 08:41:42.470596 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.470521 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-node-exporter-textfile\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.470596 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.470526 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.471054 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.470610 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/04cf4d4f-3397-4603-bb37-6ce19013885c-volume-directive-shadow\") pod \"kube-state-metrics-7479c89684-sbzrt\" (UID: \"04cf4d4f-3397-4603-bb37-6ce19013885c\") " pod="openshift-monitoring/kube-state-metrics-7479c89684-sbzrt" Apr 16 08:41:42.471054 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:42.470682 2578 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Apr 16 08:41:42.471054 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:42.470741 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-node-exporter-tls podName:aeba0bf6-c3d1-48b1-a406-a3cff57b6544 nodeName:}" failed. No retries permitted until 2026-04-16 08:41:42.970722536 +0000 UTC m=+151.339706585 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-node-exporter-tls") pod "node-exporter-z79cw" (UID: "aeba0bf6-c3d1-48b1-a406-a3cff57b6544") : secret "node-exporter-tls" not found Apr 16 08:41:42.471054 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.470983 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04cf4d4f-3397-4603-bb37-6ce19013885c-metrics-client-ca\") pod \"kube-state-metrics-7479c89684-sbzrt\" (UID: \"04cf4d4f-3397-4603-bb37-6ce19013885c\") " pod="openshift-monitoring/kube-state-metrics-7479c89684-sbzrt" Apr 16 08:41:42.471272 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.471254 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-node-exporter-accelerators-collector-config\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.471308 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.471269 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/04cf4d4f-3397-4603-bb37-6ce19013885c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7479c89684-sbzrt\" (UID: \"04cf4d4f-3397-4603-bb37-6ce19013885c\") " pod="openshift-monitoring/kube-state-metrics-7479c89684-sbzrt" Apr 16 08:41:42.471934 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.471904 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-metrics-client-ca\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.472758 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.472741 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.472997 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.472978 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04cf4d4f-3397-4603-bb37-6ce19013885c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7479c89684-sbzrt\" (UID: \"04cf4d4f-3397-4603-bb37-6ce19013885c\") " pod="openshift-monitoring/kube-state-metrics-7479c89684-sbzrt" Apr 16 08:41:42.473426 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.473408 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/04cf4d4f-3397-4603-bb37-6ce19013885c-kube-state-metrics-tls\") pod \"kube-state-metrics-7479c89684-sbzrt\" (UID: \"04cf4d4f-3397-4603-bb37-6ce19013885c\") " pod="openshift-monitoring/kube-state-metrics-7479c89684-sbzrt" Apr 16 08:41:42.480860 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.480831 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8fz7\" (UniqueName: \"kubernetes.io/projected/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-kube-api-access-j8fz7\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.483827 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.483804 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frn7k\" (UniqueName: \"kubernetes.io/projected/04cf4d4f-3397-4603-bb37-6ce19013885c-kube-api-access-frn7k\") pod \"kube-state-metrics-7479c89684-sbzrt\" (UID: \"04cf4d4f-3397-4603-bb37-6ce19013885c\") " pod="openshift-monitoring/kube-state-metrics-7479c89684-sbzrt" Apr 16 08:41:42.492256 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.492232 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7479c89684-sbzrt" Apr 16 08:41:42.613571 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.613545 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7479c89684-sbzrt"] Apr 16 08:41:42.615934 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:41:42.615909 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04cf4d4f_3397_4603_bb37_6ce19013885c.slice/crio-43d76ade3a47d511c1c6f72702b294d67bd9dc8df01b975b7c74a8a38747f632 WatchSource:0}: Error finding container 43d76ade3a47d511c1c6f72702b294d67bd9dc8df01b975b7c74a8a38747f632: Status 404 returned error can't find the container with id 43d76ade3a47d511c1c6f72702b294d67bd9dc8df01b975b7c74a8a38747f632 Apr 16 08:41:42.650445 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.650416 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7479c89684-sbzrt" event={"ID":"04cf4d4f-3397-4603-bb37-6ce19013885c","Type":"ContainerStarted","Data":"43d76ade3a47d511c1c6f72702b294d67bd9dc8df01b975b7c74a8a38747f632"} Apr 16 08:41:42.874342 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.874270 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/170f730a-ba17-492c-94d6-cc2f6c0d9177-openshift-state-metrics-tls\") pod \"openshift-state-metrics-5669946b84-tgwn7\" (UID: \"170f730a-ba17-492c-94d6-cc2f6c0d9177\") " pod="openshift-monitoring/openshift-state-metrics-5669946b84-tgwn7" Apr 16 08:41:42.876572 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.876545 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/170f730a-ba17-492c-94d6-cc2f6c0d9177-openshift-state-metrics-tls\") pod \"openshift-state-metrics-5669946b84-tgwn7\" (UID: \"170f730a-ba17-492c-94d6-cc2f6c0d9177\") " pod="openshift-monitoring/openshift-state-metrics-5669946b84-tgwn7" Apr 16 08:41:42.974725 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.974697 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-node-exporter-tls\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:42.976870 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:42.976853 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aeba0bf6-c3d1-48b1-a406-a3cff57b6544-node-exporter-tls\") pod \"node-exporter-z79cw\" (UID: \"aeba0bf6-c3d1-48b1-a406-a3cff57b6544\") " pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:43.062861 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:43.062841 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-5669946b84-tgwn7" Apr 16 08:41:43.103181 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:43.103144 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-z79cw" Apr 16 08:41:43.189937 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:43.189812 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-5669946b84-tgwn7"] Apr 16 08:41:43.192372 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:41:43.192348 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod170f730a_ba17_492c_94d6_cc2f6c0d9177.slice/crio-86487e60ee874cdc75832fa49ad1a5986d1ab55aab1073134b878ef77b31c561 WatchSource:0}: Error finding container 86487e60ee874cdc75832fa49ad1a5986d1ab55aab1073134b878ef77b31c561: Status 404 returned error can't find the container with id 86487e60ee874cdc75832fa49ad1a5986d1ab55aab1073134b878ef77b31c561 Apr 16 08:41:43.654610 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:43.654557 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z79cw" event={"ID":"aeba0bf6-c3d1-48b1-a406-a3cff57b6544","Type":"ContainerStarted","Data":"9370f3990fd23923f618eb8e4f5b4254f6f43c666a85ca51413d44ccfe328da4"} Apr 16 08:41:43.656181 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:43.656113 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-5669946b84-tgwn7" event={"ID":"170f730a-ba17-492c-94d6-cc2f6c0d9177","Type":"ContainerStarted","Data":"91fe4467dd452037fe83575fcb478f0b9053ec565f2fce08df384e4c4d73bbc3"} Apr 16 08:41:43.656181 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:43.656146 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-5669946b84-tgwn7" event={"ID":"170f730a-ba17-492c-94d6-cc2f6c0d9177","Type":"ContainerStarted","Data":"dfc7d312e776997b4044aa99e562e08bd2fef26d45e8feba5b5fcfe417a9a385"} Apr 16 08:41:43.656181 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:43.656161 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-5669946b84-tgwn7" event={"ID":"170f730a-ba17-492c-94d6-cc2f6c0d9177","Type":"ContainerStarted","Data":"86487e60ee874cdc75832fa49ad1a5986d1ab55aab1073134b878ef77b31c561"} Apr 16 08:41:44.138666 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.138645 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5658999486-4m4xg"] Apr 16 08:41:44.145583 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.145564 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:44.147403 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.147382 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Apr 16 08:41:44.147959 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.147938 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Apr 16 08:41:44.148393 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.148256 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Apr 16 08:41:44.148393 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.148270 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Apr 16 08:41:44.148393 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.148267 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-knx9x\"" Apr 16 08:41:44.148393 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.148369 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-fuu0u7ljt5550\"" Apr 16 08:41:44.148621 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.148404 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Apr 16 08:41:44.154548 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.154533 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5658999486-4m4xg"] Apr 16 08:41:44.182958 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.182930 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6929b02b-d393-44b8-a0a4-74c8eef0ced4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5658999486-4m4xg\" (UID: \"6929b02b-d393-44b8-a0a4-74c8eef0ced4\") " pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:44.183043 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.182977 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6929b02b-d393-44b8-a0a4-74c8eef0ced4-secret-thanos-querier-tls\") pod \"thanos-querier-5658999486-4m4xg\" (UID: \"6929b02b-d393-44b8-a0a4-74c8eef0ced4\") " pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:44.183098 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.183054 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6929b02b-d393-44b8-a0a4-74c8eef0ced4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5658999486-4m4xg\" (UID: \"6929b02b-d393-44b8-a0a4-74c8eef0ced4\") " pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:44.183098 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.183085 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6929b02b-d393-44b8-a0a4-74c8eef0ced4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5658999486-4m4xg\" (UID: \"6929b02b-d393-44b8-a0a4-74c8eef0ced4\") " pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:44.183176 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.183119 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6929b02b-d393-44b8-a0a4-74c8eef0ced4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5658999486-4m4xg\" (UID: \"6929b02b-d393-44b8-a0a4-74c8eef0ced4\") " pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:44.183176 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.183144 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6929b02b-d393-44b8-a0a4-74c8eef0ced4-secret-grpc-tls\") pod \"thanos-querier-5658999486-4m4xg\" (UID: \"6929b02b-d393-44b8-a0a4-74c8eef0ced4\") " pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:44.183176 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.183171 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6929b02b-d393-44b8-a0a4-74c8eef0ced4-metrics-client-ca\") pod \"thanos-querier-5658999486-4m4xg\" (UID: \"6929b02b-d393-44b8-a0a4-74c8eef0ced4\") " pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:44.183291 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.183214 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nddh\" (UniqueName: \"kubernetes.io/projected/6929b02b-d393-44b8-a0a4-74c8eef0ced4-kube-api-access-9nddh\") pod \"thanos-querier-5658999486-4m4xg\" (UID: \"6929b02b-d393-44b8-a0a4-74c8eef0ced4\") " pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:44.284523 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.284491 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6929b02b-d393-44b8-a0a4-74c8eef0ced4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5658999486-4m4xg\" (UID: \"6929b02b-d393-44b8-a0a4-74c8eef0ced4\") " pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:44.284917 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.284539 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6929b02b-d393-44b8-a0a4-74c8eef0ced4-secret-thanos-querier-tls\") pod \"thanos-querier-5658999486-4m4xg\" (UID: \"6929b02b-d393-44b8-a0a4-74c8eef0ced4\") " pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:44.284917 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.284579 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6929b02b-d393-44b8-a0a4-74c8eef0ced4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5658999486-4m4xg\" (UID: \"6929b02b-d393-44b8-a0a4-74c8eef0ced4\") " pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:44.284917 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.284610 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6929b02b-d393-44b8-a0a4-74c8eef0ced4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5658999486-4m4xg\" (UID: \"6929b02b-d393-44b8-a0a4-74c8eef0ced4\") " pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:44.284917 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.284656 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6929b02b-d393-44b8-a0a4-74c8eef0ced4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5658999486-4m4xg\" (UID: \"6929b02b-d393-44b8-a0a4-74c8eef0ced4\") " pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:44.284917 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.284682 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6929b02b-d393-44b8-a0a4-74c8eef0ced4-secret-grpc-tls\") pod \"thanos-querier-5658999486-4m4xg\" (UID: \"6929b02b-d393-44b8-a0a4-74c8eef0ced4\") " pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:44.284917 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.284708 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6929b02b-d393-44b8-a0a4-74c8eef0ced4-metrics-client-ca\") pod \"thanos-querier-5658999486-4m4xg\" (UID: \"6929b02b-d393-44b8-a0a4-74c8eef0ced4\") " pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:44.284917 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.284742 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9nddh\" (UniqueName: \"kubernetes.io/projected/6929b02b-d393-44b8-a0a4-74c8eef0ced4-kube-api-access-9nddh\") pod \"thanos-querier-5658999486-4m4xg\" (UID: \"6929b02b-d393-44b8-a0a4-74c8eef0ced4\") " pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:44.286172 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.286127 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6929b02b-d393-44b8-a0a4-74c8eef0ced4-metrics-client-ca\") pod \"thanos-querier-5658999486-4m4xg\" (UID: \"6929b02b-d393-44b8-a0a4-74c8eef0ced4\") " pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:44.288648 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.288616 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6929b02b-d393-44b8-a0a4-74c8eef0ced4-secret-thanos-querier-tls\") pod \"thanos-querier-5658999486-4m4xg\" (UID: \"6929b02b-d393-44b8-a0a4-74c8eef0ced4\") " pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:44.289421 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.289363 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6929b02b-d393-44b8-a0a4-74c8eef0ced4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5658999486-4m4xg\" (UID: \"6929b02b-d393-44b8-a0a4-74c8eef0ced4\") " pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:44.289594 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.289551 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6929b02b-d393-44b8-a0a4-74c8eef0ced4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5658999486-4m4xg\" (UID: \"6929b02b-d393-44b8-a0a4-74c8eef0ced4\") " pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:44.293596 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.292953 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6929b02b-d393-44b8-a0a4-74c8eef0ced4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5658999486-4m4xg\" (UID: \"6929b02b-d393-44b8-a0a4-74c8eef0ced4\") " pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:44.293596 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.293333 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6929b02b-d393-44b8-a0a4-74c8eef0ced4-secret-grpc-tls\") pod \"thanos-querier-5658999486-4m4xg\" (UID: \"6929b02b-d393-44b8-a0a4-74c8eef0ced4\") " pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:44.295519 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.295483 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6929b02b-d393-44b8-a0a4-74c8eef0ced4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5658999486-4m4xg\" (UID: \"6929b02b-d393-44b8-a0a4-74c8eef0ced4\") " pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:44.296289 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.296246 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nddh\" (UniqueName: \"kubernetes.io/projected/6929b02b-d393-44b8-a0a4-74c8eef0ced4-kube-api-access-9nddh\") pod \"thanos-querier-5658999486-4m4xg\" (UID: \"6929b02b-d393-44b8-a0a4-74c8eef0ced4\") " pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:44.456802 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.456760 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:44.596086 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.596038 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5658999486-4m4xg"] Apr 16 08:41:44.662787 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.662703 2578 generic.go:358] "Generic (PLEG): container finished" podID="aeba0bf6-c3d1-48b1-a406-a3cff57b6544" containerID="d4c2380048057ac6a3e12b550fc0368c7c0e71b7d8436d7b15bbe66646766549" exitCode=0 Apr 16 08:41:44.662905 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.662800 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z79cw" event={"ID":"aeba0bf6-c3d1-48b1-a406-a3cff57b6544","Type":"ContainerDied","Data":"d4c2380048057ac6a3e12b550fc0368c7c0e71b7d8436d7b15bbe66646766549"} Apr 16 08:41:44.664998 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.664965 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7479c89684-sbzrt" event={"ID":"04cf4d4f-3397-4603-bb37-6ce19013885c","Type":"ContainerStarted","Data":"25d12c35c4f3eacd3260c6b2f01c9ffa68712acd0072eca335504a18e800f3e7"} Apr 16 08:41:44.664998 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.664993 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7479c89684-sbzrt" event={"ID":"04cf4d4f-3397-4603-bb37-6ce19013885c","Type":"ContainerStarted","Data":"92832b0b17892c6f5ea92c868d80d619ada166e9b8d827ddabfa472a4aff986d"} Apr 16 08:41:44.665150 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.665006 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7479c89684-sbzrt" event={"ID":"04cf4d4f-3397-4603-bb37-6ce19013885c","Type":"ContainerStarted","Data":"8ec153afffe6fa8769382f01901730057a33b276325ff4531919dd89d7eb0ae7"} Apr 16 08:41:44.701757 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:44.701721 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7479c89684-sbzrt" podStartSLOduration=1.224072969 podStartE2EDuration="2.701707238s" podCreationTimestamp="2026-04-16 08:41:42 +0000 UTC" firstStartedPulling="2026-04-16 08:41:42.617756865 +0000 UTC m=+150.986740900" lastFinishedPulling="2026-04-16 08:41:44.095391125 +0000 UTC m=+152.464375169" observedRunningTime="2026-04-16 08:41:44.70074252 +0000 UTC m=+153.069726576" watchObservedRunningTime="2026-04-16 08:41:44.701707238 +0000 UTC m=+153.070691295" Apr 16 08:41:44.749675 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:41:44.749649 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6929b02b_d393_44b8_a0a4_74c8eef0ced4.slice/crio-179bf28ae4603d150e98dc08549d530a6ef002fe54e55d3825d172a515ac1cf5 WatchSource:0}: Error finding container 179bf28ae4603d150e98dc08549d530a6ef002fe54e55d3825d172a515ac1cf5: Status 404 returned error can't find the container with id 179bf28ae4603d150e98dc08549d530a6ef002fe54e55d3825d172a515ac1cf5 Apr 16 08:41:45.670281 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:45.670242 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z79cw" event={"ID":"aeba0bf6-c3d1-48b1-a406-a3cff57b6544","Type":"ContainerStarted","Data":"d1adb8ff433f2b72b8a94ed8de31518474b53cf98a1e045aa6c571a2c1946fb3"} Apr 16 08:41:45.670281 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:45.670285 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z79cw" event={"ID":"aeba0bf6-c3d1-48b1-a406-a3cff57b6544","Type":"ContainerStarted","Data":"419619b3a9676db5c2a37991c5c719d4e4babb111d4958984553626ddc0f9ec8"} Apr 16 08:41:45.672413 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:45.672357 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-5669946b84-tgwn7" event={"ID":"170f730a-ba17-492c-94d6-cc2f6c0d9177","Type":"ContainerStarted","Data":"00fc18082de4bf2c39148ba100f55c8d45308a51ed6c1b72b9031d206bbaa764"} Apr 16 08:41:45.673582 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:45.673552 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" event={"ID":"6929b02b-d393-44b8-a0a4-74c8eef0ced4","Type":"ContainerStarted","Data":"179bf28ae4603d150e98dc08549d530a6ef002fe54e55d3825d172a515ac1cf5"} Apr 16 08:41:45.686611 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:45.686567 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-z79cw" podStartSLOduration=2.706453738 podStartE2EDuration="3.686551412s" podCreationTimestamp="2026-04-16 08:41:42 +0000 UTC" firstStartedPulling="2026-04-16 08:41:43.116849608 +0000 UTC m=+151.485833648" lastFinishedPulling="2026-04-16 08:41:44.096947284 +0000 UTC m=+152.465931322" observedRunningTime="2026-04-16 08:41:45.685781429 +0000 UTC m=+154.054765481" watchObservedRunningTime="2026-04-16 08:41:45.686551412 +0000 UTC m=+154.055535471" Apr 16 08:41:45.701839 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:45.701761 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-5669946b84-tgwn7" podStartSLOduration=2.250011368 podStartE2EDuration="3.701747816s" podCreationTimestamp="2026-04-16 08:41:42 +0000 UTC" firstStartedPulling="2026-04-16 08:41:43.342561255 +0000 UTC m=+151.711545296" lastFinishedPulling="2026-04-16 08:41:44.794297709 +0000 UTC m=+153.163281744" observedRunningTime="2026-04-16 08:41:45.699740228 +0000 UTC m=+154.068724319" watchObservedRunningTime="2026-04-16 08:41:45.701747816 +0000 UTC m=+154.070731872" Apr 16 08:41:46.142879 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:46.142845 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79c5cdd649-xthk8" Apr 16 08:41:46.143022 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:46.142894 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-79c5cdd649-xthk8" Apr 16 08:41:46.144372 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:46.144344 2578 patch_prober.go:28] interesting pod/console-79c5cdd649-xthk8 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.15:8443/health\": dial tcp 10.134.0.15:8443: connect: connection refused" start-of-body= Apr 16 08:41:46.144486 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:46.144395 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-79c5cdd649-xthk8" podUID="19866fa8-b487-41d7-9eba-73fe93fcb783" containerName="console" probeResult="failure" output="Get \"https://10.134.0.15:8443/health\": dial tcp 10.134.0.15:8443: connect: connection refused" Apr 16 08:41:47.350528 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.350113 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-749764bc74-49p74"] Apr 16 08:41:47.354113 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.354091 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-749764bc74-49p74" Apr 16 08:41:47.356059 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.356039 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Apr 16 08:41:47.356059 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.356054 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Apr 16 08:41:47.356208 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.356041 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Apr 16 08:41:47.356208 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.356122 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-zhwkj\"" Apr 16 08:41:47.356321 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.356306 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Apr 16 08:41:47.356574 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.356560 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Apr 16 08:41:47.361281 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.361261 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Apr 16 08:41:47.365244 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.365217 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-749764bc74-49p74"] Apr 16 08:41:47.414568 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.414547 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bc059993-986e-4758-8e80-fa0a512bc2d4-secret-telemeter-client\") pod \"telemeter-client-749764bc74-49p74\" (UID: \"bc059993-986e-4758-8e80-fa0a512bc2d4\") " pod="openshift-monitoring/telemeter-client-749764bc74-49p74" Apr 16 08:41:47.414663 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.414612 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc059993-986e-4758-8e80-fa0a512bc2d4-serving-certs-ca-bundle\") pod \"telemeter-client-749764bc74-49p74\" (UID: \"bc059993-986e-4758-8e80-fa0a512bc2d4\") " pod="openshift-monitoring/telemeter-client-749764bc74-49p74" Apr 16 08:41:47.414663 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.414646 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bc059993-986e-4758-8e80-fa0a512bc2d4-telemeter-client-tls\") pod \"telemeter-client-749764bc74-49p74\" (UID: \"bc059993-986e-4758-8e80-fa0a512bc2d4\") " pod="openshift-monitoring/telemeter-client-749764bc74-49p74" Apr 16 08:41:47.414761 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.414673 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc059993-986e-4758-8e80-fa0a512bc2d4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-749764bc74-49p74\" (UID: \"bc059993-986e-4758-8e80-fa0a512bc2d4\") " pod="openshift-monitoring/telemeter-client-749764bc74-49p74" Apr 16 08:41:47.414761 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.414690 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bc059993-986e-4758-8e80-fa0a512bc2d4-federate-client-tls\") pod \"telemeter-client-749764bc74-49p74\" (UID: \"bc059993-986e-4758-8e80-fa0a512bc2d4\") " pod="openshift-monitoring/telemeter-client-749764bc74-49p74" Apr 16 08:41:47.414853 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.414793 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpjjp\" (UniqueName: \"kubernetes.io/projected/bc059993-986e-4758-8e80-fa0a512bc2d4-kube-api-access-hpjjp\") pod \"telemeter-client-749764bc74-49p74\" (UID: \"bc059993-986e-4758-8e80-fa0a512bc2d4\") " pod="openshift-monitoring/telemeter-client-749764bc74-49p74" Apr 16 08:41:47.414853 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.414815 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bc059993-986e-4758-8e80-fa0a512bc2d4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-749764bc74-49p74\" (UID: \"bc059993-986e-4758-8e80-fa0a512bc2d4\") " pod="openshift-monitoring/telemeter-client-749764bc74-49p74" Apr 16 08:41:47.414853 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.414831 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc059993-986e-4758-8e80-fa0a512bc2d4-metrics-client-ca\") pod \"telemeter-client-749764bc74-49p74\" (UID: \"bc059993-986e-4758-8e80-fa0a512bc2d4\") " pod="openshift-monitoring/telemeter-client-749764bc74-49p74" Apr 16 08:41:47.515643 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.515615 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bc059993-986e-4758-8e80-fa0a512bc2d4-secret-telemeter-client\") pod \"telemeter-client-749764bc74-49p74\" (UID: \"bc059993-986e-4758-8e80-fa0a512bc2d4\") " pod="openshift-monitoring/telemeter-client-749764bc74-49p74" Apr 16 08:41:47.515790 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.515685 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc059993-986e-4758-8e80-fa0a512bc2d4-serving-certs-ca-bundle\") pod \"telemeter-client-749764bc74-49p74\" (UID: \"bc059993-986e-4758-8e80-fa0a512bc2d4\") " pod="openshift-monitoring/telemeter-client-749764bc74-49p74" Apr 16 08:41:47.515790 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.515718 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bc059993-986e-4758-8e80-fa0a512bc2d4-telemeter-client-tls\") pod \"telemeter-client-749764bc74-49p74\" (UID: \"bc059993-986e-4758-8e80-fa0a512bc2d4\") " pod="openshift-monitoring/telemeter-client-749764bc74-49p74" Apr 16 08:41:47.515790 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.515751 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc059993-986e-4758-8e80-fa0a512bc2d4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-749764bc74-49p74\" (UID: \"bc059993-986e-4758-8e80-fa0a512bc2d4\") " pod="openshift-monitoring/telemeter-client-749764bc74-49p74" Apr 16 08:41:47.515962 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.515792 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bc059993-986e-4758-8e80-fa0a512bc2d4-federate-client-tls\") pod \"telemeter-client-749764bc74-49p74\" (UID: \"bc059993-986e-4758-8e80-fa0a512bc2d4\") " pod="openshift-monitoring/telemeter-client-749764bc74-49p74" Apr 16 08:41:47.515962 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.515850 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hpjjp\" (UniqueName: \"kubernetes.io/projected/bc059993-986e-4758-8e80-fa0a512bc2d4-kube-api-access-hpjjp\") pod \"telemeter-client-749764bc74-49p74\" (UID: \"bc059993-986e-4758-8e80-fa0a512bc2d4\") " pod="openshift-monitoring/telemeter-client-749764bc74-49p74" Apr 16 08:41:47.515962 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.515881 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bc059993-986e-4758-8e80-fa0a512bc2d4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-749764bc74-49p74\" (UID: \"bc059993-986e-4758-8e80-fa0a512bc2d4\") " pod="openshift-monitoring/telemeter-client-749764bc74-49p74" Apr 16 08:41:47.515962 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.515910 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc059993-986e-4758-8e80-fa0a512bc2d4-metrics-client-ca\") pod \"telemeter-client-749764bc74-49p74\" (UID: \"bc059993-986e-4758-8e80-fa0a512bc2d4\") " pod="openshift-monitoring/telemeter-client-749764bc74-49p74" Apr 16 08:41:47.517014 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.516868 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc059993-986e-4758-8e80-fa0a512bc2d4-metrics-client-ca\") pod \"telemeter-client-749764bc74-49p74\" (UID: \"bc059993-986e-4758-8e80-fa0a512bc2d4\") " pod="openshift-monitoring/telemeter-client-749764bc74-49p74" Apr 16 08:41:47.517014 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.516972 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc059993-986e-4758-8e80-fa0a512bc2d4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-749764bc74-49p74\" (UID: \"bc059993-986e-4758-8e80-fa0a512bc2d4\") " pod="openshift-monitoring/telemeter-client-749764bc74-49p74" Apr 16 08:41:47.517913 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.517891 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc059993-986e-4758-8e80-fa0a512bc2d4-serving-certs-ca-bundle\") pod \"telemeter-client-749764bc74-49p74\" (UID: \"bc059993-986e-4758-8e80-fa0a512bc2d4\") " pod="openshift-monitoring/telemeter-client-749764bc74-49p74" Apr 16 08:41:47.518870 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.518848 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bc059993-986e-4758-8e80-fa0a512bc2d4-secret-telemeter-client\") pod \"telemeter-client-749764bc74-49p74\" (UID: \"bc059993-986e-4758-8e80-fa0a512bc2d4\") " pod="openshift-monitoring/telemeter-client-749764bc74-49p74" Apr 16 08:41:47.519571 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.519538 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bc059993-986e-4758-8e80-fa0a512bc2d4-federate-client-tls\") pod \"telemeter-client-749764bc74-49p74\" (UID: \"bc059993-986e-4758-8e80-fa0a512bc2d4\") " pod="openshift-monitoring/telemeter-client-749764bc74-49p74" Apr 16 08:41:47.519704 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.519675 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bc059993-986e-4758-8e80-fa0a512bc2d4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-749764bc74-49p74\" (UID: \"bc059993-986e-4758-8e80-fa0a512bc2d4\") " pod="openshift-monitoring/telemeter-client-749764bc74-49p74" Apr 16 08:41:47.519873 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.519835 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bc059993-986e-4758-8e80-fa0a512bc2d4-telemeter-client-tls\") pod \"telemeter-client-749764bc74-49p74\" (UID: \"bc059993-986e-4758-8e80-fa0a512bc2d4\") " pod="openshift-monitoring/telemeter-client-749764bc74-49p74" Apr 16 08:41:47.523257 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.523235 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpjjp\" (UniqueName: \"kubernetes.io/projected/bc059993-986e-4758-8e80-fa0a512bc2d4-kube-api-access-hpjjp\") pod \"telemeter-client-749764bc74-49p74\" (UID: \"bc059993-986e-4758-8e80-fa0a512bc2d4\") " pod="openshift-monitoring/telemeter-client-749764bc74-49p74" Apr 16 08:41:47.553287 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:47.553257 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-rqjhz" podUID="b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8" Apr 16 08:41:47.558408 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:41:47.558375 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-nlp9v" podUID="6136520a-182b-44c5-837e-1dd3dc6af3f7" Apr 16 08:41:47.663795 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.663698 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-749764bc74-49p74" Apr 16 08:41:47.680834 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.680811 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rqjhz" Apr 16 08:41:47.680937 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.680803 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" event={"ID":"6929b02b-d393-44b8-a0a4-74c8eef0ced4","Type":"ContainerStarted","Data":"d728ddae64339f65108687f4c31ada89d49bd9656a23bb04f8ccb2b79dfa905a"} Apr 16 08:41:47.680937 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.680861 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" event={"ID":"6929b02b-d393-44b8-a0a4-74c8eef0ced4","Type":"ContainerStarted","Data":"c4cdf98b943be72eb420f0bdda1a23a6f373977562f8591fcd5128fdddd83a00"} Apr 16 08:41:47.680937 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.680876 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" event={"ID":"6929b02b-d393-44b8-a0a4-74c8eef0ced4","Type":"ContainerStarted","Data":"e56fc1984070e8d2b4b308b3fcccaba6eeb3298b85594c2879c2ae442dc28bd6"} Apr 16 08:41:47.680937 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.680874 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nlp9v" Apr 16 08:41:47.850672 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:47.850648 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-749764bc74-49p74"] Apr 16 08:41:47.855192 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:41:47.855157 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc059993_986e_4758_8e80_fa0a512bc2d4.slice/crio-84f557c0312ac86824b10784ed1fae935b8d5230ea57c8d44674de9ee7df64d2 WatchSource:0}: Error finding container 84f557c0312ac86824b10784ed1fae935b8d5230ea57c8d44674de9ee7df64d2: Status 404 returned error can't find the container with id 84f557c0312ac86824b10784ed1fae935b8d5230ea57c8d44674de9ee7df64d2 Apr 16 08:41:48.271043 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.271015 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-696844548f-bmn2l"] Apr 16 08:41:48.274214 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.274199 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:41:48.281909 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.281886 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-696844548f-bmn2l"] Apr 16 08:41:48.323644 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.323608 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tht9q\" (UniqueName: \"kubernetes.io/projected/77058410-be22-4f4a-b977-81eaa77c149a-kube-api-access-tht9q\") pod \"console-696844548f-bmn2l\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:41:48.323911 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.323893 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/77058410-be22-4f4a-b977-81eaa77c149a-service-ca\") pod \"console-696844548f-bmn2l\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:41:48.324065 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.324052 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/77058410-be22-4f4a-b977-81eaa77c149a-oauth-serving-cert\") pod \"console-696844548f-bmn2l\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:41:48.324238 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.324224 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/77058410-be22-4f4a-b977-81eaa77c149a-console-oauth-config\") pod \"console-696844548f-bmn2l\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:41:48.324408 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.324395 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/77058410-be22-4f4a-b977-81eaa77c149a-console-serving-cert\") pod \"console-696844548f-bmn2l\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:41:48.324541 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.324527 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77058410-be22-4f4a-b977-81eaa77c149a-trusted-ca-bundle\") pod \"console-696844548f-bmn2l\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:41:48.324681 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.324670 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/77058410-be22-4f4a-b977-81eaa77c149a-console-config\") pod \"console-696844548f-bmn2l\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:41:48.426064 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.426030 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/77058410-be22-4f4a-b977-81eaa77c149a-console-oauth-config\") pod \"console-696844548f-bmn2l\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:41:48.426455 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.426102 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/77058410-be22-4f4a-b977-81eaa77c149a-console-serving-cert\") pod \"console-696844548f-bmn2l\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:41:48.426455 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.426125 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77058410-be22-4f4a-b977-81eaa77c149a-trusted-ca-bundle\") pod \"console-696844548f-bmn2l\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:41:48.426455 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.426164 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/77058410-be22-4f4a-b977-81eaa77c149a-console-config\") pod \"console-696844548f-bmn2l\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:41:48.426455 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.426233 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tht9q\" (UniqueName: \"kubernetes.io/projected/77058410-be22-4f4a-b977-81eaa77c149a-kube-api-access-tht9q\") pod \"console-696844548f-bmn2l\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:41:48.426455 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.426264 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/77058410-be22-4f4a-b977-81eaa77c149a-service-ca\") pod \"console-696844548f-bmn2l\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:41:48.426455 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.426299 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/77058410-be22-4f4a-b977-81eaa77c149a-oauth-serving-cert\") pod \"console-696844548f-bmn2l\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:41:48.427055 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.427027 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/77058410-be22-4f4a-b977-81eaa77c149a-console-config\") pod \"console-696844548f-bmn2l\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:41:48.427207 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.427161 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77058410-be22-4f4a-b977-81eaa77c149a-trusted-ca-bundle\") pod \"console-696844548f-bmn2l\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:41:48.427514 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.427470 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/77058410-be22-4f4a-b977-81eaa77c149a-oauth-serving-cert\") pod \"console-696844548f-bmn2l\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:41:48.429510 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.428896 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/77058410-be22-4f4a-b977-81eaa77c149a-service-ca\") pod \"console-696844548f-bmn2l\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:41:48.429510 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.429154 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/77058410-be22-4f4a-b977-81eaa77c149a-console-oauth-config\") pod \"console-696844548f-bmn2l\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:41:48.430396 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.430377 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/77058410-be22-4f4a-b977-81eaa77c149a-console-serving-cert\") pod \"console-696844548f-bmn2l\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:41:48.435734 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.435711 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tht9q\" (UniqueName: \"kubernetes.io/projected/77058410-be22-4f4a-b977-81eaa77c149a-kube-api-access-tht9q\") pod \"console-696844548f-bmn2l\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:41:48.583426 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.583332 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:41:48.695799 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.689996 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" event={"ID":"6929b02b-d393-44b8-a0a4-74c8eef0ced4","Type":"ContainerStarted","Data":"6e594845ffc7dc4986827a335964098903245541414eff52d7d96c3d5cc04dfc"} Apr 16 08:41:48.695799 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.690036 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" event={"ID":"6929b02b-d393-44b8-a0a4-74c8eef0ced4","Type":"ContainerStarted","Data":"95b8ae0e9f9f158acf17a918d8b7b2be9996cd8ef622f8f3c5a0b27ed2740505"} Apr 16 08:41:48.695799 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.690049 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" event={"ID":"6929b02b-d393-44b8-a0a4-74c8eef0ced4","Type":"ContainerStarted","Data":"e93f2e37ed83e1dca2f66c285242b6819c25a63d1e425270d48165f38b40e2d0"} Apr 16 08:41:48.695799 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.691046 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:48.695799 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.694353 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-749764bc74-49p74" event={"ID":"bc059993-986e-4758-8e80-fa0a512bc2d4","Type":"ContainerStarted","Data":"84f557c0312ac86824b10784ed1fae935b8d5230ea57c8d44674de9ee7df64d2"} Apr 16 08:41:48.721459 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.720355 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" podStartSLOduration=1.697341497 podStartE2EDuration="4.72033685s" podCreationTimestamp="2026-04-16 08:41:44 +0000 UTC" firstStartedPulling="2026-04-16 08:41:44.751609575 +0000 UTC m=+153.120593611" lastFinishedPulling="2026-04-16 08:41:47.774604917 +0000 UTC m=+156.143588964" observedRunningTime="2026-04-16 08:41:48.713670715 +0000 UTC m=+157.082654773" watchObservedRunningTime="2026-04-16 08:41:48.72033685 +0000 UTC m=+157.089320906" Apr 16 08:41:48.733434 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:48.733402 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-696844548f-bmn2l"] Apr 16 08:41:48.738402 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:41:48.738374 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77058410_be22_4f4a_b977_81eaa77c149a.slice/crio-079691c65fd9f112b4648ec01697879e1a7dbdbc74c9ab9601de7c9603bcec42 WatchSource:0}: Error finding container 079691c65fd9f112b4648ec01697879e1a7dbdbc74c9ab9601de7c9603bcec42: Status 404 returned error can't find the container with id 079691c65fd9f112b4648ec01697879e1a7dbdbc74c9ab9601de7c9603bcec42 Apr 16 08:41:49.700559 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:49.700524 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-696844548f-bmn2l" event={"ID":"77058410-be22-4f4a-b977-81eaa77c149a","Type":"ContainerStarted","Data":"0e4012b2a9db5b45d46bb87eec6769b6b7432d1b38f62bf8d68af5183804dfee"} Apr 16 08:41:49.700559 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:49.700558 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-696844548f-bmn2l" event={"ID":"77058410-be22-4f4a-b977-81eaa77c149a","Type":"ContainerStarted","Data":"079691c65fd9f112b4648ec01697879e1a7dbdbc74c9ab9601de7c9603bcec42"} Apr 16 08:41:49.701821 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:49.701797 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-749764bc74-49p74" event={"ID":"bc059993-986e-4758-8e80-fa0a512bc2d4","Type":"ContainerStarted","Data":"94e97ae6cb4d032288e71d433e99ec1afff37ca9e1182cd87fdd921e8b7fb3c3"} Apr 16 08:41:49.718233 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:49.718146 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-696844548f-bmn2l" podStartSLOduration=1.718132113 podStartE2EDuration="1.718132113s" podCreationTimestamp="2026-04-16 08:41:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 08:41:49.716646671 +0000 UTC m=+158.085630726" watchObservedRunningTime="2026-04-16 08:41:49.718132113 +0000 UTC m=+158.087116169" Apr 16 08:41:50.707085 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:50.706999 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-749764bc74-49p74" event={"ID":"bc059993-986e-4758-8e80-fa0a512bc2d4","Type":"ContainerStarted","Data":"2d8511aa718e0881032809d35ddc7a61a985fcc2d303bd433893adb6d28380ef"} Apr 16 08:41:50.707085 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:50.707038 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-749764bc74-49p74" event={"ID":"bc059993-986e-4758-8e80-fa0a512bc2d4","Type":"ContainerStarted","Data":"c618f54e1e4adda61f3367046302161f5d014bdc991d2df216938d22f56fd700"} Apr 16 08:41:50.728883 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:50.728835 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-749764bc74-49p74" podStartSLOduration=1.1685334 podStartE2EDuration="3.728822244s" podCreationTimestamp="2026-04-16 08:41:47 +0000 UTC" firstStartedPulling="2026-04-16 08:41:47.856932789 +0000 UTC m=+156.225916830" lastFinishedPulling="2026-04-16 08:41:50.417221636 +0000 UTC m=+158.786205674" observedRunningTime="2026-04-16 08:41:50.727969717 +0000 UTC m=+159.096953775" watchObservedRunningTime="2026-04-16 08:41:50.728822244 +0000 UTC m=+159.097806300" Apr 16 08:41:52.036644 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.036607 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79c5cdd649-xthk8"] Apr 16 08:41:52.060092 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.060063 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-64d9d6dc47-qlvzx"] Apr 16 08:41:52.063042 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.063027 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:41:52.073234 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.073211 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64d9d6dc47-qlvzx"] Apr 16 08:41:52.161610 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.161587 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-service-ca\") pod \"console-64d9d6dc47-qlvzx\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:41:52.161704 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.161613 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t278\" (UniqueName: \"kubernetes.io/projected/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-kube-api-access-2t278\") pod \"console-64d9d6dc47-qlvzx\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:41:52.161704 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.161633 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-console-oauth-config\") pod \"console-64d9d6dc47-qlvzx\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:41:52.161704 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.161655 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-console-config\") pod \"console-64d9d6dc47-qlvzx\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:41:52.161829 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.161736 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-trusted-ca-bundle\") pod \"console-64d9d6dc47-qlvzx\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:41:52.161862 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.161825 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-oauth-serving-cert\") pod \"console-64d9d6dc47-qlvzx\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:41:52.161928 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.161861 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-console-serving-cert\") pod \"console-64d9d6dc47-qlvzx\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:41:52.262291 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.262262 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-service-ca\") pod \"console-64d9d6dc47-qlvzx\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:41:52.262387 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.262300 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2t278\" (UniqueName: \"kubernetes.io/projected/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-kube-api-access-2t278\") pod \"console-64d9d6dc47-qlvzx\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:41:52.262387 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.262330 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-console-oauth-config\") pod \"console-64d9d6dc47-qlvzx\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:41:52.262387 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.262379 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-console-config\") pod \"console-64d9d6dc47-qlvzx\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:41:52.262541 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.262430 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-trusted-ca-bundle\") pod \"console-64d9d6dc47-qlvzx\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:41:52.262541 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.262477 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-oauth-serving-cert\") pod \"console-64d9d6dc47-qlvzx\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:41:52.262541 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.262505 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-console-serving-cert\") pod \"console-64d9d6dc47-qlvzx\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:41:52.262963 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.262933 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-service-ca\") pod \"console-64d9d6dc47-qlvzx\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:41:52.263132 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.263111 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-oauth-serving-cert\") pod \"console-64d9d6dc47-qlvzx\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:41:52.263183 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.263111 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-console-config\") pod \"console-64d9d6dc47-qlvzx\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:41:52.263735 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.263717 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-trusted-ca-bundle\") pod \"console-64d9d6dc47-qlvzx\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:41:52.264865 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.264837 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-console-oauth-config\") pod \"console-64d9d6dc47-qlvzx\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:41:52.264936 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.264839 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-console-serving-cert\") pod \"console-64d9d6dc47-qlvzx\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:41:52.269121 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.269102 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t278\" (UniqueName: \"kubernetes.io/projected/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-kube-api-access-2t278\") pod \"console-64d9d6dc47-qlvzx\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:41:52.371075 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.371025 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:41:52.464290 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.464266 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6136520a-182b-44c5-837e-1dd3dc6af3f7-cert\") pod \"ingress-canary-nlp9v\" (UID: \"6136520a-182b-44c5-837e-1dd3dc6af3f7\") " pod="openshift-ingress-canary/ingress-canary-nlp9v" Apr 16 08:41:52.464424 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.464350 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-metrics-tls\") pod \"dns-default-rqjhz\" (UID: \"b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8\") " pod="openshift-dns/dns-default-rqjhz" Apr 16 08:41:52.466742 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.466712 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8-metrics-tls\") pod \"dns-default-rqjhz\" (UID: \"b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8\") " pod="openshift-dns/dns-default-rqjhz" Apr 16 08:41:52.466855 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.466802 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6136520a-182b-44c5-837e-1dd3dc6af3f7-cert\") pod \"ingress-canary-nlp9v\" (UID: \"6136520a-182b-44c5-837e-1dd3dc6af3f7\") " pod="openshift-ingress-canary/ingress-canary-nlp9v" Apr 16 08:41:52.481201 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.481171 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64d9d6dc47-qlvzx"] Apr 16 08:41:52.483266 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.483246 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-q9sj7\"" Apr 16 08:41:52.483364 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.483269 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9d2hv\"" Apr 16 08:41:52.484452 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:41:52.484428 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ac4bc10_99f5_45c4_897f_6e82d1fa074f.slice/crio-4f161c3e10e3b17e1f9cefeb2f0e5cf87b4b7ed72ac6099c5e76e5b334e174ac WatchSource:0}: Error finding container 4f161c3e10e3b17e1f9cefeb2f0e5cf87b4b7ed72ac6099c5e76e5b334e174ac: Status 404 returned error can't find the container with id 4f161c3e10e3b17e1f9cefeb2f0e5cf87b4b7ed72ac6099c5e76e5b334e174ac Apr 16 08:41:52.492181 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.492168 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nlp9v" Apr 16 08:41:52.492267 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.492195 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rqjhz" Apr 16 08:41:52.622049 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.621945 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rqjhz"] Apr 16 08:41:52.630423 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:41:52.630391 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1cd8102_c016_4e4b_96ee_2f3e5b0ec7b8.slice/crio-9cc5ddfba4d3af0d3f1507c622b41bf8d6ddcdc360bb7227b12bf70f52f6d904 WatchSource:0}: Error finding container 9cc5ddfba4d3af0d3f1507c622b41bf8d6ddcdc360bb7227b12bf70f52f6d904: Status 404 returned error can't find the container with id 9cc5ddfba4d3af0d3f1507c622b41bf8d6ddcdc360bb7227b12bf70f52f6d904 Apr 16 08:41:52.641471 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.641444 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nlp9v"] Apr 16 08:41:52.645000 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:41:52.644973 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6136520a_182b_44c5_837e_1dd3dc6af3f7.slice/crio-313d0ef685f4cf8b1797e29cbaa8a4f06beda6529c4a0259757d2d3093e7cde6 WatchSource:0}: Error finding container 313d0ef685f4cf8b1797e29cbaa8a4f06beda6529c4a0259757d2d3093e7cde6: Status 404 returned error can't find the container with id 313d0ef685f4cf8b1797e29cbaa8a4f06beda6529c4a0259757d2d3093e7cde6 Apr 16 08:41:52.713258 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.713225 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nlp9v" event={"ID":"6136520a-182b-44c5-837e-1dd3dc6af3f7","Type":"ContainerStarted","Data":"313d0ef685f4cf8b1797e29cbaa8a4f06beda6529c4a0259757d2d3093e7cde6"} Apr 16 08:41:52.714508 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.714491 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64d9d6dc47-qlvzx" event={"ID":"6ac4bc10-99f5-45c4-897f-6e82d1fa074f","Type":"ContainerStarted","Data":"7fc89162911b77ed539019383be12394c0d4cb9af2accc96b79fa9cfe73e2bdf"} Apr 16 08:41:52.714588 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.714513 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64d9d6dc47-qlvzx" event={"ID":"6ac4bc10-99f5-45c4-897f-6e82d1fa074f","Type":"ContainerStarted","Data":"4f161c3e10e3b17e1f9cefeb2f0e5cf87b4b7ed72ac6099c5e76e5b334e174ac"} Apr 16 08:41:52.715588 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.715570 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rqjhz" event={"ID":"b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8","Type":"ContainerStarted","Data":"9cc5ddfba4d3af0d3f1507c622b41bf8d6ddcdc360bb7227b12bf70f52f6d904"} Apr 16 08:41:52.730799 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:52.730743 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-64d9d6dc47-qlvzx" podStartSLOduration=0.730733179 podStartE2EDuration="730.733179ms" podCreationTimestamp="2026-04-16 08:41:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 08:41:52.728524128 +0000 UTC m=+161.097508184" watchObservedRunningTime="2026-04-16 08:41:52.730733179 +0000 UTC m=+161.099717235" Apr 16 08:41:54.723904 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:54.723870 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rqjhz" event={"ID":"b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8","Type":"ContainerStarted","Data":"b0b638cfa893aaca8e18bd220c8fa663e932a9973113c016a2457f41ae3f78be"} Apr 16 08:41:54.724268 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:54.723910 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rqjhz" event={"ID":"b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8","Type":"ContainerStarted","Data":"408805ca20bc3c95b9bb024136afd468029afb5adda596ec7c4222b0c03fe3a7"} Apr 16 08:41:54.724268 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:54.724023 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rqjhz" Apr 16 08:41:54.725172 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:54.725147 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nlp9v" event={"ID":"6136520a-182b-44c5-837e-1dd3dc6af3f7","Type":"ContainerStarted","Data":"8229b05726f2f4d68df298e0127755572161c29e1d2a03fb4f87e300b06d2942"} Apr 16 08:41:54.740089 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:54.739236 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-rqjhz" podStartSLOduration=128.979590113 podStartE2EDuration="2m10.739221334s" podCreationTimestamp="2026-04-16 08:39:44 +0000 UTC" firstStartedPulling="2026-04-16 08:41:52.632299057 +0000 UTC m=+161.001283093" lastFinishedPulling="2026-04-16 08:41:54.391930267 +0000 UTC m=+162.760914314" observedRunningTime="2026-04-16 08:41:54.737704269 +0000 UTC m=+163.106688316" watchObservedRunningTime="2026-04-16 08:41:54.739221334 +0000 UTC m=+163.108205391" Apr 16 08:41:54.752282 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:54.752248 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nlp9v" podStartSLOduration=129.00274367 podStartE2EDuration="2m10.752236832s" podCreationTimestamp="2026-04-16 08:39:44 +0000 UTC" firstStartedPulling="2026-04-16 08:41:52.646563385 +0000 UTC m=+161.015547421" lastFinishedPulling="2026-04-16 08:41:54.396056533 +0000 UTC m=+162.765040583" observedRunningTime="2026-04-16 08:41:54.750681145 +0000 UTC m=+163.119665202" watchObservedRunningTime="2026-04-16 08:41:54.752236832 +0000 UTC m=+163.121220951" Apr 16 08:41:55.712851 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:55.712822 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5658999486-4m4xg" Apr 16 08:41:58.584446 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:58.584409 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:41:58.584924 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:58.584533 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:41:58.589225 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:58.589203 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:41:58.742732 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:41:58.742709 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:42:02.372114 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:02.372074 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:42:02.372623 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:02.372224 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:42:02.377264 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:02.377242 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:42:02.754461 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:02.754433 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:42:02.799076 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:02.799043 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-696844548f-bmn2l"] Apr 16 08:42:04.729658 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:04.729626 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rqjhz" Apr 16 08:42:15.055977 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:15.055948 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54fdbc7b78-dxx97_f45d6895-a6e2-4531-ac1e-6e8c46c51012/router/0.log" Apr 16 08:42:15.067943 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:15.067923 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nlp9v_6136520a-182b-44c5-837e-1dd3dc6af3f7/serve-healthcheck-canary/0.log" Apr 16 08:42:17.055421 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.055368 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-79c5cdd649-xthk8" podUID="19866fa8-b487-41d7-9eba-73fe93fcb783" containerName="console" containerID="cri-o://301e891f71f668942ad556a3a9df4f9d5cab75e59fa8f84554425dab02712fce" gracePeriod=15 Apr 16 08:42:17.321082 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.321064 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-79c5cdd649-xthk8_19866fa8-b487-41d7-9eba-73fe93fcb783/console/0.log" Apr 16 08:42:17.321185 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.321123 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79c5cdd649-xthk8" Apr 16 08:42:17.458328 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.458294 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5rl7\" (UniqueName: \"kubernetes.io/projected/19866fa8-b487-41d7-9eba-73fe93fcb783-kube-api-access-z5rl7\") pod \"19866fa8-b487-41d7-9eba-73fe93fcb783\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " Apr 16 08:42:17.458328 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.458328 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19866fa8-b487-41d7-9eba-73fe93fcb783-console-config\") pod \"19866fa8-b487-41d7-9eba-73fe93fcb783\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " Apr 16 08:42:17.458556 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.458352 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19866fa8-b487-41d7-9eba-73fe93fcb783-service-ca\") pod \"19866fa8-b487-41d7-9eba-73fe93fcb783\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " Apr 16 08:42:17.458556 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.458367 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19866fa8-b487-41d7-9eba-73fe93fcb783-console-oauth-config\") pod \"19866fa8-b487-41d7-9eba-73fe93fcb783\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " Apr 16 08:42:17.458556 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.458386 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19866fa8-b487-41d7-9eba-73fe93fcb783-console-serving-cert\") pod \"19866fa8-b487-41d7-9eba-73fe93fcb783\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " Apr 16 08:42:17.458556 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.458415 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19866fa8-b487-41d7-9eba-73fe93fcb783-oauth-serving-cert\") pod \"19866fa8-b487-41d7-9eba-73fe93fcb783\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " Apr 16 08:42:17.458556 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.458473 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19866fa8-b487-41d7-9eba-73fe93fcb783-trusted-ca-bundle\") pod \"19866fa8-b487-41d7-9eba-73fe93fcb783\" (UID: \"19866fa8-b487-41d7-9eba-73fe93fcb783\") " Apr 16 08:42:17.458834 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.458802 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19866fa8-b487-41d7-9eba-73fe93fcb783-service-ca" (OuterVolumeSpecName: "service-ca") pod "19866fa8-b487-41d7-9eba-73fe93fcb783" (UID: "19866fa8-b487-41d7-9eba-73fe93fcb783"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 08:42:17.458884 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.458866 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19866fa8-b487-41d7-9eba-73fe93fcb783-console-config" (OuterVolumeSpecName: "console-config") pod "19866fa8-b487-41d7-9eba-73fe93fcb783" (UID: "19866fa8-b487-41d7-9eba-73fe93fcb783"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 08:42:17.458936 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.458914 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19866fa8-b487-41d7-9eba-73fe93fcb783-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "19866fa8-b487-41d7-9eba-73fe93fcb783" (UID: "19866fa8-b487-41d7-9eba-73fe93fcb783"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 08:42:17.459286 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.459258 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19866fa8-b487-41d7-9eba-73fe93fcb783-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "19866fa8-b487-41d7-9eba-73fe93fcb783" (UID: "19866fa8-b487-41d7-9eba-73fe93fcb783"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 08:42:17.460673 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.460648 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19866fa8-b487-41d7-9eba-73fe93fcb783-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "19866fa8-b487-41d7-9eba-73fe93fcb783" (UID: "19866fa8-b487-41d7-9eba-73fe93fcb783"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 08:42:17.460841 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.460691 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19866fa8-b487-41d7-9eba-73fe93fcb783-kube-api-access-z5rl7" (OuterVolumeSpecName: "kube-api-access-z5rl7") pod "19866fa8-b487-41d7-9eba-73fe93fcb783" (UID: "19866fa8-b487-41d7-9eba-73fe93fcb783"). InnerVolumeSpecName "kube-api-access-z5rl7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 08:42:17.460841 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.460746 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19866fa8-b487-41d7-9eba-73fe93fcb783-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "19866fa8-b487-41d7-9eba-73fe93fcb783" (UID: "19866fa8-b487-41d7-9eba-73fe93fcb783"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 08:42:17.559537 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.559474 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5rl7\" (UniqueName: \"kubernetes.io/projected/19866fa8-b487-41d7-9eba-73fe93fcb783-kube-api-access-z5rl7\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:42:17.561664 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.559823 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19866fa8-b487-41d7-9eba-73fe93fcb783-console-config\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:42:17.561664 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.559854 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19866fa8-b487-41d7-9eba-73fe93fcb783-service-ca\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:42:17.561664 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.559870 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19866fa8-b487-41d7-9eba-73fe93fcb783-console-oauth-config\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:42:17.561664 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.559943 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19866fa8-b487-41d7-9eba-73fe93fcb783-console-serving-cert\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:42:17.561664 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.559975 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19866fa8-b487-41d7-9eba-73fe93fcb783-oauth-serving-cert\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:42:17.561664 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.559992 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19866fa8-b487-41d7-9eba-73fe93fcb783-trusted-ca-bundle\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:42:17.792900 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.792873 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-79c5cdd649-xthk8_19866fa8-b487-41d7-9eba-73fe93fcb783/console/0.log" Apr 16 08:42:17.793034 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.792909 2578 generic.go:358] "Generic (PLEG): container finished" podID="19866fa8-b487-41d7-9eba-73fe93fcb783" containerID="301e891f71f668942ad556a3a9df4f9d5cab75e59fa8f84554425dab02712fce" exitCode=2 Apr 16 08:42:17.793034 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.792941 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79c5cdd649-xthk8" event={"ID":"19866fa8-b487-41d7-9eba-73fe93fcb783","Type":"ContainerDied","Data":"301e891f71f668942ad556a3a9df4f9d5cab75e59fa8f84554425dab02712fce"} Apr 16 08:42:17.793034 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.792981 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79c5cdd649-xthk8" event={"ID":"19866fa8-b487-41d7-9eba-73fe93fcb783","Type":"ContainerDied","Data":"eabe3e9341b6c62416316bd1b27ee8c073437f2a9859ea8c0f2ec419722a915b"} Apr 16 08:42:17.793034 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.792989 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79c5cdd649-xthk8" Apr 16 08:42:17.793034 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.792996 2578 scope.go:117] "RemoveContainer" containerID="301e891f71f668942ad556a3a9df4f9d5cab75e59fa8f84554425dab02712fce" Apr 16 08:42:17.811350 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.811331 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79c5cdd649-xthk8"] Apr 16 08:42:17.811426 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.811381 2578 scope.go:117] "RemoveContainer" containerID="301e891f71f668942ad556a3a9df4f9d5cab75e59fa8f84554425dab02712fce" Apr 16 08:42:17.811647 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:42:17.811625 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"301e891f71f668942ad556a3a9df4f9d5cab75e59fa8f84554425dab02712fce\": container with ID starting with 301e891f71f668942ad556a3a9df4f9d5cab75e59fa8f84554425dab02712fce not found: ID does not exist" containerID="301e891f71f668942ad556a3a9df4f9d5cab75e59fa8f84554425dab02712fce" Apr 16 08:42:17.811703 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.811657 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"301e891f71f668942ad556a3a9df4f9d5cab75e59fa8f84554425dab02712fce"} err="failed to get container status \"301e891f71f668942ad556a3a9df4f9d5cab75e59fa8f84554425dab02712fce\": rpc error: code = NotFound desc = could not find container \"301e891f71f668942ad556a3a9df4f9d5cab75e59fa8f84554425dab02712fce\": container with ID starting with 301e891f71f668942ad556a3a9df4f9d5cab75e59fa8f84554425dab02712fce not found: ID does not exist" Apr 16 08:42:17.814226 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:17.814206 2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-79c5cdd649-xthk8"] Apr 16 08:42:18.213776 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:18.213748 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19866fa8-b487-41d7-9eba-73fe93fcb783" path="/var/lib/kubelet/pods/19866fa8-b487-41d7-9eba-73fe93fcb783/volumes" Apr 16 08:42:27.818216 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:27.818154 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-696844548f-bmn2l" podUID="77058410-be22-4f4a-b977-81eaa77c149a" containerName="console" containerID="cri-o://0e4012b2a9db5b45d46bb87eec6769b6b7432d1b38f62bf8d68af5183804dfee" gracePeriod=15 Apr 16 08:42:28.097066 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.097044 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-696844548f-bmn2l_77058410-be22-4f4a-b977-81eaa77c149a/console/0.log" Apr 16 08:42:28.097181 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.097106 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:42:28.250569 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.250533 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77058410-be22-4f4a-b977-81eaa77c149a-trusted-ca-bundle\") pod \"77058410-be22-4f4a-b977-81eaa77c149a\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " Apr 16 08:42:28.250731 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.250593 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/77058410-be22-4f4a-b977-81eaa77c149a-console-serving-cert\") pod \"77058410-be22-4f4a-b977-81eaa77c149a\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " Apr 16 08:42:28.250731 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.250629 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/77058410-be22-4f4a-b977-81eaa77c149a-console-oauth-config\") pod \"77058410-be22-4f4a-b977-81eaa77c149a\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " Apr 16 08:42:28.250731 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.250691 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/77058410-be22-4f4a-b977-81eaa77c149a-service-ca\") pod \"77058410-be22-4f4a-b977-81eaa77c149a\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " Apr 16 08:42:28.250731 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.250717 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/77058410-be22-4f4a-b977-81eaa77c149a-console-config\") pod \"77058410-be22-4f4a-b977-81eaa77c149a\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " Apr 16 08:42:28.250967 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.250744 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tht9q\" (UniqueName: \"kubernetes.io/projected/77058410-be22-4f4a-b977-81eaa77c149a-kube-api-access-tht9q\") pod \"77058410-be22-4f4a-b977-81eaa77c149a\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " Apr 16 08:42:28.250967 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.250813 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/77058410-be22-4f4a-b977-81eaa77c149a-oauth-serving-cert\") pod \"77058410-be22-4f4a-b977-81eaa77c149a\" (UID: \"77058410-be22-4f4a-b977-81eaa77c149a\") " Apr 16 08:42:28.250967 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.250847 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77058410-be22-4f4a-b977-81eaa77c149a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "77058410-be22-4f4a-b977-81eaa77c149a" (UID: "77058410-be22-4f4a-b977-81eaa77c149a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 08:42:28.251117 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.251061 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77058410-be22-4f4a-b977-81eaa77c149a-trusted-ca-bundle\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:42:28.251117 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.251064 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77058410-be22-4f4a-b977-81eaa77c149a-console-config" (OuterVolumeSpecName: "console-config") pod "77058410-be22-4f4a-b977-81eaa77c149a" (UID: "77058410-be22-4f4a-b977-81eaa77c149a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 08:42:28.251362 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.251336 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77058410-be22-4f4a-b977-81eaa77c149a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "77058410-be22-4f4a-b977-81eaa77c149a" (UID: "77058410-be22-4f4a-b977-81eaa77c149a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 08:42:28.251512 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.251389 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77058410-be22-4f4a-b977-81eaa77c149a-service-ca" (OuterVolumeSpecName: "service-ca") pod "77058410-be22-4f4a-b977-81eaa77c149a" (UID: "77058410-be22-4f4a-b977-81eaa77c149a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 08:42:28.252923 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.252891 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77058410-be22-4f4a-b977-81eaa77c149a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "77058410-be22-4f4a-b977-81eaa77c149a" (UID: "77058410-be22-4f4a-b977-81eaa77c149a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 08:42:28.253011 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.252983 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77058410-be22-4f4a-b977-81eaa77c149a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "77058410-be22-4f4a-b977-81eaa77c149a" (UID: "77058410-be22-4f4a-b977-81eaa77c149a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 08:42:28.253066 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.253020 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77058410-be22-4f4a-b977-81eaa77c149a-kube-api-access-tht9q" (OuterVolumeSpecName: "kube-api-access-tht9q") pod "77058410-be22-4f4a-b977-81eaa77c149a" (UID: "77058410-be22-4f4a-b977-81eaa77c149a"). InnerVolumeSpecName "kube-api-access-tht9q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 08:42:28.352092 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.352004 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/77058410-be22-4f4a-b977-81eaa77c149a-console-serving-cert\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:42:28.352092 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.352043 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/77058410-be22-4f4a-b977-81eaa77c149a-console-oauth-config\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:42:28.352092 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.352060 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/77058410-be22-4f4a-b977-81eaa77c149a-service-ca\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:42:28.352092 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.352076 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/77058410-be22-4f4a-b977-81eaa77c149a-console-config\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:42:28.352092 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.352089 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tht9q\" (UniqueName: \"kubernetes.io/projected/77058410-be22-4f4a-b977-81eaa77c149a-kube-api-access-tht9q\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:42:28.352387 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.352103 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/77058410-be22-4f4a-b977-81eaa77c149a-oauth-serving-cert\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:42:28.825355 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.825329 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-696844548f-bmn2l_77058410-be22-4f4a-b977-81eaa77c149a/console/0.log" Apr 16 08:42:28.825801 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.825373 2578 generic.go:358] "Generic (PLEG): container finished" podID="77058410-be22-4f4a-b977-81eaa77c149a" containerID="0e4012b2a9db5b45d46bb87eec6769b6b7432d1b38f62bf8d68af5183804dfee" exitCode=2 Apr 16 08:42:28.825801 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.825438 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-696844548f-bmn2l" event={"ID":"77058410-be22-4f4a-b977-81eaa77c149a","Type":"ContainerDied","Data":"0e4012b2a9db5b45d46bb87eec6769b6b7432d1b38f62bf8d68af5183804dfee"} Apr 16 08:42:28.825801 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.825467 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-696844548f-bmn2l" Apr 16 08:42:28.825801 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.825479 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-696844548f-bmn2l" event={"ID":"77058410-be22-4f4a-b977-81eaa77c149a","Type":"ContainerDied","Data":"079691c65fd9f112b4648ec01697879e1a7dbdbc74c9ab9601de7c9603bcec42"} Apr 16 08:42:28.825801 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.825500 2578 scope.go:117] "RemoveContainer" containerID="0e4012b2a9db5b45d46bb87eec6769b6b7432d1b38f62bf8d68af5183804dfee" Apr 16 08:42:28.835670 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.835200 2578 scope.go:117] "RemoveContainer" containerID="0e4012b2a9db5b45d46bb87eec6769b6b7432d1b38f62bf8d68af5183804dfee" Apr 16 08:42:28.835670 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:42:28.835573 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e4012b2a9db5b45d46bb87eec6769b6b7432d1b38f62bf8d68af5183804dfee\": container with ID starting with 0e4012b2a9db5b45d46bb87eec6769b6b7432d1b38f62bf8d68af5183804dfee not found: ID does not exist" containerID="0e4012b2a9db5b45d46bb87eec6769b6b7432d1b38f62bf8d68af5183804dfee" Apr 16 08:42:28.835670 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.835609 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e4012b2a9db5b45d46bb87eec6769b6b7432d1b38f62bf8d68af5183804dfee"} err="failed to get container status \"0e4012b2a9db5b45d46bb87eec6769b6b7432d1b38f62bf8d68af5183804dfee\": rpc error: code = NotFound desc = could not find container \"0e4012b2a9db5b45d46bb87eec6769b6b7432d1b38f62bf8d68af5183804dfee\": container with ID starting with 0e4012b2a9db5b45d46bb87eec6769b6b7432d1b38f62bf8d68af5183804dfee not found: ID does not exist" Apr 16 08:42:28.854276 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.854250 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-696844548f-bmn2l"] Apr 16 08:42:28.858146 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:28.858122 2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-696844548f-bmn2l"] Apr 16 08:42:30.213865 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:30.213836 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77058410-be22-4f4a-b977-81eaa77c149a" path="/var/lib/kubelet/pods/77058410-be22-4f4a-b977-81eaa77c149a/volumes" Apr 16 08:42:33.842003 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:33.841971 2578 generic.go:358] "Generic (PLEG): container finished" podID="47490d5e-7b28-4469-acf3-5cd7e9637c19" containerID="5e352fc0369bd143a6ab660dda71fe9ca640dc9428bc64556c3f3c063e4a8c6f" exitCode=0 Apr 16 08:42:33.842402 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:33.842043 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-q8lbg" event={"ID":"47490d5e-7b28-4469-acf3-5cd7e9637c19","Type":"ContainerDied","Data":"5e352fc0369bd143a6ab660dda71fe9ca640dc9428bc64556c3f3c063e4a8c6f"} Apr 16 08:42:33.842402 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:33.842356 2578 scope.go:117] "RemoveContainer" containerID="5e352fc0369bd143a6ab660dda71fe9ca640dc9428bc64556c3f3c063e4a8c6f" Apr 16 08:42:34.846337 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:42:34.846304 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-q8lbg" event={"ID":"47490d5e-7b28-4469-acf3-5cd7e9637c19","Type":"ContainerStarted","Data":"65d00b1f8342bcd40637dc78ffb52002e7400d8834a0bb0c7d158edececda957"} Apr 16 08:43:06.972162 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:06.972128 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-7bfd7b5db7-xjtmm"] Apr 16 08:43:06.972653 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:06.972445 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77058410-be22-4f4a-b977-81eaa77c149a" containerName="console" Apr 16 08:43:06.972653 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:06.972457 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="77058410-be22-4f4a-b977-81eaa77c149a" containerName="console" Apr 16 08:43:06.972653 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:06.972474 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19866fa8-b487-41d7-9eba-73fe93fcb783" containerName="console" Apr 16 08:43:06.972653 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:06.972479 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="19866fa8-b487-41d7-9eba-73fe93fcb783" containerName="console" Apr 16 08:43:06.972653 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:06.972521 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="19866fa8-b487-41d7-9eba-73fe93fcb783" containerName="console" Apr 16 08:43:06.972653 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:06.972531 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="77058410-be22-4f4a-b977-81eaa77c149a" containerName="console" Apr 16 08:43:06.975351 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:06.975336 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:43:06.984447 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:06.984424 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bfd7b5db7-xjtmm"] Apr 16 08:43:07.062583 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:07.062550 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-console-oauth-config\") pod \"console-7bfd7b5db7-xjtmm\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:43:07.062583 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:07.062586 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-service-ca\") pod \"console-7bfd7b5db7-xjtmm\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:43:07.062824 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:07.062604 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpgrb\" (UniqueName: \"kubernetes.io/projected/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-kube-api-access-cpgrb\") pod \"console-7bfd7b5db7-xjtmm\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:43:07.062824 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:07.062631 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-console-serving-cert\") pod \"console-7bfd7b5db7-xjtmm\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:43:07.062824 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:07.062689 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-trusted-ca-bundle\") pod \"console-7bfd7b5db7-xjtmm\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:43:07.062824 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:07.062762 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-console-config\") pod \"console-7bfd7b5db7-xjtmm\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:43:07.062824 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:07.062803 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-oauth-serving-cert\") pod \"console-7bfd7b5db7-xjtmm\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:43:07.163120 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:07.163085 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-console-config\") pod \"console-7bfd7b5db7-xjtmm\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:43:07.163120 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:07.163121 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-oauth-serving-cert\") pod \"console-7bfd7b5db7-xjtmm\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:43:07.163317 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:07.163145 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-console-oauth-config\") pod \"console-7bfd7b5db7-xjtmm\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:43:07.163317 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:07.163172 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-service-ca\") pod \"console-7bfd7b5db7-xjtmm\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:43:07.163317 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:07.163199 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cpgrb\" (UniqueName: \"kubernetes.io/projected/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-kube-api-access-cpgrb\") pod \"console-7bfd7b5db7-xjtmm\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:43:07.163317 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:07.163246 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-console-serving-cert\") pod \"console-7bfd7b5db7-xjtmm\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:43:07.163317 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:07.163272 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-trusted-ca-bundle\") pod \"console-7bfd7b5db7-xjtmm\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:43:07.163877 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:07.163853 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-console-config\") pod \"console-7bfd7b5db7-xjtmm\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:43:07.163982 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:07.163936 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-oauth-serving-cert\") pod \"console-7bfd7b5db7-xjtmm\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:43:07.164086 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:07.164070 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-trusted-ca-bundle\") pod \"console-7bfd7b5db7-xjtmm\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:43:07.164224 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:07.164203 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-service-ca\") pod \"console-7bfd7b5db7-xjtmm\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:43:07.165704 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:07.165686 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-console-serving-cert\") pod \"console-7bfd7b5db7-xjtmm\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:43:07.165781 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:07.165728 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-console-oauth-config\") pod \"console-7bfd7b5db7-xjtmm\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:43:07.169930 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:07.169911 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpgrb\" (UniqueName: \"kubernetes.io/projected/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-kube-api-access-cpgrb\") pod \"console-7bfd7b5db7-xjtmm\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:43:07.285648 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:07.285554 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:43:07.398673 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:07.398641 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bfd7b5db7-xjtmm"] Apr 16 08:43:07.402361 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:43:07.402335 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb783e52a_8cd5_450a_aa10_e3eeb7a492ec.slice/crio-09c0111735bd1b886bc38769788b9a1efb51f31c0d61eba5eb29f21f0daba728 WatchSource:0}: Error finding container 09c0111735bd1b886bc38769788b9a1efb51f31c0d61eba5eb29f21f0daba728: Status 404 returned error can't find the container with id 09c0111735bd1b886bc38769788b9a1efb51f31c0d61eba5eb29f21f0daba728 Apr 16 08:43:07.944211 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:07.944176 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bfd7b5db7-xjtmm" event={"ID":"b783e52a-8cd5-450a-aa10-e3eeb7a492ec","Type":"ContainerStarted","Data":"6fa83b89679a1259bb1a6af23e7f7dd9733dcf5bbba12b3c80025797f196aaa5"} Apr 16 08:43:07.944211 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:07.944213 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bfd7b5db7-xjtmm" event={"ID":"b783e52a-8cd5-450a-aa10-e3eeb7a492ec","Type":"ContainerStarted","Data":"09c0111735bd1b886bc38769788b9a1efb51f31c0d61eba5eb29f21f0daba728"} Apr 16 08:43:07.959154 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:07.959104 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-7bfd7b5db7-xjtmm" podStartSLOduration=1.95908926 podStartE2EDuration="1.95908926s" podCreationTimestamp="2026-04-16 08:43:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 08:43:07.958307889 +0000 UTC m=+236.327291945" watchObservedRunningTime="2026-04-16 08:43:07.95908926 +0000 UTC m=+236.328073319" Apr 16 08:43:17.286577 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:17.286545 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:43:17.286577 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:17.286585 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:43:17.291827 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:17.291805 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:43:17.975258 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:17.975231 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:43:18.018132 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:18.018096 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64d9d6dc47-qlvzx"] Apr 16 08:43:27.761945 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:27.761873 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-zt4sj"] Apr 16 08:43:27.764995 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:27.764975 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zt4sj" Apr 16 08:43:27.766896 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:27.766876 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Apr 16 08:43:27.770475 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:27.770457 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zt4sj"] Apr 16 08:43:27.823691 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:27.823670 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4aa4e936-5865-4143-8d38-d2b97d7514b4-dbus\") pod \"global-pull-secret-syncer-zt4sj\" (UID: \"4aa4e936-5865-4143-8d38-d2b97d7514b4\") " pod="kube-system/global-pull-secret-syncer-zt4sj" Apr 16 08:43:27.823797 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:27.823697 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4aa4e936-5865-4143-8d38-d2b97d7514b4-kubelet-config\") pod \"global-pull-secret-syncer-zt4sj\" (UID: \"4aa4e936-5865-4143-8d38-d2b97d7514b4\") " pod="kube-system/global-pull-secret-syncer-zt4sj" Apr 16 08:43:27.823797 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:27.823727 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4aa4e936-5865-4143-8d38-d2b97d7514b4-original-pull-secret\") pod \"global-pull-secret-syncer-zt4sj\" (UID: \"4aa4e936-5865-4143-8d38-d2b97d7514b4\") " pod="kube-system/global-pull-secret-syncer-zt4sj" Apr 16 08:43:27.924232 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:27.924214 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4aa4e936-5865-4143-8d38-d2b97d7514b4-kubelet-config\") pod \"global-pull-secret-syncer-zt4sj\" (UID: \"4aa4e936-5865-4143-8d38-d2b97d7514b4\") " pod="kube-system/global-pull-secret-syncer-zt4sj" Apr 16 08:43:27.924314 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:27.924249 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4aa4e936-5865-4143-8d38-d2b97d7514b4-original-pull-secret\") pod \"global-pull-secret-syncer-zt4sj\" (UID: \"4aa4e936-5865-4143-8d38-d2b97d7514b4\") " pod="kube-system/global-pull-secret-syncer-zt4sj" Apr 16 08:43:27.924314 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:27.924302 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4aa4e936-5865-4143-8d38-d2b97d7514b4-dbus\") pod \"global-pull-secret-syncer-zt4sj\" (UID: \"4aa4e936-5865-4143-8d38-d2b97d7514b4\") " pod="kube-system/global-pull-secret-syncer-zt4sj" Apr 16 08:43:27.924396 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:27.924322 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4aa4e936-5865-4143-8d38-d2b97d7514b4-kubelet-config\") pod \"global-pull-secret-syncer-zt4sj\" (UID: \"4aa4e936-5865-4143-8d38-d2b97d7514b4\") " pod="kube-system/global-pull-secret-syncer-zt4sj" Apr 16 08:43:27.924442 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:27.924429 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4aa4e936-5865-4143-8d38-d2b97d7514b4-dbus\") pod \"global-pull-secret-syncer-zt4sj\" (UID: \"4aa4e936-5865-4143-8d38-d2b97d7514b4\") " pod="kube-system/global-pull-secret-syncer-zt4sj" Apr 16 08:43:27.926540 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:27.926520 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4aa4e936-5865-4143-8d38-d2b97d7514b4-original-pull-secret\") pod \"global-pull-secret-syncer-zt4sj\" (UID: \"4aa4e936-5865-4143-8d38-d2b97d7514b4\") " pod="kube-system/global-pull-secret-syncer-zt4sj" Apr 16 08:43:28.075093 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:28.075044 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zt4sj" Apr 16 08:43:28.216484 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:28.216456 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zt4sj"] Apr 16 08:43:28.219579 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:43:28.219553 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aa4e936_5865_4143_8d38_d2b97d7514b4.slice/crio-1034239d9bbfc77f25efc9db8ffb4ef0f046426b3959d48db3110192afa2ac3e WatchSource:0}: Error finding container 1034239d9bbfc77f25efc9db8ffb4ef0f046426b3959d48db3110192afa2ac3e: Status 404 returned error can't find the container with id 1034239d9bbfc77f25efc9db8ffb4ef0f046426b3959d48db3110192afa2ac3e Apr 16 08:43:29.001903 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:29.001863 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zt4sj" event={"ID":"4aa4e936-5865-4143-8d38-d2b97d7514b4","Type":"ContainerStarted","Data":"1034239d9bbfc77f25efc9db8ffb4ef0f046426b3959d48db3110192afa2ac3e"} Apr 16 08:43:32.011830 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:32.011785 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zt4sj" event={"ID":"4aa4e936-5865-4143-8d38-d2b97d7514b4","Type":"ContainerStarted","Data":"fc7a62d1821248615fe31c6e221e06044a17e8e05e1cefbae2f407c0a0dd9e6f"} Apr 16 08:43:32.026586 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:32.026536 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zt4sj" podStartSLOduration=1.4521739550000001 podStartE2EDuration="5.026522025s" podCreationTimestamp="2026-04-16 08:43:27 +0000 UTC" firstStartedPulling="2026-04-16 08:43:28.221072672 +0000 UTC m=+256.590056707" lastFinishedPulling="2026-04-16 08:43:31.795420741 +0000 UTC m=+260.164404777" observedRunningTime="2026-04-16 08:43:32.024593724 +0000 UTC m=+260.393577779" watchObservedRunningTime="2026-04-16 08:43:32.026522025 +0000 UTC m=+260.395506081" Apr 16 08:43:43.039969 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:43.039911 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-64d9d6dc47-qlvzx" podUID="6ac4bc10-99f5-45c4-897f-6e82d1fa074f" containerName="console" containerID="cri-o://7fc89162911b77ed539019383be12394c0d4cb9af2accc96b79fa9cfe73e2bdf" gracePeriod=15 Apr 16 08:43:43.264559 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:43.264538 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-64d9d6dc47-qlvzx_6ac4bc10-99f5-45c4-897f-6e82d1fa074f/console/0.log" Apr 16 08:43:43.264659 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:43.264597 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:43:43.342216 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:43.342133 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-service-ca\") pod \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " Apr 16 08:43:43.342216 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:43.342181 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-console-oauth-config\") pod \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " Apr 16 08:43:43.342216 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:43.342213 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-console-serving-cert\") pod \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " Apr 16 08:43:43.342477 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:43.342297 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2t278\" (UniqueName: \"kubernetes.io/projected/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-kube-api-access-2t278\") pod \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " Apr 16 08:43:43.342477 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:43.342329 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-oauth-serving-cert\") pod \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " Apr 16 08:43:43.342477 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:43.342368 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-trusted-ca-bundle\") pod \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " Apr 16 08:43:43.342477 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:43.342406 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-console-config\") pod \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\" (UID: \"6ac4bc10-99f5-45c4-897f-6e82d1fa074f\") " Apr 16 08:43:43.342671 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:43.342492 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-service-ca" (OuterVolumeSpecName: "service-ca") pod "6ac4bc10-99f5-45c4-897f-6e82d1fa074f" (UID: "6ac4bc10-99f5-45c4-897f-6e82d1fa074f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 08:43:43.342831 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:43.342806 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-service-ca\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:43:43.342921 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:43.342829 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6ac4bc10-99f5-45c4-897f-6e82d1fa074f" (UID: "6ac4bc10-99f5-45c4-897f-6e82d1fa074f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 08:43:43.342921 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:43.342869 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-console-config" (OuterVolumeSpecName: "console-config") pod "6ac4bc10-99f5-45c4-897f-6e82d1fa074f" (UID: "6ac4bc10-99f5-45c4-897f-6e82d1fa074f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 08:43:43.343190 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:43.342920 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6ac4bc10-99f5-45c4-897f-6e82d1fa074f" (UID: "6ac4bc10-99f5-45c4-897f-6e82d1fa074f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 08:43:43.344629 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:43.344608 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6ac4bc10-99f5-45c4-897f-6e82d1fa074f" (UID: "6ac4bc10-99f5-45c4-897f-6e82d1fa074f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 08:43:43.344732 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:43.344717 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-kube-api-access-2t278" (OuterVolumeSpecName: "kube-api-access-2t278") pod "6ac4bc10-99f5-45c4-897f-6e82d1fa074f" (UID: "6ac4bc10-99f5-45c4-897f-6e82d1fa074f"). InnerVolumeSpecName "kube-api-access-2t278". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 08:43:43.344797 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:43.344713 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6ac4bc10-99f5-45c4-897f-6e82d1fa074f" (UID: "6ac4bc10-99f5-45c4-897f-6e82d1fa074f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 08:43:43.444109 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:43.444077 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-console-oauth-config\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:43:43.444109 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:43.444106 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-console-serving-cert\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:43:43.444109 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:43.444116 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2t278\" (UniqueName: \"kubernetes.io/projected/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-kube-api-access-2t278\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:43:43.444324 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:43.444126 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-oauth-serving-cert\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:43:43.444324 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:43.444136 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-trusted-ca-bundle\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:43:43.444324 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:43.444145 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ac4bc10-99f5-45c4-897f-6e82d1fa074f-console-config\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:43:44.046054 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:44.046027 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-64d9d6dc47-qlvzx_6ac4bc10-99f5-45c4-897f-6e82d1fa074f/console/0.log" Apr 16 08:43:44.046399 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:44.046063 2578 generic.go:358] "Generic (PLEG): container finished" podID="6ac4bc10-99f5-45c4-897f-6e82d1fa074f" containerID="7fc89162911b77ed539019383be12394c0d4cb9af2accc96b79fa9cfe73e2bdf" exitCode=2 Apr 16 08:43:44.046399 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:44.046150 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64d9d6dc47-qlvzx" event={"ID":"6ac4bc10-99f5-45c4-897f-6e82d1fa074f","Type":"ContainerDied","Data":"7fc89162911b77ed539019383be12394c0d4cb9af2accc96b79fa9cfe73e2bdf"} Apr 16 08:43:44.046399 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:44.046189 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64d9d6dc47-qlvzx" event={"ID":"6ac4bc10-99f5-45c4-897f-6e82d1fa074f","Type":"ContainerDied","Data":"4f161c3e10e3b17e1f9cefeb2f0e5cf87b4b7ed72ac6099c5e76e5b334e174ac"} Apr 16 08:43:44.046399 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:44.046203 2578 scope.go:117] "RemoveContainer" containerID="7fc89162911b77ed539019383be12394c0d4cb9af2accc96b79fa9cfe73e2bdf" Apr 16 08:43:44.046399 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:44.046161 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64d9d6dc47-qlvzx" Apr 16 08:43:44.054695 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:44.054675 2578 scope.go:117] "RemoveContainer" containerID="7fc89162911b77ed539019383be12394c0d4cb9af2accc96b79fa9cfe73e2bdf" Apr 16 08:43:44.054950 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:43:44.054934 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fc89162911b77ed539019383be12394c0d4cb9af2accc96b79fa9cfe73e2bdf\": container with ID starting with 7fc89162911b77ed539019383be12394c0d4cb9af2accc96b79fa9cfe73e2bdf not found: ID does not exist" containerID="7fc89162911b77ed539019383be12394c0d4cb9af2accc96b79fa9cfe73e2bdf" Apr 16 08:43:44.055021 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:44.054961 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fc89162911b77ed539019383be12394c0d4cb9af2accc96b79fa9cfe73e2bdf"} err="failed to get container status \"7fc89162911b77ed539019383be12394c0d4cb9af2accc96b79fa9cfe73e2bdf\": rpc error: code = NotFound desc = could not find container \"7fc89162911b77ed539019383be12394c0d4cb9af2accc96b79fa9cfe73e2bdf\": container with ID starting with 7fc89162911b77ed539019383be12394c0d4cb9af2accc96b79fa9cfe73e2bdf not found: ID does not exist" Apr 16 08:43:44.064895 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:44.064870 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64d9d6dc47-qlvzx"] Apr 16 08:43:44.070175 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:44.070152 2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64d9d6dc47-qlvzx"] Apr 16 08:43:44.213975 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:43:44.213938 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ac4bc10-99f5-45c4-897f-6e82d1fa074f" path="/var/lib/kubelet/pods/6ac4bc10-99f5-45c4-897f-6e82d1fa074f/volumes" Apr 16 08:44:12.116103 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:12.116080 2578 kubelet.go:1628] "Image garbage collection succeeded" Apr 16 08:44:17.862860 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:17.862826 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw"] Apr 16 08:44:17.865250 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:17.863130 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ac4bc10-99f5-45c4-897f-6e82d1fa074f" containerName="console" Apr 16 08:44:17.865250 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:17.863142 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ac4bc10-99f5-45c4-897f-6e82d1fa074f" containerName="console" Apr 16 08:44:17.865250 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:17.863210 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="6ac4bc10-99f5-45c4-897f-6e82d1fa074f" containerName="console" Apr 16 08:44:17.866145 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:17.866130 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw" Apr 16 08:44:17.868400 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:17.868380 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tlgdk\"" Apr 16 08:44:17.868595 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:17.868583 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Apr 16 08:44:17.869333 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:17.869319 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Apr 16 08:44:17.874032 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:17.874013 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw"] Apr 16 08:44:17.914409 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:17.914382 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxhgq\" (UniqueName: \"kubernetes.io/projected/ff5b6672-25e1-475a-b1a5-b2fa8a6021b0-kube-api-access-rxhgq\") pod \"925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw\" (UID: \"ff5b6672-25e1-475a-b1a5-b2fa8a6021b0\") " pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw" Apr 16 08:44:17.914562 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:17.914415 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff5b6672-25e1-475a-b1a5-b2fa8a6021b0-bundle\") pod \"925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw\" (UID: \"ff5b6672-25e1-475a-b1a5-b2fa8a6021b0\") " pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw" Apr 16 08:44:17.914562 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:17.914433 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff5b6672-25e1-475a-b1a5-b2fa8a6021b0-util\") pod \"925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw\" (UID: \"ff5b6672-25e1-475a-b1a5-b2fa8a6021b0\") " pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw" Apr 16 08:44:18.015726 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:18.015689 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rxhgq\" (UniqueName: \"kubernetes.io/projected/ff5b6672-25e1-475a-b1a5-b2fa8a6021b0-kube-api-access-rxhgq\") pod \"925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw\" (UID: \"ff5b6672-25e1-475a-b1a5-b2fa8a6021b0\") " pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw" Apr 16 08:44:18.015726 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:18.015729 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff5b6672-25e1-475a-b1a5-b2fa8a6021b0-bundle\") pod \"925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw\" (UID: \"ff5b6672-25e1-475a-b1a5-b2fa8a6021b0\") " pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw" Apr 16 08:44:18.015986 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:18.015751 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff5b6672-25e1-475a-b1a5-b2fa8a6021b0-util\") pod \"925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw\" (UID: \"ff5b6672-25e1-475a-b1a5-b2fa8a6021b0\") " pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw" Apr 16 08:44:18.016122 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:18.016098 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff5b6672-25e1-475a-b1a5-b2fa8a6021b0-bundle\") pod \"925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw\" (UID: \"ff5b6672-25e1-475a-b1a5-b2fa8a6021b0\") " pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw" Apr 16 08:44:18.016190 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:18.016146 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff5b6672-25e1-475a-b1a5-b2fa8a6021b0-util\") pod \"925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw\" (UID: \"ff5b6672-25e1-475a-b1a5-b2fa8a6021b0\") " pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw" Apr 16 08:44:18.023718 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:18.023691 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxhgq\" (UniqueName: \"kubernetes.io/projected/ff5b6672-25e1-475a-b1a5-b2fa8a6021b0-kube-api-access-rxhgq\") pod \"925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw\" (UID: \"ff5b6672-25e1-475a-b1a5-b2fa8a6021b0\") " pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw" Apr 16 08:44:18.175015 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:18.174941 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw" Apr 16 08:44:18.296608 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:18.296578 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw"] Apr 16 08:44:18.303261 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:18.303243 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 16 08:44:19.148122 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:19.148083 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw" event={"ID":"ff5b6672-25e1-475a-b1a5-b2fa8a6021b0","Type":"ContainerStarted","Data":"1bfb5e957d9bfc7d203c74ffc45e3660fecc5aee1336c139b7a9e84133714cce"} Apr 16 08:44:24.169711 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:24.169671 2578 generic.go:358] "Generic (PLEG): container finished" podID="ff5b6672-25e1-475a-b1a5-b2fa8a6021b0" containerID="45d8b34abe090dfd4c2b66a2d68276b29abee7fc9043db9c6d5b53dabd128668" exitCode=0 Apr 16 08:44:24.170211 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:24.169758 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw" event={"ID":"ff5b6672-25e1-475a-b1a5-b2fa8a6021b0","Type":"ContainerDied","Data":"45d8b34abe090dfd4c2b66a2d68276b29abee7fc9043db9c6d5b53dabd128668"} Apr 16 08:44:33.200194 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:33.200154 2578 generic.go:358] "Generic (PLEG): container finished" podID="ff5b6672-25e1-475a-b1a5-b2fa8a6021b0" containerID="96e5d70778ffa9de107637d5acc1f0e346487f486b49c045141a5a74f76c8c97" exitCode=0 Apr 16 08:44:33.200584 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:33.200231 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw" event={"ID":"ff5b6672-25e1-475a-b1a5-b2fa8a6021b0","Type":"ContainerDied","Data":"96e5d70778ffa9de107637d5acc1f0e346487f486b49c045141a5a74f76c8c97"} Apr 16 08:44:39.223311 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:39.223278 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw" event={"ID":"ff5b6672-25e1-475a-b1a5-b2fa8a6021b0","Type":"ContainerStarted","Data":"030ed45d2b61734d22bbf9b0264642108b6daaa13008429c2a4fb48cfc6c5764"} Apr 16 08:44:39.240380 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:39.240333 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw" podStartSLOduration=1.495846874 podStartE2EDuration="22.240314206s" podCreationTimestamp="2026-04-16 08:44:17 +0000 UTC" firstStartedPulling="2026-04-16 08:44:18.30336124 +0000 UTC m=+306.672345276" lastFinishedPulling="2026-04-16 08:44:39.047828559 +0000 UTC m=+327.416812608" observedRunningTime="2026-04-16 08:44:39.239039122 +0000 UTC m=+327.608023178" watchObservedRunningTime="2026-04-16 08:44:39.240314206 +0000 UTC m=+327.609298263" Apr 16 08:44:40.227436 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:40.227401 2578 generic.go:358] "Generic (PLEG): container finished" podID="ff5b6672-25e1-475a-b1a5-b2fa8a6021b0" containerID="030ed45d2b61734d22bbf9b0264642108b6daaa13008429c2a4fb48cfc6c5764" exitCode=0 Apr 16 08:44:40.227818 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:40.227487 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw" event={"ID":"ff5b6672-25e1-475a-b1a5-b2fa8a6021b0","Type":"ContainerDied","Data":"030ed45d2b61734d22bbf9b0264642108b6daaa13008429c2a4fb48cfc6c5764"} Apr 16 08:44:41.350062 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:41.350040 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw" Apr 16 08:44:41.413093 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:41.413061 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxhgq\" (UniqueName: \"kubernetes.io/projected/ff5b6672-25e1-475a-b1a5-b2fa8a6021b0-kube-api-access-rxhgq\") pod \"ff5b6672-25e1-475a-b1a5-b2fa8a6021b0\" (UID: \"ff5b6672-25e1-475a-b1a5-b2fa8a6021b0\") " Apr 16 08:44:41.413093 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:41.413101 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff5b6672-25e1-475a-b1a5-b2fa8a6021b0-util\") pod \"ff5b6672-25e1-475a-b1a5-b2fa8a6021b0\" (UID: \"ff5b6672-25e1-475a-b1a5-b2fa8a6021b0\") " Apr 16 08:44:41.413285 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:41.413169 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff5b6672-25e1-475a-b1a5-b2fa8a6021b0-bundle\") pod \"ff5b6672-25e1-475a-b1a5-b2fa8a6021b0\" (UID: \"ff5b6672-25e1-475a-b1a5-b2fa8a6021b0\") " Apr 16 08:44:41.413749 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:41.413724 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff5b6672-25e1-475a-b1a5-b2fa8a6021b0-bundle" (OuterVolumeSpecName: "bundle") pod "ff5b6672-25e1-475a-b1a5-b2fa8a6021b0" (UID: "ff5b6672-25e1-475a-b1a5-b2fa8a6021b0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 08:44:41.415370 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:41.415346 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff5b6672-25e1-475a-b1a5-b2fa8a6021b0-kube-api-access-rxhgq" (OuterVolumeSpecName: "kube-api-access-rxhgq") pod "ff5b6672-25e1-475a-b1a5-b2fa8a6021b0" (UID: "ff5b6672-25e1-475a-b1a5-b2fa8a6021b0"). InnerVolumeSpecName "kube-api-access-rxhgq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 08:44:41.417071 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:41.417049 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff5b6672-25e1-475a-b1a5-b2fa8a6021b0-util" (OuterVolumeSpecName: "util") pod "ff5b6672-25e1-475a-b1a5-b2fa8a6021b0" (UID: "ff5b6672-25e1-475a-b1a5-b2fa8a6021b0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 08:44:41.514409 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:41.514321 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxhgq\" (UniqueName: \"kubernetes.io/projected/ff5b6672-25e1-475a-b1a5-b2fa8a6021b0-kube-api-access-rxhgq\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:44:41.514409 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:41.514350 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff5b6672-25e1-475a-b1a5-b2fa8a6021b0-util\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:44:41.514409 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:41.514360 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff5b6672-25e1-475a-b1a5-b2fa8a6021b0-bundle\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:44:42.236115 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:42.236082 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw" event={"ID":"ff5b6672-25e1-475a-b1a5-b2fa8a6021b0","Type":"ContainerDied","Data":"1bfb5e957d9bfc7d203c74ffc45e3660fecc5aee1336c139b7a9e84133714cce"} Apr 16 08:44:42.236115 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:42.236103 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e54qslw" Apr 16 08:44:42.236115 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:42.236115 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1bfb5e957d9bfc7d203c74ffc45e3660fecc5aee1336c139b7a9e84133714cce" Apr 16 08:44:45.159480 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:45.159441 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-dqwvk"] Apr 16 08:44:45.160060 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:45.159895 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff5b6672-25e1-475a-b1a5-b2fa8a6021b0" containerName="util" Apr 16 08:44:45.160060 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:45.159912 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff5b6672-25e1-475a-b1a5-b2fa8a6021b0" containerName="util" Apr 16 08:44:45.160060 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:45.159932 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff5b6672-25e1-475a-b1a5-b2fa8a6021b0" containerName="pull" Apr 16 08:44:45.160060 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:45.159940 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff5b6672-25e1-475a-b1a5-b2fa8a6021b0" containerName="pull" Apr 16 08:44:45.160060 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:45.159962 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff5b6672-25e1-475a-b1a5-b2fa8a6021b0" containerName="extract" Apr 16 08:44:45.160060 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:45.159970 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff5b6672-25e1-475a-b1a5-b2fa8a6021b0" containerName="extract" Apr 16 08:44:45.160060 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:45.160051 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="ff5b6672-25e1-475a-b1a5-b2fa8a6021b0" containerName="extract" Apr 16 08:44:45.164435 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:45.164415 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-dqwvk" Apr 16 08:44:45.166644 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:45.166615 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-2dxn9\"" Apr 16 08:44:45.166803 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:45.166627 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Apr 16 08:44:45.166857 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:45.166801 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Apr 16 08:44:45.172837 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:45.172812 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-dqwvk"] Apr 16 08:44:45.244522 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:45.244477 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwqps\" (UniqueName: \"kubernetes.io/projected/90080e54-2c70-4972-96d6-c00eec7702b2-kube-api-access-dwqps\") pod \"cert-manager-operator-controller-manager-7ccfb878b5-dqwvk\" (UID: \"90080e54-2c70-4972-96d6-c00eec7702b2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-dqwvk" Apr 16 08:44:45.244720 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:45.244621 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90080e54-2c70-4972-96d6-c00eec7702b2-tmp\") pod \"cert-manager-operator-controller-manager-7ccfb878b5-dqwvk\" (UID: \"90080e54-2c70-4972-96d6-c00eec7702b2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-dqwvk" Apr 16 08:44:45.345004 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:45.344958 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dwqps\" (UniqueName: \"kubernetes.io/projected/90080e54-2c70-4972-96d6-c00eec7702b2-kube-api-access-dwqps\") pod \"cert-manager-operator-controller-manager-7ccfb878b5-dqwvk\" (UID: \"90080e54-2c70-4972-96d6-c00eec7702b2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-dqwvk" Apr 16 08:44:45.345182 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:45.345138 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90080e54-2c70-4972-96d6-c00eec7702b2-tmp\") pod \"cert-manager-operator-controller-manager-7ccfb878b5-dqwvk\" (UID: \"90080e54-2c70-4972-96d6-c00eec7702b2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-dqwvk" Apr 16 08:44:45.345545 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:45.345526 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90080e54-2c70-4972-96d6-c00eec7702b2-tmp\") pod \"cert-manager-operator-controller-manager-7ccfb878b5-dqwvk\" (UID: \"90080e54-2c70-4972-96d6-c00eec7702b2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-dqwvk" Apr 16 08:44:45.352648 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:45.352625 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwqps\" (UniqueName: \"kubernetes.io/projected/90080e54-2c70-4972-96d6-c00eec7702b2-kube-api-access-dwqps\") pod \"cert-manager-operator-controller-manager-7ccfb878b5-dqwvk\" (UID: \"90080e54-2c70-4972-96d6-c00eec7702b2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-dqwvk" Apr 16 08:44:45.474082 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:45.474057 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-dqwvk" Apr 16 08:44:45.602127 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:45.602103 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-dqwvk"] Apr 16 08:44:45.604737 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:44:45.604707 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90080e54_2c70_4972_96d6_c00eec7702b2.slice/crio-c8250b0e1535efc74ba69f85c1d7999f2055d749396046b5ff38c4bdfeb14324 WatchSource:0}: Error finding container c8250b0e1535efc74ba69f85c1d7999f2055d749396046b5ff38c4bdfeb14324: Status 404 returned error can't find the container with id c8250b0e1535efc74ba69f85c1d7999f2055d749396046b5ff38c4bdfeb14324 Apr 16 08:44:46.251335 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:46.251301 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-dqwvk" event={"ID":"90080e54-2c70-4972-96d6-c00eec7702b2","Type":"ContainerStarted","Data":"c8250b0e1535efc74ba69f85c1d7999f2055d749396046b5ff38c4bdfeb14324"} Apr 16 08:44:48.259447 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:48.259411 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-dqwvk" event={"ID":"90080e54-2c70-4972-96d6-c00eec7702b2","Type":"ContainerStarted","Data":"cc493788fbcc1130493395e5709d9976bde4e2c0b436a8f7541ce234a6cb9584"} Apr 16 08:44:48.279019 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:48.278962 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-dqwvk" podStartSLOduration=1.4342926409999999 podStartE2EDuration="3.278944237s" podCreationTimestamp="2026-04-16 08:44:45 +0000 UTC" firstStartedPulling="2026-04-16 08:44:45.607179274 +0000 UTC m=+333.976163309" lastFinishedPulling="2026-04-16 08:44:47.451830869 +0000 UTC m=+335.820814905" observedRunningTime="2026-04-16 08:44:48.277257696 +0000 UTC m=+336.646241749" watchObservedRunningTime="2026-04-16 08:44:48.278944237 +0000 UTC m=+336.647928295" Apr 16 08:44:49.525750 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:49.525712 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7"] Apr 16 08:44:49.529300 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:49.529279 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7" Apr 16 08:44:49.531430 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:49.531404 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Apr 16 08:44:49.531585 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:49.531461 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tlgdk\"" Apr 16 08:44:49.531585 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:49.531519 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Apr 16 08:44:49.537604 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:49.537570 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7"] Apr 16 08:44:49.681117 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:49.681080 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8fdf0577-bb86-4e62-836a-62de5559009a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7\" (UID: \"8fdf0577-bb86-4e62-836a-62de5559009a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7" Apr 16 08:44:49.681117 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:49.681117 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8fdf0577-bb86-4e62-836a-62de5559009a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7\" (UID: \"8fdf0577-bb86-4e62-836a-62de5559009a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7" Apr 16 08:44:49.681343 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:49.681139 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6vz9\" (UniqueName: \"kubernetes.io/projected/8fdf0577-bb86-4e62-836a-62de5559009a-kube-api-access-w6vz9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7\" (UID: \"8fdf0577-bb86-4e62-836a-62de5559009a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7" Apr 16 08:44:49.782415 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:49.782319 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8fdf0577-bb86-4e62-836a-62de5559009a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7\" (UID: \"8fdf0577-bb86-4e62-836a-62de5559009a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7" Apr 16 08:44:49.782415 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:49.782358 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8fdf0577-bb86-4e62-836a-62de5559009a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7\" (UID: \"8fdf0577-bb86-4e62-836a-62de5559009a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7" Apr 16 08:44:49.782415 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:49.782375 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w6vz9\" (UniqueName: \"kubernetes.io/projected/8fdf0577-bb86-4e62-836a-62de5559009a-kube-api-access-w6vz9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7\" (UID: \"8fdf0577-bb86-4e62-836a-62de5559009a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7" Apr 16 08:44:49.782742 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:49.782720 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8fdf0577-bb86-4e62-836a-62de5559009a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7\" (UID: \"8fdf0577-bb86-4e62-836a-62de5559009a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7" Apr 16 08:44:49.782847 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:49.782803 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8fdf0577-bb86-4e62-836a-62de5559009a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7\" (UID: \"8fdf0577-bb86-4e62-836a-62de5559009a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7" Apr 16 08:44:49.790503 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:49.790469 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6vz9\" (UniqueName: \"kubernetes.io/projected/8fdf0577-bb86-4e62-836a-62de5559009a-kube-api-access-w6vz9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7\" (UID: \"8fdf0577-bb86-4e62-836a-62de5559009a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7" Apr 16 08:44:49.839327 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:49.839274 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7" Apr 16 08:44:49.961930 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:49.961897 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7"] Apr 16 08:44:49.964881 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:44:49.964848 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fdf0577_bb86_4e62_836a_62de5559009a.slice/crio-492df52ed9cf5ce9e47afdbec48805198e1fd7ca8a21e17ec9d41c9f18fe0ddd WatchSource:0}: Error finding container 492df52ed9cf5ce9e47afdbec48805198e1fd7ca8a21e17ec9d41c9f18fe0ddd: Status 404 returned error can't find the container with id 492df52ed9cf5ce9e47afdbec48805198e1fd7ca8a21e17ec9d41c9f18fe0ddd Apr 16 08:44:50.266626 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:50.266595 2578 generic.go:358] "Generic (PLEG): container finished" podID="8fdf0577-bb86-4e62-836a-62de5559009a" containerID="175bc318df3859871b85083491b20f25910494b813787543329b45e03f6b5495" exitCode=0 Apr 16 08:44:50.266820 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:50.266687 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7" event={"ID":"8fdf0577-bb86-4e62-836a-62de5559009a","Type":"ContainerDied","Data":"175bc318df3859871b85083491b20f25910494b813787543329b45e03f6b5495"} Apr 16 08:44:50.266820 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:50.266723 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7" event={"ID":"8fdf0577-bb86-4e62-836a-62de5559009a","Type":"ContainerStarted","Data":"492df52ed9cf5ce9e47afdbec48805198e1fd7ca8a21e17ec9d41c9f18fe0ddd"} Apr 16 08:44:51.418191 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:51.418158 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-597b96b99b-jb8qk"] Apr 16 08:44:51.421655 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:51.421631 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-597b96b99b-jb8qk" Apr 16 08:44:51.423676 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:51.423651 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Apr 16 08:44:51.423833 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:51.423713 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Apr 16 08:44:51.424227 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:51.424207 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-t8qnh\"" Apr 16 08:44:51.428505 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:51.428481 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-597b96b99b-jb8qk"] Apr 16 08:44:51.497217 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:51.497179 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwxnv\" (UniqueName: \"kubernetes.io/projected/4c73fa39-5fbc-4c7f-918c-ac5bd1c30ef7-kube-api-access-fwxnv\") pod \"cert-manager-webhook-597b96b99b-jb8qk\" (UID: \"4c73fa39-5fbc-4c7f-918c-ac5bd1c30ef7\") " pod="cert-manager/cert-manager-webhook-597b96b99b-jb8qk" Apr 16 08:44:51.497392 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:51.497264 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4c73fa39-5fbc-4c7f-918c-ac5bd1c30ef7-bound-sa-token\") pod \"cert-manager-webhook-597b96b99b-jb8qk\" (UID: \"4c73fa39-5fbc-4c7f-918c-ac5bd1c30ef7\") " pod="cert-manager/cert-manager-webhook-597b96b99b-jb8qk" Apr 16 08:44:51.598510 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:51.598476 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4c73fa39-5fbc-4c7f-918c-ac5bd1c30ef7-bound-sa-token\") pod \"cert-manager-webhook-597b96b99b-jb8qk\" (UID: \"4c73fa39-5fbc-4c7f-918c-ac5bd1c30ef7\") " pod="cert-manager/cert-manager-webhook-597b96b99b-jb8qk" Apr 16 08:44:51.598658 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:51.598554 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fwxnv\" (UniqueName: \"kubernetes.io/projected/4c73fa39-5fbc-4c7f-918c-ac5bd1c30ef7-kube-api-access-fwxnv\") pod \"cert-manager-webhook-597b96b99b-jb8qk\" (UID: \"4c73fa39-5fbc-4c7f-918c-ac5bd1c30ef7\") " pod="cert-manager/cert-manager-webhook-597b96b99b-jb8qk" Apr 16 08:44:51.606264 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:51.606233 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4c73fa39-5fbc-4c7f-918c-ac5bd1c30ef7-bound-sa-token\") pod \"cert-manager-webhook-597b96b99b-jb8qk\" (UID: \"4c73fa39-5fbc-4c7f-918c-ac5bd1c30ef7\") " pod="cert-manager/cert-manager-webhook-597b96b99b-jb8qk" Apr 16 08:44:51.606433 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:51.606417 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwxnv\" (UniqueName: \"kubernetes.io/projected/4c73fa39-5fbc-4c7f-918c-ac5bd1c30ef7-kube-api-access-fwxnv\") pod \"cert-manager-webhook-597b96b99b-jb8qk\" (UID: \"4c73fa39-5fbc-4c7f-918c-ac5bd1c30ef7\") " pod="cert-manager/cert-manager-webhook-597b96b99b-jb8qk" Apr 16 08:44:51.740668 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:51.740628 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-597b96b99b-jb8qk" Apr 16 08:44:52.363363 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:52.363313 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-597b96b99b-jb8qk"] Apr 16 08:44:52.365650 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:44:52.365624 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c73fa39_5fbc_4c7f_918c_ac5bd1c30ef7.slice/crio-46dd390b06040ca54f4bfb3a36b4c4c4a82f6a8667682799cf6ae51c0ac1f706 WatchSource:0}: Error finding container 46dd390b06040ca54f4bfb3a36b4c4c4a82f6a8667682799cf6ae51c0ac1f706: Status 404 returned error can't find the container with id 46dd390b06040ca54f4bfb3a36b4c4c4a82f6a8667682799cf6ae51c0ac1f706 Apr 16 08:44:53.279446 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:53.279402 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-597b96b99b-jb8qk" event={"ID":"4c73fa39-5fbc-4c7f-918c-ac5bd1c30ef7","Type":"ContainerStarted","Data":"46dd390b06040ca54f4bfb3a36b4c4c4a82f6a8667682799cf6ae51c0ac1f706"} Apr 16 08:44:53.281105 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:53.281074 2578 generic.go:358] "Generic (PLEG): container finished" podID="8fdf0577-bb86-4e62-836a-62de5559009a" containerID="689282888d8dbb9ee6f9e4353617d135182530e7e5f11946d4a00c003a547358" exitCode=0 Apr 16 08:44:53.281238 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:53.281131 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7" event={"ID":"8fdf0577-bb86-4e62-836a-62de5559009a","Type":"ContainerDied","Data":"689282888d8dbb9ee6f9e4353617d135182530e7e5f11946d4a00c003a547358"} Apr 16 08:44:54.286846 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:54.286809 2578 generic.go:358] "Generic (PLEG): container finished" podID="8fdf0577-bb86-4e62-836a-62de5559009a" containerID="0bc1d3ba61734b3144c38d8a3d4fb02830dc04a4ac6365bb7d07e1d95bc885ea" exitCode=0 Apr 16 08:44:54.287262 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:54.286890 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7" event={"ID":"8fdf0577-bb86-4e62-836a-62de5559009a","Type":"ContainerDied","Data":"0bc1d3ba61734b3144c38d8a3d4fb02830dc04a4ac6365bb7d07e1d95bc885ea"} Apr 16 08:44:55.291456 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:55.291426 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-597b96b99b-jb8qk" event={"ID":"4c73fa39-5fbc-4c7f-918c-ac5bd1c30ef7","Type":"ContainerStarted","Data":"36e3960b98e3932e73aa873d2b2f18f67d8ddc6193a5d579c5041d4188168adb"} Apr 16 08:44:55.291897 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:55.291522 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-597b96b99b-jb8qk" Apr 16 08:44:55.306961 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:55.306920 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-597b96b99b-jb8qk" podStartSLOduration=1.615389815 podStartE2EDuration="4.306902832s" podCreationTimestamp="2026-04-16 08:44:51 +0000 UTC" firstStartedPulling="2026-04-16 08:44:52.36765273 +0000 UTC m=+340.736636766" lastFinishedPulling="2026-04-16 08:44:55.059165748 +0000 UTC m=+343.428149783" observedRunningTime="2026-04-16 08:44:55.305542245 +0000 UTC m=+343.674526301" watchObservedRunningTime="2026-04-16 08:44:55.306902832 +0000 UTC m=+343.675886891" Apr 16 08:44:55.409058 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:55.409037 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7" Apr 16 08:44:55.532318 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:55.532280 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8fdf0577-bb86-4e62-836a-62de5559009a-util\") pod \"8fdf0577-bb86-4e62-836a-62de5559009a\" (UID: \"8fdf0577-bb86-4e62-836a-62de5559009a\") " Apr 16 08:44:55.532488 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:55.532345 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6vz9\" (UniqueName: \"kubernetes.io/projected/8fdf0577-bb86-4e62-836a-62de5559009a-kube-api-access-w6vz9\") pod \"8fdf0577-bb86-4e62-836a-62de5559009a\" (UID: \"8fdf0577-bb86-4e62-836a-62de5559009a\") " Apr 16 08:44:55.532488 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:55.532379 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8fdf0577-bb86-4e62-836a-62de5559009a-bundle\") pod \"8fdf0577-bb86-4e62-836a-62de5559009a\" (UID: \"8fdf0577-bb86-4e62-836a-62de5559009a\") " Apr 16 08:44:55.532782 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:55.532747 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fdf0577-bb86-4e62-836a-62de5559009a-bundle" (OuterVolumeSpecName: "bundle") pod "8fdf0577-bb86-4e62-836a-62de5559009a" (UID: "8fdf0577-bb86-4e62-836a-62de5559009a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 08:44:55.534678 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:55.534650 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fdf0577-bb86-4e62-836a-62de5559009a-kube-api-access-w6vz9" (OuterVolumeSpecName: "kube-api-access-w6vz9") pod "8fdf0577-bb86-4e62-836a-62de5559009a" (UID: "8fdf0577-bb86-4e62-836a-62de5559009a"). InnerVolumeSpecName "kube-api-access-w6vz9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 08:44:55.537382 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:55.537361 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fdf0577-bb86-4e62-836a-62de5559009a-util" (OuterVolumeSpecName: "util") pod "8fdf0577-bb86-4e62-836a-62de5559009a" (UID: "8fdf0577-bb86-4e62-836a-62de5559009a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 08:44:55.633853 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:55.633738 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8fdf0577-bb86-4e62-836a-62de5559009a-bundle\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:44:55.633853 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:55.633787 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8fdf0577-bb86-4e62-836a-62de5559009a-util\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:44:55.633853 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:55.633798 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6vz9\" (UniqueName: \"kubernetes.io/projected/8fdf0577-bb86-4e62-836a-62de5559009a-kube-api-access-w6vz9\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:44:56.302745 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:56.302708 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7" event={"ID":"8fdf0577-bb86-4e62-836a-62de5559009a","Type":"ContainerDied","Data":"492df52ed9cf5ce9e47afdbec48805198e1fd7ca8a21e17ec9d41c9f18fe0ddd"} Apr 16 08:44:56.303192 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:56.302750 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="492df52ed9cf5ce9e47afdbec48805198e1fd7ca8a21e17ec9d41c9f18fe0ddd" Apr 16 08:44:56.303192 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:44:56.302963 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbd2v7" Apr 16 08:45:00.539913 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:00.539881 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-xkx66"] Apr 16 08:45:00.540266 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:00.540196 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fdf0577-bb86-4e62-836a-62de5559009a" containerName="pull" Apr 16 08:45:00.540266 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:00.540212 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fdf0577-bb86-4e62-836a-62de5559009a" containerName="pull" Apr 16 08:45:00.540266 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:00.540222 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fdf0577-bb86-4e62-836a-62de5559009a" containerName="extract" Apr 16 08:45:00.540266 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:00.540228 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fdf0577-bb86-4e62-836a-62de5559009a" containerName="extract" Apr 16 08:45:00.540266 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:00.540237 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fdf0577-bb86-4e62-836a-62de5559009a" containerName="util" Apr 16 08:45:00.540266 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:00.540242 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fdf0577-bb86-4e62-836a-62de5559009a" containerName="util" Apr 16 08:45:00.540440 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:00.540301 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="8fdf0577-bb86-4e62-836a-62de5559009a" containerName="extract" Apr 16 08:45:00.543502 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:00.543486 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-xkx66" Apr 16 08:45:00.545431 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:00.545393 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Apr 16 08:45:00.546010 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:00.545991 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Apr 16 08:45:00.546113 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:00.546054 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-bc57h\"" Apr 16 08:45:00.549722 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:00.549703 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-xkx66"] Apr 16 08:45:00.680104 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:00.680068 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv2vp\" (UniqueName: \"kubernetes.io/projected/411dd942-5ef1-47ef-8412-c218f0bcae6b-kube-api-access-xv2vp\") pod \"openshift-lws-operator-bfc7f696d-xkx66\" (UID: \"411dd942-5ef1-47ef-8412-c218f0bcae6b\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-xkx66" Apr 16 08:45:00.680272 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:00.680130 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/411dd942-5ef1-47ef-8412-c218f0bcae6b-tmp\") pod \"openshift-lws-operator-bfc7f696d-xkx66\" (UID: \"411dd942-5ef1-47ef-8412-c218f0bcae6b\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-xkx66" Apr 16 08:45:00.780571 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:00.780535 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/411dd942-5ef1-47ef-8412-c218f0bcae6b-tmp\") pod \"openshift-lws-operator-bfc7f696d-xkx66\" (UID: \"411dd942-5ef1-47ef-8412-c218f0bcae6b\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-xkx66" Apr 16 08:45:00.780738 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:00.780594 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xv2vp\" (UniqueName: \"kubernetes.io/projected/411dd942-5ef1-47ef-8412-c218f0bcae6b-kube-api-access-xv2vp\") pod \"openshift-lws-operator-bfc7f696d-xkx66\" (UID: \"411dd942-5ef1-47ef-8412-c218f0bcae6b\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-xkx66" Apr 16 08:45:00.780990 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:00.780968 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/411dd942-5ef1-47ef-8412-c218f0bcae6b-tmp\") pod \"openshift-lws-operator-bfc7f696d-xkx66\" (UID: \"411dd942-5ef1-47ef-8412-c218f0bcae6b\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-xkx66" Apr 16 08:45:00.787893 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:00.787871 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv2vp\" (UniqueName: \"kubernetes.io/projected/411dd942-5ef1-47ef-8412-c218f0bcae6b-kube-api-access-xv2vp\") pod \"openshift-lws-operator-bfc7f696d-xkx66\" (UID: \"411dd942-5ef1-47ef-8412-c218f0bcae6b\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-xkx66" Apr 16 08:45:00.853864 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:00.853785 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-xkx66" Apr 16 08:45:00.978285 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:00.978250 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-xkx66"] Apr 16 08:45:00.981734 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:45:00.981710 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod411dd942_5ef1_47ef_8412_c218f0bcae6b.slice/crio-cda173e67590e7d5bb1e26b80da44deadb37ae5ab5bf295c3924a827ec4a382d WatchSource:0}: Error finding container cda173e67590e7d5bb1e26b80da44deadb37ae5ab5bf295c3924a827ec4a382d: Status 404 returned error can't find the container with id cda173e67590e7d5bb1e26b80da44deadb37ae5ab5bf295c3924a827ec4a382d Apr 16 08:45:01.304922 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:01.304892 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-597b96b99b-jb8qk" Apr 16 08:45:01.321697 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:01.321667 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-xkx66" event={"ID":"411dd942-5ef1-47ef-8412-c218f0bcae6b","Type":"ContainerStarted","Data":"cda173e67590e7d5bb1e26b80da44deadb37ae5ab5bf295c3924a827ec4a382d"} Apr 16 08:45:04.333571 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:04.333532 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-xkx66" event={"ID":"411dd942-5ef1-47ef-8412-c218f0bcae6b","Type":"ContainerStarted","Data":"8a2822fd1b07663cff699b96fd9cc8daace4cb41360c5fb04ddd02332cbd8802"} Apr 16 08:45:04.348783 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:04.348722 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-xkx66" podStartSLOduration=1.5967105780000002 podStartE2EDuration="4.34870714s" podCreationTimestamp="2026-04-16 08:45:00 +0000 UTC" firstStartedPulling="2026-04-16 08:45:00.983037398 +0000 UTC m=+349.352021433" lastFinishedPulling="2026-04-16 08:45:03.735033959 +0000 UTC m=+352.104017995" observedRunningTime="2026-04-16 08:45:04.346863634 +0000 UTC m=+352.715847691" watchObservedRunningTime="2026-04-16 08:45:04.34870714 +0000 UTC m=+352.717691197" Apr 16 08:45:06.497176 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:06.497142 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr"] Apr 16 08:45:06.504145 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:06.504126 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr" Apr 16 08:45:06.507337 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:06.506889 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Apr 16 08:45:06.507337 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:06.506909 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tlgdk\"" Apr 16 08:45:06.507337 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:06.507165 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Apr 16 08:45:06.508905 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:06.508878 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr"] Apr 16 08:45:06.628546 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:06.628512 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsdpx\" (UniqueName: \"kubernetes.io/projected/9bfdb99b-3d16-44e0-b472-f71f5cb0df68-kube-api-access-jsdpx\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr\" (UID: \"9bfdb99b-3d16-44e0-b472-f71f5cb0df68\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr" Apr 16 08:45:06.628714 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:06.628560 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bfdb99b-3d16-44e0-b472-f71f5cb0df68-util\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr\" (UID: \"9bfdb99b-3d16-44e0-b472-f71f5cb0df68\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr" Apr 16 08:45:06.628714 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:06.628583 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bfdb99b-3d16-44e0-b472-f71f5cb0df68-bundle\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr\" (UID: \"9bfdb99b-3d16-44e0-b472-f71f5cb0df68\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr" Apr 16 08:45:06.729431 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:06.729390 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bfdb99b-3d16-44e0-b472-f71f5cb0df68-bundle\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr\" (UID: \"9bfdb99b-3d16-44e0-b472-f71f5cb0df68\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr" Apr 16 08:45:06.729627 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:06.729481 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jsdpx\" (UniqueName: \"kubernetes.io/projected/9bfdb99b-3d16-44e0-b472-f71f5cb0df68-kube-api-access-jsdpx\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr\" (UID: \"9bfdb99b-3d16-44e0-b472-f71f5cb0df68\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr" Apr 16 08:45:06.729627 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:06.729512 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bfdb99b-3d16-44e0-b472-f71f5cb0df68-util\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr\" (UID: \"9bfdb99b-3d16-44e0-b472-f71f5cb0df68\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr" Apr 16 08:45:06.729842 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:06.729819 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bfdb99b-3d16-44e0-b472-f71f5cb0df68-bundle\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr\" (UID: \"9bfdb99b-3d16-44e0-b472-f71f5cb0df68\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr" Apr 16 08:45:06.729919 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:06.729843 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bfdb99b-3d16-44e0-b472-f71f5cb0df68-util\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr\" (UID: \"9bfdb99b-3d16-44e0-b472-f71f5cb0df68\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr" Apr 16 08:45:06.737350 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:06.737318 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsdpx\" (UniqueName: \"kubernetes.io/projected/9bfdb99b-3d16-44e0-b472-f71f5cb0df68-kube-api-access-jsdpx\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr\" (UID: \"9bfdb99b-3d16-44e0-b472-f71f5cb0df68\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr" Apr 16 08:45:06.813933 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:06.813852 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr" Apr 16 08:45:06.936875 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:06.936851 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr"] Apr 16 08:45:06.939186 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:45:06.939160 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bfdb99b_3d16_44e0_b472_f71f5cb0df68.slice/crio-4296bee2d764302af23bbf150062cf6efd6c58f5526e38d0b4859035ea06c9a3 WatchSource:0}: Error finding container 4296bee2d764302af23bbf150062cf6efd6c58f5526e38d0b4859035ea06c9a3: Status 404 returned error can't find the container with id 4296bee2d764302af23bbf150062cf6efd6c58f5526e38d0b4859035ea06c9a3 Apr 16 08:45:07.345406 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:07.345372 2578 generic.go:358] "Generic (PLEG): container finished" podID="9bfdb99b-3d16-44e0-b472-f71f5cb0df68" containerID="77d4833d27ebc0d0dcc6c1112718c89375c5e411cdb9c16c2efc6d1ce252fdd5" exitCode=0 Apr 16 08:45:07.345551 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:07.345458 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr" event={"ID":"9bfdb99b-3d16-44e0-b472-f71f5cb0df68","Type":"ContainerDied","Data":"77d4833d27ebc0d0dcc6c1112718c89375c5e411cdb9c16c2efc6d1ce252fdd5"} Apr 16 08:45:07.345551 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:07.345498 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr" event={"ID":"9bfdb99b-3d16-44e0-b472-f71f5cb0df68","Type":"ContainerStarted","Data":"4296bee2d764302af23bbf150062cf6efd6c58f5526e38d0b4859035ea06c9a3"} Apr 16 08:45:08.350528 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:08.350444 2578 generic.go:358] "Generic (PLEG): container finished" podID="9bfdb99b-3d16-44e0-b472-f71f5cb0df68" containerID="37de58b459b35c0ee90fd1917f38877b0bdf76783a363a42b908cce0f3a043e4" exitCode=0 Apr 16 08:45:08.350528 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:08.350482 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr" event={"ID":"9bfdb99b-3d16-44e0-b472-f71f5cb0df68","Type":"ContainerDied","Data":"37de58b459b35c0ee90fd1917f38877b0bdf76783a363a42b908cce0f3a043e4"} Apr 16 08:45:09.355822 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:09.355755 2578 generic.go:358] "Generic (PLEG): container finished" podID="9bfdb99b-3d16-44e0-b472-f71f5cb0df68" containerID="045a30630818dbe6804fcb033ed5272738f47e5bd940523067e7c2e2b4367508" exitCode=0 Apr 16 08:45:09.355822 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:09.355809 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr" event={"ID":"9bfdb99b-3d16-44e0-b472-f71f5cb0df68","Type":"ContainerDied","Data":"045a30630818dbe6804fcb033ed5272738f47e5bd940523067e7c2e2b4367508"} Apr 16 08:45:10.487574 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:10.487553 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr" Apr 16 08:45:10.662122 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:10.662030 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bfdb99b-3d16-44e0-b472-f71f5cb0df68-bundle\") pod \"9bfdb99b-3d16-44e0-b472-f71f5cb0df68\" (UID: \"9bfdb99b-3d16-44e0-b472-f71f5cb0df68\") " Apr 16 08:45:10.662122 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:10.662092 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jsdpx\" (UniqueName: \"kubernetes.io/projected/9bfdb99b-3d16-44e0-b472-f71f5cb0df68-kube-api-access-jsdpx\") pod \"9bfdb99b-3d16-44e0-b472-f71f5cb0df68\" (UID: \"9bfdb99b-3d16-44e0-b472-f71f5cb0df68\") " Apr 16 08:45:10.662339 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:10.662144 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bfdb99b-3d16-44e0-b472-f71f5cb0df68-util\") pod \"9bfdb99b-3d16-44e0-b472-f71f5cb0df68\" (UID: \"9bfdb99b-3d16-44e0-b472-f71f5cb0df68\") " Apr 16 08:45:10.662800 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:10.662713 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bfdb99b-3d16-44e0-b472-f71f5cb0df68-bundle" (OuterVolumeSpecName: "bundle") pod "9bfdb99b-3d16-44e0-b472-f71f5cb0df68" (UID: "9bfdb99b-3d16-44e0-b472-f71f5cb0df68"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 08:45:10.664400 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:10.664367 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bfdb99b-3d16-44e0-b472-f71f5cb0df68-kube-api-access-jsdpx" (OuterVolumeSpecName: "kube-api-access-jsdpx") pod "9bfdb99b-3d16-44e0-b472-f71f5cb0df68" (UID: "9bfdb99b-3d16-44e0-b472-f71f5cb0df68"). InnerVolumeSpecName "kube-api-access-jsdpx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 08:45:10.667490 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:10.667455 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bfdb99b-3d16-44e0-b472-f71f5cb0df68-util" (OuterVolumeSpecName: "util") pod "9bfdb99b-3d16-44e0-b472-f71f5cb0df68" (UID: "9bfdb99b-3d16-44e0-b472-f71f5cb0df68"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 08:45:10.763202 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:10.763163 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bfdb99b-3d16-44e0-b472-f71f5cb0df68-bundle\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:45:10.763202 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:10.763190 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jsdpx\" (UniqueName: \"kubernetes.io/projected/9bfdb99b-3d16-44e0-b472-f71f5cb0df68-kube-api-access-jsdpx\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:45:10.763202 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:10.763207 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bfdb99b-3d16-44e0-b472-f71f5cb0df68-util\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:45:11.363937 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:11.363901 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr" event={"ID":"9bfdb99b-3d16-44e0-b472-f71f5cb0df68","Type":"ContainerDied","Data":"4296bee2d764302af23bbf150062cf6efd6c58f5526e38d0b4859035ea06c9a3"} Apr 16 08:45:11.363937 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:11.363935 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4296bee2d764302af23bbf150062cf6efd6c58f5526e38d0b4859035ea06c9a3" Apr 16 08:45:11.364111 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:11.363945 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5vr7qr" Apr 16 08:45:17.679171 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:17.679130 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q"] Apr 16 08:45:17.679653 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:17.679602 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bfdb99b-3d16-44e0-b472-f71f5cb0df68" containerName="extract" Apr 16 08:45:17.679653 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:17.679619 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bfdb99b-3d16-44e0-b472-f71f5cb0df68" containerName="extract" Apr 16 08:45:17.679653 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:17.679632 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bfdb99b-3d16-44e0-b472-f71f5cb0df68" containerName="pull" Apr 16 08:45:17.679653 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:17.679639 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bfdb99b-3d16-44e0-b472-f71f5cb0df68" containerName="pull" Apr 16 08:45:17.679653 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:17.679651 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bfdb99b-3d16-44e0-b472-f71f5cb0df68" containerName="util" Apr 16 08:45:17.679916 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:17.679659 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bfdb99b-3d16-44e0-b472-f71f5cb0df68" containerName="util" Apr 16 08:45:17.679916 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:17.679760 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="9bfdb99b-3d16-44e0-b472-f71f5cb0df68" containerName="extract" Apr 16 08:45:17.687944 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:17.687909 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q" Apr 16 08:45:17.690864 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:17.690829 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tlgdk\"" Apr 16 08:45:17.691319 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:17.691128 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Apr 16 08:45:17.691319 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:17.691265 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Apr 16 08:45:17.692690 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:17.692666 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q"] Apr 16 08:45:17.715648 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:17.715614 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bd49ba32-6003-4416-95f1-a6e3ffab42a4-bundle\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q\" (UID: \"bd49ba32-6003-4416-95f1-a6e3ffab42a4\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q" Apr 16 08:45:17.715648 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:17.715645 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bd49ba32-6003-4416-95f1-a6e3ffab42a4-util\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q\" (UID: \"bd49ba32-6003-4416-95f1-a6e3ffab42a4\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q" Apr 16 08:45:17.715919 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:17.715693 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgjgb\" (UniqueName: \"kubernetes.io/projected/bd49ba32-6003-4416-95f1-a6e3ffab42a4-kube-api-access-xgjgb\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q\" (UID: \"bd49ba32-6003-4416-95f1-a6e3ffab42a4\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q" Apr 16 08:45:17.817062 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:17.817028 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bd49ba32-6003-4416-95f1-a6e3ffab42a4-bundle\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q\" (UID: \"bd49ba32-6003-4416-95f1-a6e3ffab42a4\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q" Apr 16 08:45:17.817062 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:17.817060 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bd49ba32-6003-4416-95f1-a6e3ffab42a4-util\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q\" (UID: \"bd49ba32-6003-4416-95f1-a6e3ffab42a4\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q" Apr 16 08:45:17.817307 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:17.817123 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xgjgb\" (UniqueName: \"kubernetes.io/projected/bd49ba32-6003-4416-95f1-a6e3ffab42a4-kube-api-access-xgjgb\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q\" (UID: \"bd49ba32-6003-4416-95f1-a6e3ffab42a4\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q" Apr 16 08:45:17.817481 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:17.817459 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bd49ba32-6003-4416-95f1-a6e3ffab42a4-bundle\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q\" (UID: \"bd49ba32-6003-4416-95f1-a6e3ffab42a4\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q" Apr 16 08:45:17.817536 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:17.817476 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bd49ba32-6003-4416-95f1-a6e3ffab42a4-util\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q\" (UID: \"bd49ba32-6003-4416-95f1-a6e3ffab42a4\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q" Apr 16 08:45:17.824755 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:17.824734 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgjgb\" (UniqueName: \"kubernetes.io/projected/bd49ba32-6003-4416-95f1-a6e3ffab42a4-kube-api-access-xgjgb\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q\" (UID: \"bd49ba32-6003-4416-95f1-a6e3ffab42a4\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q" Apr 16 08:45:17.998754 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:17.998721 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q" Apr 16 08:45:18.135006 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:18.134973 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q"] Apr 16 08:45:18.138470 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:45:18.138447 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd49ba32_6003_4416_95f1_a6e3ffab42a4.slice/crio-ad35263fde5b12e643b325c1d7404d2e0aa648c005598c59345b845b67795711 WatchSource:0}: Error finding container ad35263fde5b12e643b325c1d7404d2e0aa648c005598c59345b845b67795711: Status 404 returned error can't find the container with id ad35263fde5b12e643b325c1d7404d2e0aa648c005598c59345b845b67795711 Apr 16 08:45:18.388605 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:18.388512 2578 generic.go:358] "Generic (PLEG): container finished" podID="bd49ba32-6003-4416-95f1-a6e3ffab42a4" containerID="8523d0404757319c09ba7635742a6aa83558bcec2fa70d7714e99e830f7e05cb" exitCode=0 Apr 16 08:45:18.388759 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:18.388596 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q" event={"ID":"bd49ba32-6003-4416-95f1-a6e3ffab42a4","Type":"ContainerDied","Data":"8523d0404757319c09ba7635742a6aa83558bcec2fa70d7714e99e830f7e05cb"} Apr 16 08:45:18.388759 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:18.388632 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q" event={"ID":"bd49ba32-6003-4416-95f1-a6e3ffab42a4","Type":"ContainerStarted","Data":"ad35263fde5b12e643b325c1d7404d2e0aa648c005598c59345b845b67795711"} Apr 16 08:45:19.131347 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:19.131306 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-569944d57d-rk86c"] Apr 16 08:45:19.136313 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:19.136289 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-rk86c" Apr 16 08:45:19.138539 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:19.138512 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Apr 16 08:45:19.138706 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:19.138624 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-k8z2v\"" Apr 16 08:45:19.138706 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:19.138622 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Apr 16 08:45:19.138860 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:19.138746 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Apr 16 08:45:19.138911 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:19.138859 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Apr 16 08:45:19.146505 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:19.146481 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-569944d57d-rk86c"] Apr 16 08:45:19.232808 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:19.232756 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q4tr\" (UniqueName: \"kubernetes.io/projected/5b4974c7-139a-43f3-b681-f42333201e5f-kube-api-access-4q4tr\") pod \"opendatahub-operator-controller-manager-569944d57d-rk86c\" (UID: \"5b4974c7-139a-43f3-b681-f42333201e5f\") " pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-rk86c" Apr 16 08:45:19.232981 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:19.232825 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5b4974c7-139a-43f3-b681-f42333201e5f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-569944d57d-rk86c\" (UID: \"5b4974c7-139a-43f3-b681-f42333201e5f\") " pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-rk86c" Apr 16 08:45:19.232981 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:19.232855 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5b4974c7-139a-43f3-b681-f42333201e5f-webhook-cert\") pod \"opendatahub-operator-controller-manager-569944d57d-rk86c\" (UID: \"5b4974c7-139a-43f3-b681-f42333201e5f\") " pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-rk86c" Apr 16 08:45:19.333941 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:19.333856 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5b4974c7-139a-43f3-b681-f42333201e5f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-569944d57d-rk86c\" (UID: \"5b4974c7-139a-43f3-b681-f42333201e5f\") " pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-rk86c" Apr 16 08:45:19.333941 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:19.333896 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5b4974c7-139a-43f3-b681-f42333201e5f-webhook-cert\") pod \"opendatahub-operator-controller-manager-569944d57d-rk86c\" (UID: \"5b4974c7-139a-43f3-b681-f42333201e5f\") " pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-rk86c" Apr 16 08:45:19.334139 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:19.333996 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4q4tr\" (UniqueName: \"kubernetes.io/projected/5b4974c7-139a-43f3-b681-f42333201e5f-kube-api-access-4q4tr\") pod \"opendatahub-operator-controller-manager-569944d57d-rk86c\" (UID: \"5b4974c7-139a-43f3-b681-f42333201e5f\") " pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-rk86c" Apr 16 08:45:19.336578 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:19.336558 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5b4974c7-139a-43f3-b681-f42333201e5f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-569944d57d-rk86c\" (UID: \"5b4974c7-139a-43f3-b681-f42333201e5f\") " pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-rk86c" Apr 16 08:45:19.336678 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:19.336588 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5b4974c7-139a-43f3-b681-f42333201e5f-webhook-cert\") pod \"opendatahub-operator-controller-manager-569944d57d-rk86c\" (UID: \"5b4974c7-139a-43f3-b681-f42333201e5f\") " pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-rk86c" Apr 16 08:45:19.341251 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:19.341224 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q4tr\" (UniqueName: \"kubernetes.io/projected/5b4974c7-139a-43f3-b681-f42333201e5f-kube-api-access-4q4tr\") pod \"opendatahub-operator-controller-manager-569944d57d-rk86c\" (UID: \"5b4974c7-139a-43f3-b681-f42333201e5f\") " pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-rk86c" Apr 16 08:45:19.394247 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:19.394217 2578 generic.go:358] "Generic (PLEG): container finished" podID="bd49ba32-6003-4416-95f1-a6e3ffab42a4" containerID="222aa72146e6611a329e6effa095f8a0b6bd240e18c0e1e4b1b3bea7dae8f80f" exitCode=0 Apr 16 08:45:19.394445 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:19.394277 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q" event={"ID":"bd49ba32-6003-4416-95f1-a6e3ffab42a4","Type":"ContainerDied","Data":"222aa72146e6611a329e6effa095f8a0b6bd240e18c0e1e4b1b3bea7dae8f80f"} Apr 16 08:45:19.490418 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:19.490386 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-rk86c" Apr 16 08:45:19.619623 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:19.619552 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-569944d57d-rk86c"] Apr 16 08:45:19.623261 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:45:19.623232 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b4974c7_139a_43f3_b681_f42333201e5f.slice/crio-59320ebec6341090e7517ff1301c952e7a8c471b9e062aa5368bd8a2339c82e7 WatchSource:0}: Error finding container 59320ebec6341090e7517ff1301c952e7a8c471b9e062aa5368bd8a2339c82e7: Status 404 returned error can't find the container with id 59320ebec6341090e7517ff1301c952e7a8c471b9e062aa5368bd8a2339c82e7 Apr 16 08:45:20.399959 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:20.399921 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-rk86c" event={"ID":"5b4974c7-139a-43f3-b681-f42333201e5f","Type":"ContainerStarted","Data":"59320ebec6341090e7517ff1301c952e7a8c471b9e062aa5368bd8a2339c82e7"} Apr 16 08:45:20.402201 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:20.402146 2578 generic.go:358] "Generic (PLEG): container finished" podID="bd49ba32-6003-4416-95f1-a6e3ffab42a4" containerID="22733ba4e958608c74313e778ffea44f5082abe509348c69dc21c58ced078d5c" exitCode=0 Apr 16 08:45:20.402346 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:20.402206 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q" event={"ID":"bd49ba32-6003-4416-95f1-a6e3ffab42a4","Type":"ContainerDied","Data":"22733ba4e958608c74313e778ffea44f5082abe509348c69dc21c58ced078d5c"} Apr 16 08:45:22.005720 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:22.005698 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q" Apr 16 08:45:22.058134 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:22.058110 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bd49ba32-6003-4416-95f1-a6e3ffab42a4-util\") pod \"bd49ba32-6003-4416-95f1-a6e3ffab42a4\" (UID: \"bd49ba32-6003-4416-95f1-a6e3ffab42a4\") " Apr 16 08:45:22.058261 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:22.058142 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgjgb\" (UniqueName: \"kubernetes.io/projected/bd49ba32-6003-4416-95f1-a6e3ffab42a4-kube-api-access-xgjgb\") pod \"bd49ba32-6003-4416-95f1-a6e3ffab42a4\" (UID: \"bd49ba32-6003-4416-95f1-a6e3ffab42a4\") " Apr 16 08:45:22.058261 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:22.058172 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bd49ba32-6003-4416-95f1-a6e3ffab42a4-bundle\") pod \"bd49ba32-6003-4416-95f1-a6e3ffab42a4\" (UID: \"bd49ba32-6003-4416-95f1-a6e3ffab42a4\") " Apr 16 08:45:22.059101 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:22.059052 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd49ba32-6003-4416-95f1-a6e3ffab42a4-bundle" (OuterVolumeSpecName: "bundle") pod "bd49ba32-6003-4416-95f1-a6e3ffab42a4" (UID: "bd49ba32-6003-4416-95f1-a6e3ffab42a4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 08:45:22.060385 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:22.060361 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd49ba32-6003-4416-95f1-a6e3ffab42a4-kube-api-access-xgjgb" (OuterVolumeSpecName: "kube-api-access-xgjgb") pod "bd49ba32-6003-4416-95f1-a6e3ffab42a4" (UID: "bd49ba32-6003-4416-95f1-a6e3ffab42a4"). InnerVolumeSpecName "kube-api-access-xgjgb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 08:45:22.062988 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:22.062958 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd49ba32-6003-4416-95f1-a6e3ffab42a4-util" (OuterVolumeSpecName: "util") pod "bd49ba32-6003-4416-95f1-a6e3ffab42a4" (UID: "bd49ba32-6003-4416-95f1-a6e3ffab42a4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 08:45:22.159800 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:22.159740 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bd49ba32-6003-4416-95f1-a6e3ffab42a4-bundle\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:45:22.159800 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:22.159798 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bd49ba32-6003-4416-95f1-a6e3ffab42a4-util\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:45:22.159995 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:22.159811 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgjgb\" (UniqueName: \"kubernetes.io/projected/bd49ba32-6003-4416-95f1-a6e3ffab42a4-kube-api-access-xgjgb\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:45:22.410922 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:22.410834 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-rk86c" event={"ID":"5b4974c7-139a-43f3-b681-f42333201e5f","Type":"ContainerStarted","Data":"eb60be0c8b58f9579a2394ff0028bf35889a9d44781d9e708a8ba791002149c6"} Apr 16 08:45:22.410922 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:22.410881 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-rk86c" Apr 16 08:45:22.412530 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:22.412509 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q" event={"ID":"bd49ba32-6003-4416-95f1-a6e3ffab42a4","Type":"ContainerDied","Data":"ad35263fde5b12e643b325c1d7404d2e0aa648c005598c59345b845b67795711"} Apr 16 08:45:22.412530 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:22.412532 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ad35263fde5b12e643b325c1d7404d2e0aa648c005598c59345b845b67795711" Apr 16 08:45:22.412722 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:22.412565 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c99tz2q" Apr 16 08:45:22.428062 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:22.428008 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-rk86c" podStartSLOduration=1.016027251 podStartE2EDuration="3.427547752s" podCreationTimestamp="2026-04-16 08:45:19 +0000 UTC" firstStartedPulling="2026-04-16 08:45:19.624901598 +0000 UTC m=+367.993885633" lastFinishedPulling="2026-04-16 08:45:22.036422096 +0000 UTC m=+370.405406134" observedRunningTime="2026-04-16 08:45:22.426828061 +0000 UTC m=+370.795812118" watchObservedRunningTime="2026-04-16 08:45:22.427547752 +0000 UTC m=+370.796531810" Apr 16 08:45:33.418295 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:33.418265 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-rk86c" Apr 16 08:45:39.544950 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.544915 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cbc7f8cc-dd79v"] Apr 16 08:45:39.545310 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.545233 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd49ba32-6003-4416-95f1-a6e3ffab42a4" containerName="pull" Apr 16 08:45:39.545310 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.545243 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd49ba32-6003-4416-95f1-a6e3ffab42a4" containerName="pull" Apr 16 08:45:39.545310 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.545259 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd49ba32-6003-4416-95f1-a6e3ffab42a4" containerName="extract" Apr 16 08:45:39.545310 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.545264 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd49ba32-6003-4416-95f1-a6e3ffab42a4" containerName="extract" Apr 16 08:45:39.545310 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.545275 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd49ba32-6003-4416-95f1-a6e3ffab42a4" containerName="util" Apr 16 08:45:39.545310 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.545279 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd49ba32-6003-4416-95f1-a6e3ffab42a4" containerName="util" Apr 16 08:45:39.545486 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.545327 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="bd49ba32-6003-4416-95f1-a6e3ffab42a4" containerName="extract" Apr 16 08:45:39.549304 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.549288 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7cbc7f8cc-dd79v" Apr 16 08:45:39.551907 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.551880 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Apr 16 08:45:39.552049 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.551887 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Apr 16 08:45:39.552049 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.551895 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Apr 16 08:45:39.552049 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.551942 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-pkkzs\"" Apr 16 08:45:39.556300 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.556274 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cbc7f8cc-dd79v"] Apr 16 08:45:39.605100 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.605066 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7b083bc-b402-45b5-9d6d-a3fbb0cd96dd-cert\") pod \"lws-controller-manager-7cbc7f8cc-dd79v\" (UID: \"b7b083bc-b402-45b5-9d6d-a3fbb0cd96dd\") " pod="openshift-lws-operator/lws-controller-manager-7cbc7f8cc-dd79v" Apr 16 08:45:39.605100 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.605100 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b7b083bc-b402-45b5-9d6d-a3fbb0cd96dd-metrics-cert\") pod \"lws-controller-manager-7cbc7f8cc-dd79v\" (UID: \"b7b083bc-b402-45b5-9d6d-a3fbb0cd96dd\") " pod="openshift-lws-operator/lws-controller-manager-7cbc7f8cc-dd79v" Apr 16 08:45:39.605309 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.605124 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b7b083bc-b402-45b5-9d6d-a3fbb0cd96dd-manager-config\") pod \"lws-controller-manager-7cbc7f8cc-dd79v\" (UID: \"b7b083bc-b402-45b5-9d6d-a3fbb0cd96dd\") " pod="openshift-lws-operator/lws-controller-manager-7cbc7f8cc-dd79v" Apr 16 08:45:39.605309 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.605204 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f69c\" (UniqueName: \"kubernetes.io/projected/b7b083bc-b402-45b5-9d6d-a3fbb0cd96dd-kube-api-access-2f69c\") pod \"lws-controller-manager-7cbc7f8cc-dd79v\" (UID: \"b7b083bc-b402-45b5-9d6d-a3fbb0cd96dd\") " pod="openshift-lws-operator/lws-controller-manager-7cbc7f8cc-dd79v" Apr 16 08:45:39.706325 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.706293 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7b083bc-b402-45b5-9d6d-a3fbb0cd96dd-cert\") pod \"lws-controller-manager-7cbc7f8cc-dd79v\" (UID: \"b7b083bc-b402-45b5-9d6d-a3fbb0cd96dd\") " pod="openshift-lws-operator/lws-controller-manager-7cbc7f8cc-dd79v" Apr 16 08:45:39.706325 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.706326 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b7b083bc-b402-45b5-9d6d-a3fbb0cd96dd-metrics-cert\") pod \"lws-controller-manager-7cbc7f8cc-dd79v\" (UID: \"b7b083bc-b402-45b5-9d6d-a3fbb0cd96dd\") " pod="openshift-lws-operator/lws-controller-manager-7cbc7f8cc-dd79v" Apr 16 08:45:39.706570 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.706350 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b7b083bc-b402-45b5-9d6d-a3fbb0cd96dd-manager-config\") pod \"lws-controller-manager-7cbc7f8cc-dd79v\" (UID: \"b7b083bc-b402-45b5-9d6d-a3fbb0cd96dd\") " pod="openshift-lws-operator/lws-controller-manager-7cbc7f8cc-dd79v" Apr 16 08:45:39.706570 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.706376 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2f69c\" (UniqueName: \"kubernetes.io/projected/b7b083bc-b402-45b5-9d6d-a3fbb0cd96dd-kube-api-access-2f69c\") pod \"lws-controller-manager-7cbc7f8cc-dd79v\" (UID: \"b7b083bc-b402-45b5-9d6d-a3fbb0cd96dd\") " pod="openshift-lws-operator/lws-controller-manager-7cbc7f8cc-dd79v" Apr 16 08:45:39.707097 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.707074 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b7b083bc-b402-45b5-9d6d-a3fbb0cd96dd-manager-config\") pod \"lws-controller-manager-7cbc7f8cc-dd79v\" (UID: \"b7b083bc-b402-45b5-9d6d-a3fbb0cd96dd\") " pod="openshift-lws-operator/lws-controller-manager-7cbc7f8cc-dd79v" Apr 16 08:45:39.708845 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.708828 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b7b083bc-b402-45b5-9d6d-a3fbb0cd96dd-metrics-cert\") pod \"lws-controller-manager-7cbc7f8cc-dd79v\" (UID: \"b7b083bc-b402-45b5-9d6d-a3fbb0cd96dd\") " pod="openshift-lws-operator/lws-controller-manager-7cbc7f8cc-dd79v" Apr 16 08:45:39.708945 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.708871 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7b083bc-b402-45b5-9d6d-a3fbb0cd96dd-cert\") pod \"lws-controller-manager-7cbc7f8cc-dd79v\" (UID: \"b7b083bc-b402-45b5-9d6d-a3fbb0cd96dd\") " pod="openshift-lws-operator/lws-controller-manager-7cbc7f8cc-dd79v" Apr 16 08:45:39.713523 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.713505 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f69c\" (UniqueName: \"kubernetes.io/projected/b7b083bc-b402-45b5-9d6d-a3fbb0cd96dd-kube-api-access-2f69c\") pod \"lws-controller-manager-7cbc7f8cc-dd79v\" (UID: \"b7b083bc-b402-45b5-9d6d-a3fbb0cd96dd\") " pod="openshift-lws-operator/lws-controller-manager-7cbc7f8cc-dd79v" Apr 16 08:45:39.860014 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.859919 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7cbc7f8cc-dd79v" Apr 16 08:45:39.984568 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:39.984533 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cbc7f8cc-dd79v"] Apr 16 08:45:39.991197 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:45:39.991173 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7b083bc_b402_45b5_9d6d_a3fbb0cd96dd.slice/crio-1561bb10a19c9f6d967091018631fad6bbd1939dbe1e29f2a83b1b811bdeb92d WatchSource:0}: Error finding container 1561bb10a19c9f6d967091018631fad6bbd1939dbe1e29f2a83b1b811bdeb92d: Status 404 returned error can't find the container with id 1561bb10a19c9f6d967091018631fad6bbd1939dbe1e29f2a83b1b811bdeb92d Apr 16 08:45:40.477799 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:40.477747 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7cbc7f8cc-dd79v" event={"ID":"b7b083bc-b402-45b5-9d6d-a3fbb0cd96dd","Type":"ContainerStarted","Data":"1561bb10a19c9f6d967091018631fad6bbd1939dbe1e29f2a83b1b811bdeb92d"} Apr 16 08:45:46.502761 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:46.502724 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7cbc7f8cc-dd79v" event={"ID":"b7b083bc-b402-45b5-9d6d-a3fbb0cd96dd","Type":"ContainerStarted","Data":"2edae0930738aa6fbc1074da6857017ed4dedc6b1a82d1768a02a2496fcbc97d"} Apr 16 08:45:46.503195 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:46.502819 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7cbc7f8cc-dd79v" Apr 16 08:45:46.518365 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:46.518322 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7cbc7f8cc-dd79v" podStartSLOduration=1.972079758 podStartE2EDuration="7.518307043s" podCreationTimestamp="2026-04-16 08:45:39 +0000 UTC" firstStartedPulling="2026-04-16 08:45:39.9929362 +0000 UTC m=+388.361920235" lastFinishedPulling="2026-04-16 08:45:45.53916348 +0000 UTC m=+393.908147520" observedRunningTime="2026-04-16 08:45:46.51687104 +0000 UTC m=+394.885855124" watchObservedRunningTime="2026-04-16 08:45:46.518307043 +0000 UTC m=+394.887291096" Apr 16 08:45:47.094858 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.094819 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq"] Apr 16 08:45:47.097303 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.097282 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq" Apr 16 08:45:47.099486 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.099462 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Apr 16 08:45:47.099486 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.099462 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Apr 16 08:45:47.100031 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.100016 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tlgdk\"" Apr 16 08:45:47.107347 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.107326 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq"] Apr 16 08:45:47.167693 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.167658 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4trz4\" (UniqueName: \"kubernetes.io/projected/e604f635-e358-49c5-8391-06357a2c2320-kube-api-access-4trz4\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq\" (UID: \"e604f635-e358-49c5-8391-06357a2c2320\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq" Apr 16 08:45:47.167844 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.167706 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e604f635-e358-49c5-8391-06357a2c2320-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq\" (UID: \"e604f635-e358-49c5-8391-06357a2c2320\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq" Apr 16 08:45:47.167882 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.167848 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e604f635-e358-49c5-8391-06357a2c2320-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq\" (UID: \"e604f635-e358-49c5-8391-06357a2c2320\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq" Apr 16 08:45:47.223635 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.223571 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5d5746998b-qfmv2"] Apr 16 08:45:47.238707 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.234204 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5d5746998b-qfmv2" Apr 16 08:45:47.238707 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.237906 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5d5746998b-qfmv2"] Apr 16 08:45:47.238707 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.238214 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Apr 16 08:45:47.238707 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.238229 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-lvdrx\"" Apr 16 08:45:47.238707 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.238334 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Apr 16 08:45:47.269172 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.269145 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgdk2\" (UniqueName: \"kubernetes.io/projected/efa00193-20de-451e-a822-40ea9746a0eb-kube-api-access-kgdk2\") pod \"kube-auth-proxy-5d5746998b-qfmv2\" (UID: \"efa00193-20de-451e-a822-40ea9746a0eb\") " pod="openshift-ingress/kube-auth-proxy-5d5746998b-qfmv2" Apr 16 08:45:47.269338 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.269184 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e604f635-e358-49c5-8391-06357a2c2320-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq\" (UID: \"e604f635-e358-49c5-8391-06357a2c2320\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq" Apr 16 08:45:47.269391 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.269334 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/efa00193-20de-451e-a822-40ea9746a0eb-tmp\") pod \"kube-auth-proxy-5d5746998b-qfmv2\" (UID: \"efa00193-20de-451e-a822-40ea9746a0eb\") " pod="openshift-ingress/kube-auth-proxy-5d5746998b-qfmv2" Apr 16 08:45:47.269463 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.269442 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efa00193-20de-451e-a822-40ea9746a0eb-tls-certs\") pod \"kube-auth-proxy-5d5746998b-qfmv2\" (UID: \"efa00193-20de-451e-a822-40ea9746a0eb\") " pod="openshift-ingress/kube-auth-proxy-5d5746998b-qfmv2" Apr 16 08:45:47.269524 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.269499 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e604f635-e358-49c5-8391-06357a2c2320-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq\" (UID: \"e604f635-e358-49c5-8391-06357a2c2320\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq" Apr 16 08:45:47.269572 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.269532 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4trz4\" (UniqueName: \"kubernetes.io/projected/e604f635-e358-49c5-8391-06357a2c2320-kube-api-access-4trz4\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq\" (UID: \"e604f635-e358-49c5-8391-06357a2c2320\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq" Apr 16 08:45:47.269618 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.269568 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e604f635-e358-49c5-8391-06357a2c2320-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq\" (UID: \"e604f635-e358-49c5-8391-06357a2c2320\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq" Apr 16 08:45:47.269870 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.269849 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e604f635-e358-49c5-8391-06357a2c2320-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq\" (UID: \"e604f635-e358-49c5-8391-06357a2c2320\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq" Apr 16 08:45:47.283068 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.283042 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4trz4\" (UniqueName: \"kubernetes.io/projected/e604f635-e358-49c5-8391-06357a2c2320-kube-api-access-4trz4\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq\" (UID: \"e604f635-e358-49c5-8391-06357a2c2320\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq" Apr 16 08:45:47.370844 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.370735 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kgdk2\" (UniqueName: \"kubernetes.io/projected/efa00193-20de-451e-a822-40ea9746a0eb-kube-api-access-kgdk2\") pod \"kube-auth-proxy-5d5746998b-qfmv2\" (UID: \"efa00193-20de-451e-a822-40ea9746a0eb\") " pod="openshift-ingress/kube-auth-proxy-5d5746998b-qfmv2" Apr 16 08:45:47.370844 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.370832 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/efa00193-20de-451e-a822-40ea9746a0eb-tmp\") pod \"kube-auth-proxy-5d5746998b-qfmv2\" (UID: \"efa00193-20de-451e-a822-40ea9746a0eb\") " pod="openshift-ingress/kube-auth-proxy-5d5746998b-qfmv2" Apr 16 08:45:47.371054 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.370878 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efa00193-20de-451e-a822-40ea9746a0eb-tls-certs\") pod \"kube-auth-proxy-5d5746998b-qfmv2\" (UID: \"efa00193-20de-451e-a822-40ea9746a0eb\") " pod="openshift-ingress/kube-auth-proxy-5d5746998b-qfmv2" Apr 16 08:45:47.371054 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:45:47.370996 2578 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Apr 16 08:45:47.371148 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:45:47.371076 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efa00193-20de-451e-a822-40ea9746a0eb-tls-certs podName:efa00193-20de-451e-a822-40ea9746a0eb nodeName:}" failed. No retries permitted until 2026-04-16 08:45:47.871055333 +0000 UTC m=+396.240039369 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/efa00193-20de-451e-a822-40ea9746a0eb-tls-certs") pod "kube-auth-proxy-5d5746998b-qfmv2" (UID: "efa00193-20de-451e-a822-40ea9746a0eb") : secret "kube-auth-proxy-tls" not found Apr 16 08:45:47.373114 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.373085 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/efa00193-20de-451e-a822-40ea9746a0eb-tmp\") pod \"kube-auth-proxy-5d5746998b-qfmv2\" (UID: \"efa00193-20de-451e-a822-40ea9746a0eb\") " pod="openshift-ingress/kube-auth-proxy-5d5746998b-qfmv2" Apr 16 08:45:47.382569 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.382547 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgdk2\" (UniqueName: \"kubernetes.io/projected/efa00193-20de-451e-a822-40ea9746a0eb-kube-api-access-kgdk2\") pod \"kube-auth-proxy-5d5746998b-qfmv2\" (UID: \"efa00193-20de-451e-a822-40ea9746a0eb\") " pod="openshift-ingress/kube-auth-proxy-5d5746998b-qfmv2" Apr 16 08:45:47.406355 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.406327 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq" Apr 16 08:45:47.531691 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.531660 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq"] Apr 16 08:45:47.533736 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:45:47.533712 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode604f635_e358_49c5_8391_06357a2c2320.slice/crio-1468cfc1426a18298e64da2631b4ba2a29d28370d91c6419b7d34e2521707812 WatchSource:0}: Error finding container 1468cfc1426a18298e64da2631b4ba2a29d28370d91c6419b7d34e2521707812: Status 404 returned error can't find the container with id 1468cfc1426a18298e64da2631b4ba2a29d28370d91c6419b7d34e2521707812 Apr 16 08:45:47.874390 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.874303 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efa00193-20de-451e-a822-40ea9746a0eb-tls-certs\") pod \"kube-auth-proxy-5d5746998b-qfmv2\" (UID: \"efa00193-20de-451e-a822-40ea9746a0eb\") " pod="openshift-ingress/kube-auth-proxy-5d5746998b-qfmv2" Apr 16 08:45:47.876928 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:47.876901 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efa00193-20de-451e-a822-40ea9746a0eb-tls-certs\") pod \"kube-auth-proxy-5d5746998b-qfmv2\" (UID: \"efa00193-20de-451e-a822-40ea9746a0eb\") " pod="openshift-ingress/kube-auth-proxy-5d5746998b-qfmv2" Apr 16 08:45:48.146762 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:48.146683 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5d5746998b-qfmv2" Apr 16 08:45:48.267976 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:48.267949 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5d5746998b-qfmv2"] Apr 16 08:45:48.269677 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:45:48.269649 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefa00193_20de_451e_a822_40ea9746a0eb.slice/crio-509c9ccd2e535e474e207dbd23c7160b7b96fb0da189540716494e977b0b7d67 WatchSource:0}: Error finding container 509c9ccd2e535e474e207dbd23c7160b7b96fb0da189540716494e977b0b7d67: Status 404 returned error can't find the container with id 509c9ccd2e535e474e207dbd23c7160b7b96fb0da189540716494e977b0b7d67 Apr 16 08:45:48.511243 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:48.511199 2578 generic.go:358] "Generic (PLEG): container finished" podID="e604f635-e358-49c5-8391-06357a2c2320" containerID="dd1230cbfd2a5ed8cf1c5e93f6ba4cbab1c35fb8f9b4b1d0da238d87c407afc0" exitCode=0 Apr 16 08:45:48.511454 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:48.511284 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq" event={"ID":"e604f635-e358-49c5-8391-06357a2c2320","Type":"ContainerDied","Data":"dd1230cbfd2a5ed8cf1c5e93f6ba4cbab1c35fb8f9b4b1d0da238d87c407afc0"} Apr 16 08:45:48.511454 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:48.511321 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq" event={"ID":"e604f635-e358-49c5-8391-06357a2c2320","Type":"ContainerStarted","Data":"1468cfc1426a18298e64da2631b4ba2a29d28370d91c6419b7d34e2521707812"} Apr 16 08:45:48.512363 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:48.512343 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5d5746998b-qfmv2" event={"ID":"efa00193-20de-451e-a822-40ea9746a0eb","Type":"ContainerStarted","Data":"509c9ccd2e535e474e207dbd23c7160b7b96fb0da189540716494e977b0b7d67"} Apr 16 08:45:49.517299 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:49.517270 2578 generic.go:358] "Generic (PLEG): container finished" podID="e604f635-e358-49c5-8391-06357a2c2320" containerID="59de6d49064e52e4961f52315b116772eeafad33f882cc33a778372dd960345e" exitCode=0 Apr 16 08:45:49.517738 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:49.517332 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq" event={"ID":"e604f635-e358-49c5-8391-06357a2c2320","Type":"ContainerDied","Data":"59de6d49064e52e4961f52315b116772eeafad33f882cc33a778372dd960345e"} Apr 16 08:45:50.524997 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:50.524951 2578 generic.go:358] "Generic (PLEG): container finished" podID="e604f635-e358-49c5-8391-06357a2c2320" containerID="1467a413fcf9d4552865688a45f237e2659d45fe11b0884ce64eb601c07fab66" exitCode=0 Apr 16 08:45:50.525434 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:50.525022 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq" event={"ID":"e604f635-e358-49c5-8391-06357a2c2320","Type":"ContainerDied","Data":"1467a413fcf9d4552865688a45f237e2659d45fe11b0884ce64eb601c07fab66"} Apr 16 08:45:51.731236 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:51.731214 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq" Apr 16 08:45:51.807708 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:51.807684 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e604f635-e358-49c5-8391-06357a2c2320-util\") pod \"e604f635-e358-49c5-8391-06357a2c2320\" (UID: \"e604f635-e358-49c5-8391-06357a2c2320\") " Apr 16 08:45:51.807830 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:51.807787 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4trz4\" (UniqueName: \"kubernetes.io/projected/e604f635-e358-49c5-8391-06357a2c2320-kube-api-access-4trz4\") pod \"e604f635-e358-49c5-8391-06357a2c2320\" (UID: \"e604f635-e358-49c5-8391-06357a2c2320\") " Apr 16 08:45:51.807886 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:51.807870 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e604f635-e358-49c5-8391-06357a2c2320-bundle\") pod \"e604f635-e358-49c5-8391-06357a2c2320\" (UID: \"e604f635-e358-49c5-8391-06357a2c2320\") " Apr 16 08:45:51.808682 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:51.808651 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e604f635-e358-49c5-8391-06357a2c2320-bundle" (OuterVolumeSpecName: "bundle") pod "e604f635-e358-49c5-8391-06357a2c2320" (UID: "e604f635-e358-49c5-8391-06357a2c2320"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 08:45:51.810190 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:51.810161 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e604f635-e358-49c5-8391-06357a2c2320-kube-api-access-4trz4" (OuterVolumeSpecName: "kube-api-access-4trz4") pod "e604f635-e358-49c5-8391-06357a2c2320" (UID: "e604f635-e358-49c5-8391-06357a2c2320"). InnerVolumeSpecName "kube-api-access-4trz4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 08:45:51.813474 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:51.813450 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e604f635-e358-49c5-8391-06357a2c2320-util" (OuterVolumeSpecName: "util") pod "e604f635-e358-49c5-8391-06357a2c2320" (UID: "e604f635-e358-49c5-8391-06357a2c2320"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 08:45:51.909082 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:51.909044 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e604f635-e358-49c5-8391-06357a2c2320-bundle\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:45:51.909082 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:51.909077 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e604f635-e358-49c5-8391-06357a2c2320-util\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:45:51.909273 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:51.909090 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4trz4\" (UniqueName: \"kubernetes.io/projected/e604f635-e358-49c5-8391-06357a2c2320-kube-api-access-4trz4\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:45:52.533112 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:52.533031 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5d5746998b-qfmv2" event={"ID":"efa00193-20de-451e-a822-40ea9746a0eb","Type":"ContainerStarted","Data":"b0da26936f91e718bd729bb44c6116f3e6796db6de9e6006ef1302f1d32c671c"} Apr 16 08:45:52.534657 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:52.534626 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq" event={"ID":"e604f635-e358-49c5-8391-06357a2c2320","Type":"ContainerDied","Data":"1468cfc1426a18298e64da2631b4ba2a29d28370d91c6419b7d34e2521707812"} Apr 16 08:45:52.534657 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:52.534646 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835q8rbq" Apr 16 08:45:52.534850 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:52.534653 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1468cfc1426a18298e64da2631b4ba2a29d28370d91c6419b7d34e2521707812" Apr 16 08:45:52.547823 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:52.547751 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5d5746998b-qfmv2" podStartSLOduration=2.071738653 podStartE2EDuration="5.547737866s" podCreationTimestamp="2026-04-16 08:45:47 +0000 UTC" firstStartedPulling="2026-04-16 08:45:48.271782652 +0000 UTC m=+396.640766700" lastFinishedPulling="2026-04-16 08:45:51.747781856 +0000 UTC m=+400.116765913" observedRunningTime="2026-04-16 08:45:52.545941383 +0000 UTC m=+400.914925466" watchObservedRunningTime="2026-04-16 08:45:52.547737866 +0000 UTC m=+400.916721975" Apr 16 08:45:57.509177 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:45:57.509148 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7cbc7f8cc-dd79v" Apr 16 08:46:00.855640 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:00.855604 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc"] Apr 16 08:46:00.856014 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:00.855955 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e604f635-e358-49c5-8391-06357a2c2320" containerName="pull" Apr 16 08:46:00.856014 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:00.855966 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e604f635-e358-49c5-8391-06357a2c2320" containerName="pull" Apr 16 08:46:00.856014 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:00.855978 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e604f635-e358-49c5-8391-06357a2c2320" containerName="util" Apr 16 08:46:00.856014 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:00.855983 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e604f635-e358-49c5-8391-06357a2c2320" containerName="util" Apr 16 08:46:00.856014 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:00.855990 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e604f635-e358-49c5-8391-06357a2c2320" containerName="extract" Apr 16 08:46:00.856014 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:00.855996 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e604f635-e358-49c5-8391-06357a2c2320" containerName="extract" Apr 16 08:46:00.856194 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:00.856044 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="e604f635-e358-49c5-8391-06357a2c2320" containerName="extract" Apr 16 08:46:00.859124 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:00.859109 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc" Apr 16 08:46:00.861967 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:00.861947 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Apr 16 08:46:00.862846 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:00.862824 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tlgdk\"" Apr 16 08:46:00.862977 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:00.862960 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Apr 16 08:46:00.935510 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:00.935473 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc"] Apr 16 08:46:00.984701 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:00.984667 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e25c0466-ed4f-4d8c-b2c5-159e90e2f16e-bundle\") pod \"d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc\" (UID: \"e25c0466-ed4f-4d8c-b2c5-159e90e2f16e\") " pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc" Apr 16 08:46:00.984701 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:00.984705 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e25c0466-ed4f-4d8c-b2c5-159e90e2f16e-util\") pod \"d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc\" (UID: \"e25c0466-ed4f-4d8c-b2c5-159e90e2f16e\") " pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc" Apr 16 08:46:00.984935 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:00.984821 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbtd7\" (UniqueName: \"kubernetes.io/projected/e25c0466-ed4f-4d8c-b2c5-159e90e2f16e-kube-api-access-hbtd7\") pod \"d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc\" (UID: \"e25c0466-ed4f-4d8c-b2c5-159e90e2f16e\") " pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc" Apr 16 08:46:01.085880 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:01.085842 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hbtd7\" (UniqueName: \"kubernetes.io/projected/e25c0466-ed4f-4d8c-b2c5-159e90e2f16e-kube-api-access-hbtd7\") pod \"d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc\" (UID: \"e25c0466-ed4f-4d8c-b2c5-159e90e2f16e\") " pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc" Apr 16 08:46:01.086060 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:01.085939 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e25c0466-ed4f-4d8c-b2c5-159e90e2f16e-bundle\") pod \"d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc\" (UID: \"e25c0466-ed4f-4d8c-b2c5-159e90e2f16e\") " pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc" Apr 16 08:46:01.086060 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:01.085978 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e25c0466-ed4f-4d8c-b2c5-159e90e2f16e-util\") pod \"d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc\" (UID: \"e25c0466-ed4f-4d8c-b2c5-159e90e2f16e\") " pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc" Apr 16 08:46:01.086319 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:01.086299 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e25c0466-ed4f-4d8c-b2c5-159e90e2f16e-bundle\") pod \"d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc\" (UID: \"e25c0466-ed4f-4d8c-b2c5-159e90e2f16e\") " pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc" Apr 16 08:46:01.086354 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:01.086338 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e25c0466-ed4f-4d8c-b2c5-159e90e2f16e-util\") pod \"d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc\" (UID: \"e25c0466-ed4f-4d8c-b2c5-159e90e2f16e\") " pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc" Apr 16 08:46:01.094149 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:01.094126 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbtd7\" (UniqueName: \"kubernetes.io/projected/e25c0466-ed4f-4d8c-b2c5-159e90e2f16e-kube-api-access-hbtd7\") pod \"d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc\" (UID: \"e25c0466-ed4f-4d8c-b2c5-159e90e2f16e\") " pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc" Apr 16 08:46:01.168202 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:01.168105 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc" Apr 16 08:46:01.297970 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:01.297863 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc"] Apr 16 08:46:01.300649 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:46:01.300620 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode25c0466_ed4f_4d8c_b2c5_159e90e2f16e.slice/crio-5ca233e397a0579e2cbcb3270bc0a7dfaf4e2d68c41584076a5984301b2f8700 WatchSource:0}: Error finding container 5ca233e397a0579e2cbcb3270bc0a7dfaf4e2d68c41584076a5984301b2f8700: Status 404 returned error can't find the container with id 5ca233e397a0579e2cbcb3270bc0a7dfaf4e2d68c41584076a5984301b2f8700 Apr 16 08:46:01.572033 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:01.572002 2578 generic.go:358] "Generic (PLEG): container finished" podID="e25c0466-ed4f-4d8c-b2c5-159e90e2f16e" containerID="38ba5935030cc7f1edb570b4017946741dfe41ec327b7dd599171fff1d341fcc" exitCode=0 Apr 16 08:46:01.572234 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:01.572090 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc" event={"ID":"e25c0466-ed4f-4d8c-b2c5-159e90e2f16e","Type":"ContainerDied","Data":"38ba5935030cc7f1edb570b4017946741dfe41ec327b7dd599171fff1d341fcc"} Apr 16 08:46:01.572234 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:01.572141 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc" event={"ID":"e25c0466-ed4f-4d8c-b2c5-159e90e2f16e","Type":"ContainerStarted","Data":"5ca233e397a0579e2cbcb3270bc0a7dfaf4e2d68c41584076a5984301b2f8700"} Apr 16 08:46:02.578230 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:02.578196 2578 generic.go:358] "Generic (PLEG): container finished" podID="e25c0466-ed4f-4d8c-b2c5-159e90e2f16e" containerID="adafa5b3189e3e72c802cb25e41fa0cdc5e536e81c56d140dc48f1d13752929c" exitCode=0 Apr 16 08:46:02.578629 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:02.578282 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc" event={"ID":"e25c0466-ed4f-4d8c-b2c5-159e90e2f16e","Type":"ContainerDied","Data":"adafa5b3189e3e72c802cb25e41fa0cdc5e536e81c56d140dc48f1d13752929c"} Apr 16 08:46:03.583206 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:03.583171 2578 generic.go:358] "Generic (PLEG): container finished" podID="e25c0466-ed4f-4d8c-b2c5-159e90e2f16e" containerID="587936f9b48314764a83974b167175653075a0be357da4b44a75d4ccb2535dd7" exitCode=0 Apr 16 08:46:03.583588 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:03.583247 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc" event={"ID":"e25c0466-ed4f-4d8c-b2c5-159e90e2f16e","Type":"ContainerDied","Data":"587936f9b48314764a83974b167175653075a0be357da4b44a75d4ccb2535dd7"} Apr 16 08:46:04.709101 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:04.709080 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc" Apr 16 08:46:04.819922 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:04.819893 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e25c0466-ed4f-4d8c-b2c5-159e90e2f16e-util\") pod \"e25c0466-ed4f-4d8c-b2c5-159e90e2f16e\" (UID: \"e25c0466-ed4f-4d8c-b2c5-159e90e2f16e\") " Apr 16 08:46:04.820110 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:04.819932 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e25c0466-ed4f-4d8c-b2c5-159e90e2f16e-bundle\") pod \"e25c0466-ed4f-4d8c-b2c5-159e90e2f16e\" (UID: \"e25c0466-ed4f-4d8c-b2c5-159e90e2f16e\") " Apr 16 08:46:04.820110 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:04.819990 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbtd7\" (UniqueName: \"kubernetes.io/projected/e25c0466-ed4f-4d8c-b2c5-159e90e2f16e-kube-api-access-hbtd7\") pod \"e25c0466-ed4f-4d8c-b2c5-159e90e2f16e\" (UID: \"e25c0466-ed4f-4d8c-b2c5-159e90e2f16e\") " Apr 16 08:46:04.820893 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:04.820869 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e25c0466-ed4f-4d8c-b2c5-159e90e2f16e-bundle" (OuterVolumeSpecName: "bundle") pod "e25c0466-ed4f-4d8c-b2c5-159e90e2f16e" (UID: "e25c0466-ed4f-4d8c-b2c5-159e90e2f16e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 08:46:04.822171 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:04.822142 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e25c0466-ed4f-4d8c-b2c5-159e90e2f16e-kube-api-access-hbtd7" (OuterVolumeSpecName: "kube-api-access-hbtd7") pod "e25c0466-ed4f-4d8c-b2c5-159e90e2f16e" (UID: "e25c0466-ed4f-4d8c-b2c5-159e90e2f16e"). InnerVolumeSpecName "kube-api-access-hbtd7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 08:46:04.825710 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:04.825689 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e25c0466-ed4f-4d8c-b2c5-159e90e2f16e-util" (OuterVolumeSpecName: "util") pod "e25c0466-ed4f-4d8c-b2c5-159e90e2f16e" (UID: "e25c0466-ed4f-4d8c-b2c5-159e90e2f16e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 08:46:04.921546 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:04.921491 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbtd7\" (UniqueName: \"kubernetes.io/projected/e25c0466-ed4f-4d8c-b2c5-159e90e2f16e-kube-api-access-hbtd7\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:46:04.921546 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:04.921513 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e25c0466-ed4f-4d8c-b2c5-159e90e2f16e-util\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:46:04.921546 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:04.921523 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e25c0466-ed4f-4d8c-b2c5-159e90e2f16e-bundle\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:46:05.592252 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:05.592221 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc" event={"ID":"e25c0466-ed4f-4d8c-b2c5-159e90e2f16e","Type":"ContainerDied","Data":"5ca233e397a0579e2cbcb3270bc0a7dfaf4e2d68c41584076a5984301b2f8700"} Apr 16 08:46:05.592252 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:05.592252 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5ca233e397a0579e2cbcb3270bc0a7dfaf4e2d68c41584076a5984301b2f8700" Apr 16 08:46:05.592453 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:05.592259 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c27jvpc" Apr 16 08:46:52.545895 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:52.545863 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg"] Apr 16 08:46:52.546306 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:52.546189 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e25c0466-ed4f-4d8c-b2c5-159e90e2f16e" containerName="extract" Apr 16 08:46:52.546306 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:52.546199 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e25c0466-ed4f-4d8c-b2c5-159e90e2f16e" containerName="extract" Apr 16 08:46:52.546306 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:52.546218 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e25c0466-ed4f-4d8c-b2c5-159e90e2f16e" containerName="pull" Apr 16 08:46:52.546306 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:52.546224 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e25c0466-ed4f-4d8c-b2c5-159e90e2f16e" containerName="pull" Apr 16 08:46:52.546306 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:52.546233 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e25c0466-ed4f-4d8c-b2c5-159e90e2f16e" containerName="util" Apr 16 08:46:52.546306 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:52.546238 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e25c0466-ed4f-4d8c-b2c5-159e90e2f16e" containerName="util" Apr 16 08:46:52.546306 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:52.546306 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="e25c0466-ed4f-4d8c-b2c5-159e90e2f16e" containerName="extract" Apr 16 08:46:52.553157 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:52.553134 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg" Apr 16 08:46:52.555570 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:52.555384 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Apr 16 08:46:52.555570 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:52.555430 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-cmwnl\"" Apr 16 08:46:52.555570 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:52.555477 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Apr 16 08:46:52.556739 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:52.556690 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg"] Apr 16 08:46:52.632368 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:52.632328 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/002fee38-9c5e-4e0d-b3b9-15b1e66d3b62-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg\" (UID: \"002fee38-9c5e-4e0d-b3b9-15b1e66d3b62\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg" Apr 16 08:46:52.632547 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:52.632380 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v75vq\" (UniqueName: \"kubernetes.io/projected/002fee38-9c5e-4e0d-b3b9-15b1e66d3b62-kube-api-access-v75vq\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg\" (UID: \"002fee38-9c5e-4e0d-b3b9-15b1e66d3b62\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg" Apr 16 08:46:52.632547 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:52.632463 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/002fee38-9c5e-4e0d-b3b9-15b1e66d3b62-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg\" (UID: \"002fee38-9c5e-4e0d-b3b9-15b1e66d3b62\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg" Apr 16 08:46:52.733502 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:52.733467 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/002fee38-9c5e-4e0d-b3b9-15b1e66d3b62-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg\" (UID: \"002fee38-9c5e-4e0d-b3b9-15b1e66d3b62\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg" Apr 16 08:46:52.733678 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:52.733515 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v75vq\" (UniqueName: \"kubernetes.io/projected/002fee38-9c5e-4e0d-b3b9-15b1e66d3b62-kube-api-access-v75vq\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg\" (UID: \"002fee38-9c5e-4e0d-b3b9-15b1e66d3b62\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg" Apr 16 08:46:52.733678 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:52.733552 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/002fee38-9c5e-4e0d-b3b9-15b1e66d3b62-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg\" (UID: \"002fee38-9c5e-4e0d-b3b9-15b1e66d3b62\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg" Apr 16 08:46:52.733866 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:52.733845 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/002fee38-9c5e-4e0d-b3b9-15b1e66d3b62-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg\" (UID: \"002fee38-9c5e-4e0d-b3b9-15b1e66d3b62\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg" Apr 16 08:46:52.733928 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:52.733909 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/002fee38-9c5e-4e0d-b3b9-15b1e66d3b62-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg\" (UID: \"002fee38-9c5e-4e0d-b3b9-15b1e66d3b62\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg" Apr 16 08:46:52.741031 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:52.741006 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v75vq\" (UniqueName: \"kubernetes.io/projected/002fee38-9c5e-4e0d-b3b9-15b1e66d3b62-kube-api-access-v75vq\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg\" (UID: \"002fee38-9c5e-4e0d-b3b9-15b1e66d3b62\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg" Apr 16 08:46:52.864664 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:52.864560 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg" Apr 16 08:46:53.018138 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:53.018072 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg"] Apr 16 08:46:53.020375 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:46:53.020347 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod002fee38_9c5e_4e0d_b3b9_15b1e66d3b62.slice/crio-7e1e698895db46232c83b87daacc46d82d91f637acb91a11e0425f47a3866eef WatchSource:0}: Error finding container 7e1e698895db46232c83b87daacc46d82d91f637acb91a11e0425f47a3866eef: Status 404 returned error can't find the container with id 7e1e698895db46232c83b87daacc46d82d91f637acb91a11e0425f47a3866eef Apr 16 08:46:53.335854 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:53.335814 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg"] Apr 16 08:46:53.339406 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:53.339389 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg" Apr 16 08:46:53.346345 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:53.346312 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg"] Apr 16 08:46:53.443475 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:53.443398 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e215abc-0b28-49ee-b7c1-c5c17879ebf1-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg\" (UID: \"7e215abc-0b28-49ee-b7c1-c5c17879ebf1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg" Apr 16 08:46:53.443475 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:53.443443 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e215abc-0b28-49ee-b7c1-c5c17879ebf1-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg\" (UID: \"7e215abc-0b28-49ee-b7c1-c5c17879ebf1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg" Apr 16 08:46:53.443655 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:53.443517 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnz8w\" (UniqueName: \"kubernetes.io/projected/7e215abc-0b28-49ee-b7c1-c5c17879ebf1-kube-api-access-xnz8w\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg\" (UID: \"7e215abc-0b28-49ee-b7c1-c5c17879ebf1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg" Apr 16 08:46:53.544529 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:53.544490 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e215abc-0b28-49ee-b7c1-c5c17879ebf1-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg\" (UID: \"7e215abc-0b28-49ee-b7c1-c5c17879ebf1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg" Apr 16 08:46:53.544691 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:53.544538 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e215abc-0b28-49ee-b7c1-c5c17879ebf1-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg\" (UID: \"7e215abc-0b28-49ee-b7c1-c5c17879ebf1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg" Apr 16 08:46:53.544691 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:53.544663 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xnz8w\" (UniqueName: \"kubernetes.io/projected/7e215abc-0b28-49ee-b7c1-c5c17879ebf1-kube-api-access-xnz8w\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg\" (UID: \"7e215abc-0b28-49ee-b7c1-c5c17879ebf1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg" Apr 16 08:46:53.544905 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:53.544885 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e215abc-0b28-49ee-b7c1-c5c17879ebf1-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg\" (UID: \"7e215abc-0b28-49ee-b7c1-c5c17879ebf1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg" Apr 16 08:46:53.544983 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:53.544965 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e215abc-0b28-49ee-b7c1-c5c17879ebf1-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg\" (UID: \"7e215abc-0b28-49ee-b7c1-c5c17879ebf1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg" Apr 16 08:46:53.552498 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:53.552478 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnz8w\" (UniqueName: \"kubernetes.io/projected/7e215abc-0b28-49ee-b7c1-c5c17879ebf1-kube-api-access-xnz8w\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg\" (UID: \"7e215abc-0b28-49ee-b7c1-c5c17879ebf1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg" Apr 16 08:46:53.649813 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:53.649749 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg" Apr 16 08:46:53.768384 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:53.768356 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg"] Apr 16 08:46:53.771335 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:46:53.771302 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e215abc_0b28_49ee_b7c1_c5c17879ebf1.slice/crio-bae12d21bc11c4fab48d143694f9255c455c8503197b08e85b9704d853e16560 WatchSource:0}: Error finding container bae12d21bc11c4fab48d143694f9255c455c8503197b08e85b9704d853e16560: Status 404 returned error can't find the container with id bae12d21bc11c4fab48d143694f9255c455c8503197b08e85b9704d853e16560 Apr 16 08:46:53.771925 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:53.771895 2578 generic.go:358] "Generic (PLEG): container finished" podID="002fee38-9c5e-4e0d-b3b9-15b1e66d3b62" containerID="31e1ba5e9873833b8130245617c9e230e077417094fe31ed8cf79a2bdd8f206b" exitCode=0 Apr 16 08:46:53.772036 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:53.771960 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg" event={"ID":"002fee38-9c5e-4e0d-b3b9-15b1e66d3b62","Type":"ContainerDied","Data":"31e1ba5e9873833b8130245617c9e230e077417094fe31ed8cf79a2bdd8f206b"} Apr 16 08:46:53.772036 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:53.771994 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg" event={"ID":"002fee38-9c5e-4e0d-b3b9-15b1e66d3b62","Type":"ContainerStarted","Data":"7e1e698895db46232c83b87daacc46d82d91f637acb91a11e0425f47a3866eef"} Apr 16 08:46:54.132533 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.132497 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv"] Apr 16 08:46:54.135892 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.135870 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv" Apr 16 08:46:54.143181 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.143153 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv"] Apr 16 08:46:54.251454 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.251418 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx7k5\" (UniqueName: \"kubernetes.io/projected/b1978bda-f153-481b-84bc-5457e58753c2-kube-api-access-hx7k5\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv\" (UID: \"b1978bda-f153-481b-84bc-5457e58753c2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv" Apr 16 08:46:54.251703 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.251503 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1978bda-f153-481b-84bc-5457e58753c2-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv\" (UID: \"b1978bda-f153-481b-84bc-5457e58753c2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv" Apr 16 08:46:54.251703 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.251536 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1978bda-f153-481b-84bc-5457e58753c2-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv\" (UID: \"b1978bda-f153-481b-84bc-5457e58753c2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv" Apr 16 08:46:54.352049 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.352029 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1978bda-f153-481b-84bc-5457e58753c2-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv\" (UID: \"b1978bda-f153-481b-84bc-5457e58753c2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv" Apr 16 08:46:54.352119 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.352067 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1978bda-f153-481b-84bc-5457e58753c2-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv\" (UID: \"b1978bda-f153-481b-84bc-5457e58753c2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv" Apr 16 08:46:54.352119 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.352097 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hx7k5\" (UniqueName: \"kubernetes.io/projected/b1978bda-f153-481b-84bc-5457e58753c2-kube-api-access-hx7k5\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv\" (UID: \"b1978bda-f153-481b-84bc-5457e58753c2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv" Apr 16 08:46:54.352418 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.352391 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1978bda-f153-481b-84bc-5457e58753c2-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv\" (UID: \"b1978bda-f153-481b-84bc-5457e58753c2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv" Apr 16 08:46:54.352516 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.352417 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1978bda-f153-481b-84bc-5457e58753c2-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv\" (UID: \"b1978bda-f153-481b-84bc-5457e58753c2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv" Apr 16 08:46:54.360135 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.360113 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx7k5\" (UniqueName: \"kubernetes.io/projected/b1978bda-f153-481b-84bc-5457e58753c2-kube-api-access-hx7k5\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv\" (UID: \"b1978bda-f153-481b-84bc-5457e58753c2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv" Apr 16 08:46:54.445371 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.445345 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv" Apr 16 08:46:54.539061 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.539029 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn"] Apr 16 08:46:54.543284 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.543261 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn" Apr 16 08:46:54.549188 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.549159 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn"] Apr 16 08:46:54.571693 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.571672 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv"] Apr 16 08:46:54.574150 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:46:54.574126 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1978bda_f153_481b_84bc_5457e58753c2.slice/crio-f718f16b96369029682b0491ea257238264c30f605567bc0174d04eb75f81c83 WatchSource:0}: Error finding container f718f16b96369029682b0491ea257238264c30f605567bc0174d04eb75f81c83: Status 404 returned error can't find the container with id f718f16b96369029682b0491ea257238264c30f605567bc0174d04eb75f81c83 Apr 16 08:46:54.654351 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.654275 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95qkr\" (UniqueName: \"kubernetes.io/projected/2cfad847-f015-4190-8658-10459885802b-kube-api-access-95qkr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn\" (UID: \"2cfad847-f015-4190-8658-10459885802b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn" Apr 16 08:46:54.654351 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.654317 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2cfad847-f015-4190-8658-10459885802b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn\" (UID: \"2cfad847-f015-4190-8658-10459885802b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn" Apr 16 08:46:54.654523 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.654432 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2cfad847-f015-4190-8658-10459885802b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn\" (UID: \"2cfad847-f015-4190-8658-10459885802b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn" Apr 16 08:46:54.755704 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.755666 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-95qkr\" (UniqueName: \"kubernetes.io/projected/2cfad847-f015-4190-8658-10459885802b-kube-api-access-95qkr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn\" (UID: \"2cfad847-f015-4190-8658-10459885802b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn" Apr 16 08:46:54.755912 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.755710 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2cfad847-f015-4190-8658-10459885802b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn\" (UID: \"2cfad847-f015-4190-8658-10459885802b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn" Apr 16 08:46:54.755912 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.755749 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2cfad847-f015-4190-8658-10459885802b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn\" (UID: \"2cfad847-f015-4190-8658-10459885802b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn" Apr 16 08:46:54.756115 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.756098 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2cfad847-f015-4190-8658-10459885802b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn\" (UID: \"2cfad847-f015-4190-8658-10459885802b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn" Apr 16 08:46:54.756162 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.756121 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2cfad847-f015-4190-8658-10459885802b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn\" (UID: \"2cfad847-f015-4190-8658-10459885802b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn" Apr 16 08:46:54.763877 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.763852 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95qkr\" (UniqueName: \"kubernetes.io/projected/2cfad847-f015-4190-8658-10459885802b-kube-api-access-95qkr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn\" (UID: \"2cfad847-f015-4190-8658-10459885802b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn" Apr 16 08:46:54.777948 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.777922 2578 generic.go:358] "Generic (PLEG): container finished" podID="b1978bda-f153-481b-84bc-5457e58753c2" containerID="e45c75ea94c6d3555226d35d39045ca288be05116f71160c5e8d035fdad234f6" exitCode=0 Apr 16 08:46:54.778124 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.778010 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv" event={"ID":"b1978bda-f153-481b-84bc-5457e58753c2","Type":"ContainerDied","Data":"e45c75ea94c6d3555226d35d39045ca288be05116f71160c5e8d035fdad234f6"} Apr 16 08:46:54.778124 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.778043 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv" event={"ID":"b1978bda-f153-481b-84bc-5457e58753c2","Type":"ContainerStarted","Data":"f718f16b96369029682b0491ea257238264c30f605567bc0174d04eb75f81c83"} Apr 16 08:46:54.779842 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.779820 2578 generic.go:358] "Generic (PLEG): container finished" podID="002fee38-9c5e-4e0d-b3b9-15b1e66d3b62" containerID="b451db4c27ed0ba5fb5e15fc15755848b68ccd0bce50e7b4dfb0331446c5d75b" exitCode=0 Apr 16 08:46:54.779939 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.779879 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg" event={"ID":"002fee38-9c5e-4e0d-b3b9-15b1e66d3b62","Type":"ContainerDied","Data":"b451db4c27ed0ba5fb5e15fc15755848b68ccd0bce50e7b4dfb0331446c5d75b"} Apr 16 08:46:54.781309 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.781290 2578 generic.go:358] "Generic (PLEG): container finished" podID="7e215abc-0b28-49ee-b7c1-c5c17879ebf1" containerID="229635de799548e4cb4716ff0881989ec802182cee9ed5349237466a64bdb640" exitCode=0 Apr 16 08:46:54.781393 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.781333 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg" event={"ID":"7e215abc-0b28-49ee-b7c1-c5c17879ebf1","Type":"ContainerDied","Data":"229635de799548e4cb4716ff0881989ec802182cee9ed5349237466a64bdb640"} Apr 16 08:46:54.781393 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.781355 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg" event={"ID":"7e215abc-0b28-49ee-b7c1-c5c17879ebf1","Type":"ContainerStarted","Data":"bae12d21bc11c4fab48d143694f9255c455c8503197b08e85b9704d853e16560"} Apr 16 08:46:54.854643 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.854620 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn" Apr 16 08:46:54.982820 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:54.982798 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn"] Apr 16 08:46:54.984666 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:46:54.984643 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cfad847_f015_4190_8658_10459885802b.slice/crio-2f2cc5c8ba032b95190e50c10e17a0aff114995d82e70c8ea431762fc144fdcf WatchSource:0}: Error finding container 2f2cc5c8ba032b95190e50c10e17a0aff114995d82e70c8ea431762fc144fdcf: Status 404 returned error can't find the container with id 2f2cc5c8ba032b95190e50c10e17a0aff114995d82e70c8ea431762fc144fdcf Apr 16 08:46:55.786660 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:55.786631 2578 generic.go:358] "Generic (PLEG): container finished" podID="7e215abc-0b28-49ee-b7c1-c5c17879ebf1" containerID="5271ae96c6441a672c727e33b58b86f14def166de59b0bdde79fd52a70e10c86" exitCode=0 Apr 16 08:46:55.787025 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:55.786695 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg" event={"ID":"7e215abc-0b28-49ee-b7c1-c5c17879ebf1","Type":"ContainerDied","Data":"5271ae96c6441a672c727e33b58b86f14def166de59b0bdde79fd52a70e10c86"} Apr 16 08:46:55.788138 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:55.788113 2578 generic.go:358] "Generic (PLEG): container finished" podID="2cfad847-f015-4190-8658-10459885802b" containerID="68874fb38c8983a0dc64774765b89af37bbfa3a5d653b55eb467b55778cc25e1" exitCode=0 Apr 16 08:46:55.788247 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:55.788187 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn" event={"ID":"2cfad847-f015-4190-8658-10459885802b","Type":"ContainerDied","Data":"68874fb38c8983a0dc64774765b89af37bbfa3a5d653b55eb467b55778cc25e1"} Apr 16 08:46:55.788247 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:55.788218 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn" event={"ID":"2cfad847-f015-4190-8658-10459885802b","Type":"ContainerStarted","Data":"2f2cc5c8ba032b95190e50c10e17a0aff114995d82e70c8ea431762fc144fdcf"} Apr 16 08:46:55.789893 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:55.789873 2578 generic.go:358] "Generic (PLEG): container finished" podID="b1978bda-f153-481b-84bc-5457e58753c2" containerID="60bf4e2e69ff9133e9275f29b7764058dab04c07825f1d2dc5b2d39b45354d79" exitCode=0 Apr 16 08:46:55.790014 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:55.789994 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv" event={"ID":"b1978bda-f153-481b-84bc-5457e58753c2","Type":"ContainerDied","Data":"60bf4e2e69ff9133e9275f29b7764058dab04c07825f1d2dc5b2d39b45354d79"} Apr 16 08:46:55.792169 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:55.792140 2578 generic.go:358] "Generic (PLEG): container finished" podID="002fee38-9c5e-4e0d-b3b9-15b1e66d3b62" containerID="c60c7e28c97ea892e2ba02e712bbfed1bf2c111eaa4726dd73f35fa0a69afa3d" exitCode=0 Apr 16 08:46:55.792237 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:55.792201 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg" event={"ID":"002fee38-9c5e-4e0d-b3b9-15b1e66d3b62","Type":"ContainerDied","Data":"c60c7e28c97ea892e2ba02e712bbfed1bf2c111eaa4726dd73f35fa0a69afa3d"} Apr 16 08:46:56.798143 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:56.798109 2578 generic.go:358] "Generic (PLEG): container finished" podID="7e215abc-0b28-49ee-b7c1-c5c17879ebf1" containerID="2518736f0c662847be03e07c983a5309cfaede831666201c5e3b1defbbd55173" exitCode=0 Apr 16 08:46:56.798530 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:56.798170 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg" event={"ID":"7e215abc-0b28-49ee-b7c1-c5c17879ebf1","Type":"ContainerDied","Data":"2518736f0c662847be03e07c983a5309cfaede831666201c5e3b1defbbd55173"} Apr 16 08:46:56.799849 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:56.799825 2578 generic.go:358] "Generic (PLEG): container finished" podID="2cfad847-f015-4190-8658-10459885802b" containerID="c3b9265332f3186e41283ab60ec7a8a0b1c73d986039d3ddf36daef6b34fe33d" exitCode=0 Apr 16 08:46:56.799966 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:56.799909 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn" event={"ID":"2cfad847-f015-4190-8658-10459885802b","Type":"ContainerDied","Data":"c3b9265332f3186e41283ab60ec7a8a0b1c73d986039d3ddf36daef6b34fe33d"} Apr 16 08:46:56.801891 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:56.801865 2578 generic.go:358] "Generic (PLEG): container finished" podID="b1978bda-f153-481b-84bc-5457e58753c2" containerID="656e9168a76cfc50af86d98ed8f8c8a796e84bdb3bd8f758e6cb431538f20c21" exitCode=0 Apr 16 08:46:56.801987 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:56.801937 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv" event={"ID":"b1978bda-f153-481b-84bc-5457e58753c2","Type":"ContainerDied","Data":"656e9168a76cfc50af86d98ed8f8c8a796e84bdb3bd8f758e6cb431538f20c21"} Apr 16 08:46:56.937835 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:56.937811 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg" Apr 16 08:46:57.079131 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:57.079033 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v75vq\" (UniqueName: \"kubernetes.io/projected/002fee38-9c5e-4e0d-b3b9-15b1e66d3b62-kube-api-access-v75vq\") pod \"002fee38-9c5e-4e0d-b3b9-15b1e66d3b62\" (UID: \"002fee38-9c5e-4e0d-b3b9-15b1e66d3b62\") " Apr 16 08:46:57.079131 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:57.079085 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/002fee38-9c5e-4e0d-b3b9-15b1e66d3b62-bundle\") pod \"002fee38-9c5e-4e0d-b3b9-15b1e66d3b62\" (UID: \"002fee38-9c5e-4e0d-b3b9-15b1e66d3b62\") " Apr 16 08:46:57.079353 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:57.079152 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/002fee38-9c5e-4e0d-b3b9-15b1e66d3b62-util\") pod \"002fee38-9c5e-4e0d-b3b9-15b1e66d3b62\" (UID: \"002fee38-9c5e-4e0d-b3b9-15b1e66d3b62\") " Apr 16 08:46:57.079624 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:57.079602 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/002fee38-9c5e-4e0d-b3b9-15b1e66d3b62-bundle" (OuterVolumeSpecName: "bundle") pod "002fee38-9c5e-4e0d-b3b9-15b1e66d3b62" (UID: "002fee38-9c5e-4e0d-b3b9-15b1e66d3b62"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 08:46:57.081463 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:57.081439 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/002fee38-9c5e-4e0d-b3b9-15b1e66d3b62-kube-api-access-v75vq" (OuterVolumeSpecName: "kube-api-access-v75vq") pod "002fee38-9c5e-4e0d-b3b9-15b1e66d3b62" (UID: "002fee38-9c5e-4e0d-b3b9-15b1e66d3b62"). InnerVolumeSpecName "kube-api-access-v75vq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 08:46:57.084301 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:57.084276 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/002fee38-9c5e-4e0d-b3b9-15b1e66d3b62-util" (OuterVolumeSpecName: "util") pod "002fee38-9c5e-4e0d-b3b9-15b1e66d3b62" (UID: "002fee38-9c5e-4e0d-b3b9-15b1e66d3b62"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 08:46:57.180081 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:57.180044 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/002fee38-9c5e-4e0d-b3b9-15b1e66d3b62-util\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:46:57.180081 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:57.180078 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v75vq\" (UniqueName: \"kubernetes.io/projected/002fee38-9c5e-4e0d-b3b9-15b1e66d3b62-kube-api-access-v75vq\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:46:57.180081 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:57.180088 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/002fee38-9c5e-4e0d-b3b9-15b1e66d3b62-bundle\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:46:57.810122 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:57.810092 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg" event={"ID":"002fee38-9c5e-4e0d-b3b9-15b1e66d3b62","Type":"ContainerDied","Data":"7e1e698895db46232c83b87daacc46d82d91f637acb91a11e0425f47a3866eef"} Apr 16 08:46:57.810122 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:57.810124 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e1e698895db46232c83b87daacc46d82d91f637acb91a11e0425f47a3866eef" Apr 16 08:46:57.810558 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:57.810127 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg" Apr 16 08:46:57.812040 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:57.812015 2578 generic.go:358] "Generic (PLEG): container finished" podID="2cfad847-f015-4190-8658-10459885802b" containerID="2d22c3568087f76b5fa3426fae3190e0f4d265c82ab8ea9bd758e5501afde2f3" exitCode=0 Apr 16 08:46:57.812170 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:57.812045 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn" event={"ID":"2cfad847-f015-4190-8658-10459885802b","Type":"ContainerDied","Data":"2d22c3568087f76b5fa3426fae3190e0f4d265c82ab8ea9bd758e5501afde2f3"} Apr 16 08:46:57.964569 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:57.964551 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv" Apr 16 08:46:57.967695 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:57.967676 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg" Apr 16 08:46:58.088439 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:58.088356 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e215abc-0b28-49ee-b7c1-c5c17879ebf1-bundle\") pod \"7e215abc-0b28-49ee-b7c1-c5c17879ebf1\" (UID: \"7e215abc-0b28-49ee-b7c1-c5c17879ebf1\") " Apr 16 08:46:58.088439 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:58.088416 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1978bda-f153-481b-84bc-5457e58753c2-util\") pod \"b1978bda-f153-481b-84bc-5457e58753c2\" (UID: \"b1978bda-f153-481b-84bc-5457e58753c2\") " Apr 16 08:46:58.088649 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:58.088457 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnz8w\" (UniqueName: \"kubernetes.io/projected/7e215abc-0b28-49ee-b7c1-c5c17879ebf1-kube-api-access-xnz8w\") pod \"7e215abc-0b28-49ee-b7c1-c5c17879ebf1\" (UID: \"7e215abc-0b28-49ee-b7c1-c5c17879ebf1\") " Apr 16 08:46:58.088649 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:58.088482 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1978bda-f153-481b-84bc-5457e58753c2-bundle\") pod \"b1978bda-f153-481b-84bc-5457e58753c2\" (UID: \"b1978bda-f153-481b-84bc-5457e58753c2\") " Apr 16 08:46:58.088649 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:58.088530 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e215abc-0b28-49ee-b7c1-c5c17879ebf1-util\") pod \"7e215abc-0b28-49ee-b7c1-c5c17879ebf1\" (UID: \"7e215abc-0b28-49ee-b7c1-c5c17879ebf1\") " Apr 16 08:46:58.088649 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:58.088561 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hx7k5\" (UniqueName: \"kubernetes.io/projected/b1978bda-f153-481b-84bc-5457e58753c2-kube-api-access-hx7k5\") pod \"b1978bda-f153-481b-84bc-5457e58753c2\" (UID: \"b1978bda-f153-481b-84bc-5457e58753c2\") " Apr 16 08:46:58.088905 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:58.088865 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e215abc-0b28-49ee-b7c1-c5c17879ebf1-bundle" (OuterVolumeSpecName: "bundle") pod "7e215abc-0b28-49ee-b7c1-c5c17879ebf1" (UID: "7e215abc-0b28-49ee-b7c1-c5c17879ebf1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 08:46:58.089068 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:58.089046 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1978bda-f153-481b-84bc-5457e58753c2-bundle" (OuterVolumeSpecName: "bundle") pod "b1978bda-f153-481b-84bc-5457e58753c2" (UID: "b1978bda-f153-481b-84bc-5457e58753c2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 08:46:58.090713 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:58.090682 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e215abc-0b28-49ee-b7c1-c5c17879ebf1-kube-api-access-xnz8w" (OuterVolumeSpecName: "kube-api-access-xnz8w") pod "7e215abc-0b28-49ee-b7c1-c5c17879ebf1" (UID: "7e215abc-0b28-49ee-b7c1-c5c17879ebf1"). InnerVolumeSpecName "kube-api-access-xnz8w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 08:46:58.090926 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:58.090735 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1978bda-f153-481b-84bc-5457e58753c2-kube-api-access-hx7k5" (OuterVolumeSpecName: "kube-api-access-hx7k5") pod "b1978bda-f153-481b-84bc-5457e58753c2" (UID: "b1978bda-f153-481b-84bc-5457e58753c2"). InnerVolumeSpecName "kube-api-access-hx7k5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 08:46:58.096812 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:58.096787 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e215abc-0b28-49ee-b7c1-c5c17879ebf1-util" (OuterVolumeSpecName: "util") pod "7e215abc-0b28-49ee-b7c1-c5c17879ebf1" (UID: "7e215abc-0b28-49ee-b7c1-c5c17879ebf1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 08:46:58.097170 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:58.097147 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1978bda-f153-481b-84bc-5457e58753c2-util" (OuterVolumeSpecName: "util") pod "b1978bda-f153-481b-84bc-5457e58753c2" (UID: "b1978bda-f153-481b-84bc-5457e58753c2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 08:46:58.189235 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:58.189205 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1978bda-f153-481b-84bc-5457e58753c2-util\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:46:58.189235 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:58.189233 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnz8w\" (UniqueName: \"kubernetes.io/projected/7e215abc-0b28-49ee-b7c1-c5c17879ebf1-kube-api-access-xnz8w\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:46:58.189421 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:58.189244 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1978bda-f153-481b-84bc-5457e58753c2-bundle\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:46:58.189421 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:58.189253 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e215abc-0b28-49ee-b7c1-c5c17879ebf1-util\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:46:58.189421 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:58.189261 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hx7k5\" (UniqueName: \"kubernetes.io/projected/b1978bda-f153-481b-84bc-5457e58753c2-kube-api-access-hx7k5\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:46:58.189421 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:58.189270 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e215abc-0b28-49ee-b7c1-c5c17879ebf1-bundle\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:46:58.818445 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:58.818414 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg" event={"ID":"7e215abc-0b28-49ee-b7c1-c5c17879ebf1","Type":"ContainerDied","Data":"bae12d21bc11c4fab48d143694f9255c455c8503197b08e85b9704d853e16560"} Apr 16 08:46:58.818921 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:58.818454 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bae12d21bc11c4fab48d143694f9255c455c8503197b08e85b9704d853e16560" Apr 16 08:46:58.818921 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:58.818423 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg" Apr 16 08:46:58.820136 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:58.820105 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv" event={"ID":"b1978bda-f153-481b-84bc-5457e58753c2","Type":"ContainerDied","Data":"f718f16b96369029682b0491ea257238264c30f605567bc0174d04eb75f81c83"} Apr 16 08:46:58.820250 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:58.820144 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f718f16b96369029682b0491ea257238264c30f605567bc0174d04eb75f81c83" Apr 16 08:46:58.820250 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:58.820127 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv" Apr 16 08:46:58.946820 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:58.946793 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn" Apr 16 08:46:59.097753 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:59.097657 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95qkr\" (UniqueName: \"kubernetes.io/projected/2cfad847-f015-4190-8658-10459885802b-kube-api-access-95qkr\") pod \"2cfad847-f015-4190-8658-10459885802b\" (UID: \"2cfad847-f015-4190-8658-10459885802b\") " Apr 16 08:46:59.097753 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:59.097702 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2cfad847-f015-4190-8658-10459885802b-bundle\") pod \"2cfad847-f015-4190-8658-10459885802b\" (UID: \"2cfad847-f015-4190-8658-10459885802b\") " Apr 16 08:46:59.097753 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:59.097730 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2cfad847-f015-4190-8658-10459885802b-util\") pod \"2cfad847-f015-4190-8658-10459885802b\" (UID: \"2cfad847-f015-4190-8658-10459885802b\") " Apr 16 08:46:59.098374 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:59.098339 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cfad847-f015-4190-8658-10459885802b-bundle" (OuterVolumeSpecName: "bundle") pod "2cfad847-f015-4190-8658-10459885802b" (UID: "2cfad847-f015-4190-8658-10459885802b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 08:46:59.099934 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:59.099913 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cfad847-f015-4190-8658-10459885802b-kube-api-access-95qkr" (OuterVolumeSpecName: "kube-api-access-95qkr") pod "2cfad847-f015-4190-8658-10459885802b" (UID: "2cfad847-f015-4190-8658-10459885802b"). InnerVolumeSpecName "kube-api-access-95qkr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 08:46:59.104591 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:59.104552 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cfad847-f015-4190-8658-10459885802b-util" (OuterVolumeSpecName: "util") pod "2cfad847-f015-4190-8658-10459885802b" (UID: "2cfad847-f015-4190-8658-10459885802b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 08:46:59.198501 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:59.198459 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95qkr\" (UniqueName: \"kubernetes.io/projected/2cfad847-f015-4190-8658-10459885802b-kube-api-access-95qkr\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:46:59.198501 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:59.198499 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2cfad847-f015-4190-8658-10459885802b-bundle\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:46:59.198501 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:59.198513 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2cfad847-f015-4190-8658-10459885802b-util\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:46:59.825661 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:59.825621 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn" event={"ID":"2cfad847-f015-4190-8658-10459885802b","Type":"ContainerDied","Data":"2f2cc5c8ba032b95190e50c10e17a0aff114995d82e70c8ea431762fc144fdcf"} Apr 16 08:46:59.825661 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:59.825661 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2f2cc5c8ba032b95190e50c10e17a0aff114995d82e70c8ea431762fc144fdcf" Apr 16 08:46:59.826093 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:46:59.825707 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn" Apr 16 08:47:03.348062 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:03.348035 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bfd7b5db7-xjtmm"] Apr 16 08:47:18.842385 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842350 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-dnfkq"] Apr 16 08:47:18.842803 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842689 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1978bda-f153-481b-84bc-5457e58753c2" containerName="pull" Apr 16 08:47:18.842803 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842699 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1978bda-f153-481b-84bc-5457e58753c2" containerName="pull" Apr 16 08:47:18.842803 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842706 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cfad847-f015-4190-8658-10459885802b" containerName="extract" Apr 16 08:47:18.842803 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842711 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cfad847-f015-4190-8658-10459885802b" containerName="extract" Apr 16 08:47:18.842803 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842720 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cfad847-f015-4190-8658-10459885802b" containerName="pull" Apr 16 08:47:18.842803 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842726 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cfad847-f015-4190-8658-10459885802b" containerName="pull" Apr 16 08:47:18.842803 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842735 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1978bda-f153-481b-84bc-5457e58753c2" containerName="extract" Apr 16 08:47:18.842803 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842740 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1978bda-f153-481b-84bc-5457e58753c2" containerName="extract" Apr 16 08:47:18.842803 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842753 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e215abc-0b28-49ee-b7c1-c5c17879ebf1" containerName="util" Apr 16 08:47:18.842803 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842757 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e215abc-0b28-49ee-b7c1-c5c17879ebf1" containerName="util" Apr 16 08:47:18.842803 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842780 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e215abc-0b28-49ee-b7c1-c5c17879ebf1" containerName="pull" Apr 16 08:47:18.842803 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842786 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e215abc-0b28-49ee-b7c1-c5c17879ebf1" containerName="pull" Apr 16 08:47:18.842803 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842796 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cfad847-f015-4190-8658-10459885802b" containerName="util" Apr 16 08:47:18.842803 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842801 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cfad847-f015-4190-8658-10459885802b" containerName="util" Apr 16 08:47:18.842803 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842809 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="002fee38-9c5e-4e0d-b3b9-15b1e66d3b62" containerName="extract" Apr 16 08:47:18.843298 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842816 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="002fee38-9c5e-4e0d-b3b9-15b1e66d3b62" containerName="extract" Apr 16 08:47:18.843298 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842822 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e215abc-0b28-49ee-b7c1-c5c17879ebf1" containerName="extract" Apr 16 08:47:18.843298 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842827 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e215abc-0b28-49ee-b7c1-c5c17879ebf1" containerName="extract" Apr 16 08:47:18.843298 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842835 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="002fee38-9c5e-4e0d-b3b9-15b1e66d3b62" containerName="util" Apr 16 08:47:18.843298 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842840 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="002fee38-9c5e-4e0d-b3b9-15b1e66d3b62" containerName="util" Apr 16 08:47:18.843298 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842845 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1978bda-f153-481b-84bc-5457e58753c2" containerName="util" Apr 16 08:47:18.843298 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842850 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1978bda-f153-481b-84bc-5457e58753c2" containerName="util" Apr 16 08:47:18.843298 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842859 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="002fee38-9c5e-4e0d-b3b9-15b1e66d3b62" containerName="pull" Apr 16 08:47:18.843298 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842864 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="002fee38-9c5e-4e0d-b3b9-15b1e66d3b62" containerName="pull" Apr 16 08:47:18.843298 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842925 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="2cfad847-f015-4190-8658-10459885802b" containerName="extract" Apr 16 08:47:18.843298 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842934 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="7e215abc-0b28-49ee-b7c1-c5c17879ebf1" containerName="extract" Apr 16 08:47:18.843298 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842943 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="b1978bda-f153-481b-84bc-5457e58753c2" containerName="extract" Apr 16 08:47:18.843298 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.842949 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="002fee38-9c5e-4e0d-b3b9-15b1e66d3b62" containerName="extract" Apr 16 08:47:18.847276 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.847255 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-dnfkq" Apr 16 08:47:18.849288 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.849263 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Apr 16 08:47:18.849394 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.849266 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Apr 16 08:47:18.849845 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.849832 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Apr 16 08:47:18.849891 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.849832 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-6d4sw\"" Apr 16 08:47:18.854712 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.854607 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-dnfkq"] Apr 16 08:47:18.864995 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.864962 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgjjv\" (UniqueName: \"kubernetes.io/projected/c910e87f-130e-4e62-9adb-7e289d66c95e-kube-api-access-fgjjv\") pod \"dns-operator-controller-manager-648d5c98bc-dnfkq\" (UID: \"c910e87f-130e-4e62-9adb-7e289d66c95e\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-dnfkq" Apr 16 08:47:18.965643 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.965606 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fgjjv\" (UniqueName: \"kubernetes.io/projected/c910e87f-130e-4e62-9adb-7e289d66c95e-kube-api-access-fgjjv\") pod \"dns-operator-controller-manager-648d5c98bc-dnfkq\" (UID: \"c910e87f-130e-4e62-9adb-7e289d66c95e\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-dnfkq" Apr 16 08:47:18.973415 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:18.973379 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgjjv\" (UniqueName: \"kubernetes.io/projected/c910e87f-130e-4e62-9adb-7e289d66c95e-kube-api-access-fgjjv\") pod \"dns-operator-controller-manager-648d5c98bc-dnfkq\" (UID: \"c910e87f-130e-4e62-9adb-7e289d66c95e\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-dnfkq" Apr 16 08:47:19.158690 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:19.158593 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-dnfkq" Apr 16 08:47:19.290961 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:19.290928 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-dnfkq"] Apr 16 08:47:19.293377 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:47:19.293346 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc910e87f_130e_4e62_9adb_7e289d66c95e.slice/crio-9e324c92944d8893467437903244f712d919d398df171ff4fcef02c2579c6f4e WatchSource:0}: Error finding container 9e324c92944d8893467437903244f712d919d398df171ff4fcef02c2579c6f4e: Status 404 returned error can't find the container with id 9e324c92944d8893467437903244f712d919d398df171ff4fcef02c2579c6f4e Apr 16 08:47:19.443658 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:19.443573 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-9gs8x"] Apr 16 08:47:19.448382 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:19.448359 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-9gs8x" Apr 16 08:47:19.450590 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:19.450565 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Apr 16 08:47:19.450704 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:19.450619 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-cmwnl\"" Apr 16 08:47:19.450704 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:19.450691 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Apr 16 08:47:19.456016 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:19.455987 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-9gs8x"] Apr 16 08:47:19.469851 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:19.469824 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs9b5\" (UniqueName: \"kubernetes.io/projected/cb3721bd-863e-4a70-8ef3-f341b3306113-kube-api-access-bs9b5\") pod \"kuadrant-console-plugin-6cb54b5c86-9gs8x\" (UID: \"cb3721bd-863e-4a70-8ef3-f341b3306113\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-9gs8x" Apr 16 08:47:19.470010 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:19.469883 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb3721bd-863e-4a70-8ef3-f341b3306113-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-9gs8x\" (UID: \"cb3721bd-863e-4a70-8ef3-f341b3306113\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-9gs8x" Apr 16 08:47:19.470010 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:19.469929 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cb3721bd-863e-4a70-8ef3-f341b3306113-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-9gs8x\" (UID: \"cb3721bd-863e-4a70-8ef3-f341b3306113\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-9gs8x" Apr 16 08:47:19.571235 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:19.571187 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb3721bd-863e-4a70-8ef3-f341b3306113-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-9gs8x\" (UID: \"cb3721bd-863e-4a70-8ef3-f341b3306113\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-9gs8x" Apr 16 08:47:19.571419 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:19.571253 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cb3721bd-863e-4a70-8ef3-f341b3306113-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-9gs8x\" (UID: \"cb3721bd-863e-4a70-8ef3-f341b3306113\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-9gs8x" Apr 16 08:47:19.571419 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:19.571287 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bs9b5\" (UniqueName: \"kubernetes.io/projected/cb3721bd-863e-4a70-8ef3-f341b3306113-kube-api-access-bs9b5\") pod \"kuadrant-console-plugin-6cb54b5c86-9gs8x\" (UID: \"cb3721bd-863e-4a70-8ef3-f341b3306113\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-9gs8x" Apr 16 08:47:19.571419 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:47:19.571348 2578 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Apr 16 08:47:19.571602 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:47:19.571447 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb3721bd-863e-4a70-8ef3-f341b3306113-plugin-serving-cert podName:cb3721bd-863e-4a70-8ef3-f341b3306113 nodeName:}" failed. No retries permitted until 2026-04-16 08:47:20.07142395 +0000 UTC m=+488.440407988 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/cb3721bd-863e-4a70-8ef3-f341b3306113-plugin-serving-cert") pod "kuadrant-console-plugin-6cb54b5c86-9gs8x" (UID: "cb3721bd-863e-4a70-8ef3-f341b3306113") : secret "plugin-serving-cert" not found Apr 16 08:47:19.571921 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:19.571904 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cb3721bd-863e-4a70-8ef3-f341b3306113-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-9gs8x\" (UID: \"cb3721bd-863e-4a70-8ef3-f341b3306113\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-9gs8x" Apr 16 08:47:19.579227 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:19.579194 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs9b5\" (UniqueName: \"kubernetes.io/projected/cb3721bd-863e-4a70-8ef3-f341b3306113-kube-api-access-bs9b5\") pod \"kuadrant-console-plugin-6cb54b5c86-9gs8x\" (UID: \"cb3721bd-863e-4a70-8ef3-f341b3306113\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-9gs8x" Apr 16 08:47:19.898671 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:19.898627 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-dnfkq" event={"ID":"c910e87f-130e-4e62-9adb-7e289d66c95e","Type":"ContainerStarted","Data":"9e324c92944d8893467437903244f712d919d398df171ff4fcef02c2579c6f4e"} Apr 16 08:47:20.075680 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:20.075643 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb3721bd-863e-4a70-8ef3-f341b3306113-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-9gs8x\" (UID: \"cb3721bd-863e-4a70-8ef3-f341b3306113\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-9gs8x" Apr 16 08:47:20.078126 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:20.078099 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb3721bd-863e-4a70-8ef3-f341b3306113-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-9gs8x\" (UID: \"cb3721bd-863e-4a70-8ef3-f341b3306113\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-9gs8x" Apr 16 08:47:20.359431 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:20.359403 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-9gs8x" Apr 16 08:47:20.713400 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:20.713371 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-9gs8x"] Apr 16 08:47:20.735469 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:47:20.735421 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb3721bd_863e_4a70_8ef3_f341b3306113.slice/crio-763a5782976afc12a126cd0fdd2b7586d1c5437afa967df85aedf1076908e539 WatchSource:0}: Error finding container 763a5782976afc12a126cd0fdd2b7586d1c5437afa967df85aedf1076908e539: Status 404 returned error can't find the container with id 763a5782976afc12a126cd0fdd2b7586d1c5437afa967df85aedf1076908e539 Apr 16 08:47:20.903322 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:20.903233 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-9gs8x" event={"ID":"cb3721bd-863e-4a70-8ef3-f341b3306113","Type":"ContainerStarted","Data":"763a5782976afc12a126cd0fdd2b7586d1c5437afa967df85aedf1076908e539"} Apr 16 08:47:21.908163 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:21.908130 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-dnfkq" event={"ID":"c910e87f-130e-4e62-9adb-7e289d66c95e","Type":"ContainerStarted","Data":"bc140932f020b75aba94099774f6f285b8726200be5332f2486bd0a0a60fa1ea"} Apr 16 08:47:21.908567 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:21.908286 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-dnfkq" Apr 16 08:47:21.924923 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:21.924879 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-dnfkq" podStartSLOduration=1.8728834939999999 podStartE2EDuration="3.924865685s" podCreationTimestamp="2026-04-16 08:47:18 +0000 UTC" firstStartedPulling="2026-04-16 08:47:19.295348435 +0000 UTC m=+487.664332471" lastFinishedPulling="2026-04-16 08:47:21.347330627 +0000 UTC m=+489.716314662" observedRunningTime="2026-04-16 08:47:21.923832051 +0000 UTC m=+490.292816109" watchObservedRunningTime="2026-04-16 08:47:21.924865685 +0000 UTC m=+490.293849742" Apr 16 08:47:28.372808 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.372749 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-7bfd7b5db7-xjtmm" podUID="b783e52a-8cd5-450a-aa10-e3eeb7a492ec" containerName="console" containerID="cri-o://6fa83b89679a1259bb1a6af23e7f7dd9733dcf5bbba12b3c80025797f196aaa5" gracePeriod=15 Apr 16 08:47:28.618468 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.618444 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7bfd7b5db7-xjtmm_b783e52a-8cd5-450a-aa10-e3eeb7a492ec/console/0.log" Apr 16 08:47:28.618604 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.618511 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:47:28.652954 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.652866 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-oauth-serving-cert\") pod \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " Apr 16 08:47:28.652954 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.652924 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-service-ca\") pod \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " Apr 16 08:47:28.653174 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.652959 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpgrb\" (UniqueName: \"kubernetes.io/projected/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-kube-api-access-cpgrb\") pod \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " Apr 16 08:47:28.653174 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.653007 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-console-serving-cert\") pod \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " Apr 16 08:47:28.653174 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.653028 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-console-oauth-config\") pod \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " Apr 16 08:47:28.653174 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.653056 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-console-config\") pod \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " Apr 16 08:47:28.653174 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.653087 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-trusted-ca-bundle\") pod \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\" (UID: \"b783e52a-8cd5-450a-aa10-e3eeb7a492ec\") " Apr 16 08:47:28.653419 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.653295 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b783e52a-8cd5-450a-aa10-e3eeb7a492ec" (UID: "b783e52a-8cd5-450a-aa10-e3eeb7a492ec"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 08:47:28.653520 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.653495 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-console-config" (OuterVolumeSpecName: "console-config") pod "b783e52a-8cd5-450a-aa10-e3eeb7a492ec" (UID: "b783e52a-8cd5-450a-aa10-e3eeb7a492ec"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 08:47:28.653563 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.653527 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b783e52a-8cd5-450a-aa10-e3eeb7a492ec" (UID: "b783e52a-8cd5-450a-aa10-e3eeb7a492ec"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 08:47:28.654156 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.654114 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-service-ca" (OuterVolumeSpecName: "service-ca") pod "b783e52a-8cd5-450a-aa10-e3eeb7a492ec" (UID: "b783e52a-8cd5-450a-aa10-e3eeb7a492ec"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 08:47:28.655744 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.655709 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b783e52a-8cd5-450a-aa10-e3eeb7a492ec" (UID: "b783e52a-8cd5-450a-aa10-e3eeb7a492ec"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 08:47:28.655744 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.655744 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-kube-api-access-cpgrb" (OuterVolumeSpecName: "kube-api-access-cpgrb") pod "b783e52a-8cd5-450a-aa10-e3eeb7a492ec" (UID: "b783e52a-8cd5-450a-aa10-e3eeb7a492ec"). InnerVolumeSpecName "kube-api-access-cpgrb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 08:47:28.656161 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.656134 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b783e52a-8cd5-450a-aa10-e3eeb7a492ec" (UID: "b783e52a-8cd5-450a-aa10-e3eeb7a492ec"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 08:47:28.754581 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.754542 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-service-ca\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:47:28.754581 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.754577 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpgrb\" (UniqueName: \"kubernetes.io/projected/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-kube-api-access-cpgrb\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:47:28.754581 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.754591 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-console-serving-cert\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:47:28.754900 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.754600 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-console-oauth-config\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:47:28.754900 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.754609 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-console-config\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:47:28.754900 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.754617 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-trusted-ca-bundle\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:47:28.754900 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.754625 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b783e52a-8cd5-450a-aa10-e3eeb7a492ec-oauth-serving-cert\") on node \"ip-10-0-139-84.ec2.internal\" DevicePath \"\"" Apr 16 08:47:28.941940 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.941850 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7bfd7b5db7-xjtmm_b783e52a-8cd5-450a-aa10-e3eeb7a492ec/console/0.log" Apr 16 08:47:28.941940 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.941906 2578 generic.go:358] "Generic (PLEG): container finished" podID="b783e52a-8cd5-450a-aa10-e3eeb7a492ec" containerID="6fa83b89679a1259bb1a6af23e7f7dd9733dcf5bbba12b3c80025797f196aaa5" exitCode=2 Apr 16 08:47:28.942142 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.941946 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bfd7b5db7-xjtmm" event={"ID":"b783e52a-8cd5-450a-aa10-e3eeb7a492ec","Type":"ContainerDied","Data":"6fa83b89679a1259bb1a6af23e7f7dd9733dcf5bbba12b3c80025797f196aaa5"} Apr 16 08:47:28.942142 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.941968 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bfd7b5db7-xjtmm" Apr 16 08:47:28.942142 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.941992 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bfd7b5db7-xjtmm" event={"ID":"b783e52a-8cd5-450a-aa10-e3eeb7a492ec","Type":"ContainerDied","Data":"09c0111735bd1b886bc38769788b9a1efb51f31c0d61eba5eb29f21f0daba728"} Apr 16 08:47:28.942142 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.942012 2578 scope.go:117] "RemoveContainer" containerID="6fa83b89679a1259bb1a6af23e7f7dd9733dcf5bbba12b3c80025797f196aaa5" Apr 16 08:47:28.951970 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.951951 2578 scope.go:117] "RemoveContainer" containerID="6fa83b89679a1259bb1a6af23e7f7dd9733dcf5bbba12b3c80025797f196aaa5" Apr 16 08:47:28.952258 ip-10-0-139-84 kubenswrapper[2578]: E0416 08:47:28.952238 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fa83b89679a1259bb1a6af23e7f7dd9733dcf5bbba12b3c80025797f196aaa5\": container with ID starting with 6fa83b89679a1259bb1a6af23e7f7dd9733dcf5bbba12b3c80025797f196aaa5 not found: ID does not exist" containerID="6fa83b89679a1259bb1a6af23e7f7dd9733dcf5bbba12b3c80025797f196aaa5" Apr 16 08:47:28.952336 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.952269 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fa83b89679a1259bb1a6af23e7f7dd9733dcf5bbba12b3c80025797f196aaa5"} err="failed to get container status \"6fa83b89679a1259bb1a6af23e7f7dd9733dcf5bbba12b3c80025797f196aaa5\": rpc error: code = NotFound desc = could not find container \"6fa83b89679a1259bb1a6af23e7f7dd9733dcf5bbba12b3c80025797f196aaa5\": container with ID starting with 6fa83b89679a1259bb1a6af23e7f7dd9733dcf5bbba12b3c80025797f196aaa5 not found: ID does not exist" Apr 16 08:47:28.972746 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.972717 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bfd7b5db7-xjtmm"] Apr 16 08:47:28.975482 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:28.975452 2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7bfd7b5db7-xjtmm"] Apr 16 08:47:30.215444 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:30.215398 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b783e52a-8cd5-450a-aa10-e3eeb7a492ec" path="/var/lib/kubelet/pods/b783e52a-8cd5-450a-aa10-e3eeb7a492ec/volumes" Apr 16 08:47:32.916593 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:32.916558 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-dnfkq" Apr 16 08:47:44.021676 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:44.021638 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-9gs8x" event={"ID":"cb3721bd-863e-4a70-8ef3-f341b3306113","Type":"ContainerStarted","Data":"aa8ba136890ab7ab3d0ab851f3dd8a5c555fea4bfc494bac730d0073cbe763ae"} Apr 16 08:47:44.039217 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:47:44.039162 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-9gs8x" podStartSLOduration=2.590809751 podStartE2EDuration="25.039147472s" podCreationTimestamp="2026-04-16 08:47:19 +0000 UTC" firstStartedPulling="2026-04-16 08:47:20.736910101 +0000 UTC m=+489.105894136" lastFinishedPulling="2026-04-16 08:47:43.185247821 +0000 UTC m=+511.554231857" observedRunningTime="2026-04-16 08:47:44.037344237 +0000 UTC m=+512.406328295" watchObservedRunningTime="2026-04-16 08:47:44.039147472 +0000 UTC m=+512.408131545" Apr 16 08:48:06.501480 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:48:06.501442 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:48:06.501921 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:48:06.501848 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b783e52a-8cd5-450a-aa10-e3eeb7a492ec" containerName="console" Apr 16 08:48:06.501921 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:48:06.501861 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b783e52a-8cd5-450a-aa10-e3eeb7a492ec" containerName="console" Apr 16 08:48:06.501994 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:48:06.501938 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="b783e52a-8cd5-450a-aa10-e3eeb7a492ec" containerName="console" Apr 16 08:48:06.912645 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:48:06.912566 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:48:06.912645 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:48:06.912598 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:48:06.912863 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:48:06.912717 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-m75dv" Apr 16 08:48:06.915132 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:48:06.915115 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Apr 16 08:48:07.000436 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:48:07.000402 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3df9c4f5-9aca-43b5-94de-700cbec097b3-config-file\") pod \"limitador-limitador-78c99df468-m75dv\" (UID: \"3df9c4f5-9aca-43b5-94de-700cbec097b3\") " pod="kuadrant-system/limitador-limitador-78c99df468-m75dv" Apr 16 08:48:07.000602 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:48:07.000443 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4sth\" (UniqueName: \"kubernetes.io/projected/3df9c4f5-9aca-43b5-94de-700cbec097b3-kube-api-access-v4sth\") pod \"limitador-limitador-78c99df468-m75dv\" (UID: \"3df9c4f5-9aca-43b5-94de-700cbec097b3\") " pod="kuadrant-system/limitador-limitador-78c99df468-m75dv" Apr 16 08:48:07.100890 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:48:07.100852 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3df9c4f5-9aca-43b5-94de-700cbec097b3-config-file\") pod \"limitador-limitador-78c99df468-m75dv\" (UID: \"3df9c4f5-9aca-43b5-94de-700cbec097b3\") " pod="kuadrant-system/limitador-limitador-78c99df468-m75dv" Apr 16 08:48:07.100890 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:48:07.100897 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v4sth\" (UniqueName: \"kubernetes.io/projected/3df9c4f5-9aca-43b5-94de-700cbec097b3-kube-api-access-v4sth\") pod \"limitador-limitador-78c99df468-m75dv\" (UID: \"3df9c4f5-9aca-43b5-94de-700cbec097b3\") " pod="kuadrant-system/limitador-limitador-78c99df468-m75dv" Apr 16 08:48:07.101654 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:48:07.101630 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3df9c4f5-9aca-43b5-94de-700cbec097b3-config-file\") pod \"limitador-limitador-78c99df468-m75dv\" (UID: \"3df9c4f5-9aca-43b5-94de-700cbec097b3\") " pod="kuadrant-system/limitador-limitador-78c99df468-m75dv" Apr 16 08:48:07.111165 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:48:07.111138 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4sth\" (UniqueName: \"kubernetes.io/projected/3df9c4f5-9aca-43b5-94de-700cbec097b3-kube-api-access-v4sth\") pod \"limitador-limitador-78c99df468-m75dv\" (UID: \"3df9c4f5-9aca-43b5-94de-700cbec097b3\") " pod="kuadrant-system/limitador-limitador-78c99df468-m75dv" Apr 16 08:48:07.223135 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:48:07.223103 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-m75dv" Apr 16 08:48:07.371274 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:48:07.371249 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:48:07.373187 ip-10-0-139-84 kubenswrapper[2578]: W0416 08:48:07.373154 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3df9c4f5_9aca_43b5_94de_700cbec097b3.slice/crio-a0eed8ce32597a3b192bc712a110eca5fec5cd568328b015bf851b3eea73c172 WatchSource:0}: Error finding container a0eed8ce32597a3b192bc712a110eca5fec5cd568328b015bf851b3eea73c172: Status 404 returned error can't find the container with id a0eed8ce32597a3b192bc712a110eca5fec5cd568328b015bf851b3eea73c172 Apr 16 08:48:08.115329 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:48:08.115285 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-m75dv" event={"ID":"3df9c4f5-9aca-43b5-94de-700cbec097b3","Type":"ContainerStarted","Data":"a0eed8ce32597a3b192bc712a110eca5fec5cd568328b015bf851b3eea73c172"} Apr 16 08:48:10.125800 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:48:10.125745 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-m75dv" event={"ID":"3df9c4f5-9aca-43b5-94de-700cbec097b3","Type":"ContainerStarted","Data":"abb648e134334a1ab42c0843b8a883755e071a982c108a6827a5e960cbec8155"} Apr 16 08:48:10.126189 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:48:10.125919 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-78c99df468-m75dv" Apr 16 08:48:10.142734 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:48:10.142689 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-78c99df468-m75dv" podStartSLOduration=1.515641611 podStartE2EDuration="4.14267633s" podCreationTimestamp="2026-04-16 08:48:06 +0000 UTC" firstStartedPulling="2026-04-16 08:48:07.374970554 +0000 UTC m=+535.743954589" lastFinishedPulling="2026-04-16 08:48:10.002005273 +0000 UTC m=+538.370989308" observedRunningTime="2026-04-16 08:48:10.140224139 +0000 UTC m=+538.509208196" watchObservedRunningTime="2026-04-16 08:48:10.14267633 +0000 UTC m=+538.511660387" Apr 16 08:48:21.130667 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:48:21.130625 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-78c99df468-m75dv" Apr 16 08:48:41.697856 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:48:41.697817 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:49:20.214165 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:49:20.214137 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:49:22.018878 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:49:22.018847 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:49:23.002973 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:49:23.002935 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:49:26.416805 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:49:26.416754 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:49:34.912220 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:49:34.912181 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:50:06.012649 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:50:06.012604 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:50:56.414324 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:50:56.414241 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:51:03.897293 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:51:03.897259 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:51:35.006813 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:51:35.006751 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:51:50.700292 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:51:50.700253 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:52:29.118731 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:52:29.118651 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:52:46.100289 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:52:46.100250 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:53:00.405957 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:53:00.405923 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:53:16.406106 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:53:16.406070 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:54:07.708585 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:54:07.708547 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:54:17.499888 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:54:17.499853 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:54:34.100787 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:54:34.100724 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:54:42.407713 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:54:42.407683 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:54:59.309441 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:54:59.309400 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:55:07.204245 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:55:07.204211 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:55:40.407436 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:55:40.407362 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:55:48.400460 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:55:48.400423 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:55:57.312629 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:55:57.312594 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:56:05.706911 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:56:05.706876 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:56:14.000447 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:56:14.000404 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:56:31.213097 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:56:31.213058 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:56:41.405264 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:56:41.405232 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:57:28.505447 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:57:28.505370 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:57:37.002482 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:57:37.002442 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:57:46.205398 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:57:46.205361 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:57:54.605582 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:57:54.605540 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:58:03.903271 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:58:03.903231 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:58:12.207419 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:58:12.207376 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:58:21.201919 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:58:21.201881 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:58:25.699170 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:58:25.699083 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:58:29.402129 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:58:29.402089 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:58:39.102081 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:58:39.102048 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:58:47.200932 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:58:47.200892 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:58:56.202729 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:58:56.202688 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:59:04.700017 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:59:04.699977 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:59:14.397522 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:59:14.397482 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:59:22.605586 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:59:22.605545 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:59:31.614323 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:59:31.614283 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:59:39.196920 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:59:39.196873 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:59:48.399393 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:59:48.399346 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 08:59:56.606565 ip-10-0-139-84 kubenswrapper[2578]: I0416 08:59:56.606475 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:01:07.517831 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:01:07.517792 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:01:11.907277 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:01:11.907239 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:01:22.107508 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:01:22.107469 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:01:52.704289 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:01:52.704248 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:02:35.503225 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:02:35.503186 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:02:43.307235 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:02:43.307201 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:02:52.507789 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:02:52.507696 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:03:01.105688 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:03:01.105650 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:03:09.511625 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:03:09.511582 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:03:18.007899 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:03:18.007862 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:03:27.414000 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:03:27.413969 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:03:34.509035 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:03:34.508998 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:03:43.401027 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:03:43.400994 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:03:51.198967 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:03:51.198929 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:04:00.605803 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:04:00.605755 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:04:08.799430 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:04:08.799378 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:04:25.605359 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:04:25.605285 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:04:34.108453 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:04:34.108418 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:04:42.706301 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:04:42.706258 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:04:51.198722 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:04:51.198685 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:05:07.910059 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:05:07.910028 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:05:16.504309 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:05:16.504278 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:05:25.504856 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:05:25.504818 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:05:33.506121 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:05:33.506078 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:05:42.522355 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:05:42.522315 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:05:50.998661 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:05:50.998628 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:06:00.700235 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:06:00.700139 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:06:17.603286 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:06:17.603251 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:06:26.334810 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:06:26.334748 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:06:42.408718 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:06:42.408691 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:06:52.208669 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:06:52.208634 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:06:59.706633 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:06:59.706597 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:07:09.003070 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:07:09.003035 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:07:16.600070 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:07:16.600036 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:07:32.700760 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:07:32.700669 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:07:41.604265 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:07:41.604219 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:07:50.104137 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:07:50.104099 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:07:56.812746 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:07:56.812709 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:08:19.903376 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:19.903340 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:08:32.308544 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:32.308503 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-m75dv"] Apr 16 09:08:39.034685 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:39.034651 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-569944d57d-rk86c_5b4974c7-139a-43f3-b681-f42333201e5f/manager/0.log" Apr 16 09:08:40.141357 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:40.141330 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg_7e215abc-0b28-49ee-b7c1-c5c17879ebf1/util/0.log" Apr 16 09:08:40.147819 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:40.147793 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg_7e215abc-0b28-49ee-b7c1-c5c17879ebf1/pull/0.log" Apr 16 09:08:40.154108 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:40.154087 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg_7e215abc-0b28-49ee-b7c1-c5c17879ebf1/extract/0.log" Apr 16 09:08:40.262971 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:40.262945 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg_002fee38-9c5e-4e0d-b3b9-15b1e66d3b62/util/0.log" Apr 16 09:08:40.269165 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:40.269144 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg_002fee38-9c5e-4e0d-b3b9-15b1e66d3b62/pull/0.log" Apr 16 09:08:40.275659 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:40.275638 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg_002fee38-9c5e-4e0d-b3b9-15b1e66d3b62/extract/0.log" Apr 16 09:08:40.387530 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:40.387503 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn_2cfad847-f015-4190-8658-10459885802b/util/0.log" Apr 16 09:08:40.393451 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:40.393386 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn_2cfad847-f015-4190-8658-10459885802b/pull/0.log" Apr 16 09:08:40.399300 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:40.399284 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn_2cfad847-f015-4190-8658-10459885802b/extract/0.log" Apr 16 09:08:40.508394 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:40.508367 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv_b1978bda-f153-481b-84bc-5457e58753c2/util/0.log" Apr 16 09:08:40.514658 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:40.514634 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv_b1978bda-f153-481b-84bc-5457e58753c2/pull/0.log" Apr 16 09:08:40.520198 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:40.520175 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv_b1978bda-f153-481b-84bc-5457e58753c2/extract/0.log" Apr 16 09:08:40.873275 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:40.873239 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-dnfkq_c910e87f-130e-4e62-9adb-7e289d66c95e/manager/0.log" Apr 16 09:08:40.981548 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:40.981520 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-9gs8x_cb3721bd-863e-4a70-8ef3-f341b3306113/kuadrant-console-plugin/0.log" Apr 16 09:08:41.326941 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:41.326915 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-m75dv_3df9c4f5-9aca-43b5-94de-700cbec097b3/limitador/0.log" Apr 16 09:08:42.134699 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:42.134671 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d5746998b-qfmv2_efa00193-20de-451e-a822-40ea9746a0eb/kube-auth-proxy/0.log" Apr 16 09:08:42.364603 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:42.364573 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54fdbc7b78-dxx97_f45d6895-a6e2-4531-ac1e-6e8c46c51012/router/0.log" Apr 16 09:08:47.515003 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:47.514969 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mmdlf/must-gather-jgp45"] Apr 16 09:08:47.518491 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:47.518475 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mmdlf/must-gather-jgp45" Apr 16 09:08:47.520609 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:47.520584 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mmdlf\"/\"default-dockercfg-9mlmk\"" Apr 16 09:08:47.520609 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:47.520605 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mmdlf\"/\"openshift-service-ca.crt\"" Apr 16 09:08:47.521267 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:47.521251 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mmdlf\"/\"kube-root-ca.crt\"" Apr 16 09:08:47.530707 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:47.530686 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mmdlf/must-gather-jgp45"] Apr 16 09:08:47.672181 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:47.672148 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/de9d4f65-5da6-487a-89bb-6bf7a1236a89-must-gather-output\") pod \"must-gather-jgp45\" (UID: \"de9d4f65-5da6-487a-89bb-6bf7a1236a89\") " pod="openshift-must-gather-mmdlf/must-gather-jgp45" Apr 16 09:08:47.672342 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:47.672193 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkg8x\" (UniqueName: \"kubernetes.io/projected/de9d4f65-5da6-487a-89bb-6bf7a1236a89-kube-api-access-qkg8x\") pod \"must-gather-jgp45\" (UID: \"de9d4f65-5da6-487a-89bb-6bf7a1236a89\") " pod="openshift-must-gather-mmdlf/must-gather-jgp45" Apr 16 09:08:47.773602 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:47.773504 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/de9d4f65-5da6-487a-89bb-6bf7a1236a89-must-gather-output\") pod \"must-gather-jgp45\" (UID: \"de9d4f65-5da6-487a-89bb-6bf7a1236a89\") " pod="openshift-must-gather-mmdlf/must-gather-jgp45" Apr 16 09:08:47.773602 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:47.773555 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qkg8x\" (UniqueName: \"kubernetes.io/projected/de9d4f65-5da6-487a-89bb-6bf7a1236a89-kube-api-access-qkg8x\") pod \"must-gather-jgp45\" (UID: \"de9d4f65-5da6-487a-89bb-6bf7a1236a89\") " pod="openshift-must-gather-mmdlf/must-gather-jgp45" Apr 16 09:08:47.773958 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:47.773927 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/de9d4f65-5da6-487a-89bb-6bf7a1236a89-must-gather-output\") pod \"must-gather-jgp45\" (UID: \"de9d4f65-5da6-487a-89bb-6bf7a1236a89\") " pod="openshift-must-gather-mmdlf/must-gather-jgp45" Apr 16 09:08:47.780995 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:47.780969 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkg8x\" (UniqueName: \"kubernetes.io/projected/de9d4f65-5da6-487a-89bb-6bf7a1236a89-kube-api-access-qkg8x\") pod \"must-gather-jgp45\" (UID: \"de9d4f65-5da6-487a-89bb-6bf7a1236a89\") " pod="openshift-must-gather-mmdlf/must-gather-jgp45" Apr 16 09:08:47.828127 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:47.828100 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mmdlf/must-gather-jgp45" Apr 16 09:08:47.948964 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:47.948887 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mmdlf/must-gather-jgp45"] Apr 16 09:08:47.951371 ip-10-0-139-84 kubenswrapper[2578]: W0416 09:08:47.951344 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde9d4f65_5da6_487a_89bb_6bf7a1236a89.slice/crio-b7715d575b85aabdfd7904d5e540a945783cf0f849429a1b84092f58ee5b68ac WatchSource:0}: Error finding container b7715d575b85aabdfd7904d5e540a945783cf0f849429a1b84092f58ee5b68ac: Status 404 returned error can't find the container with id b7715d575b85aabdfd7904d5e540a945783cf0f849429a1b84092f58ee5b68ac Apr 16 09:08:47.953581 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:47.953567 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 16 09:08:48.889046 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:48.889011 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mmdlf/must-gather-jgp45" event={"ID":"de9d4f65-5da6-487a-89bb-6bf7a1236a89","Type":"ContainerStarted","Data":"f20017b45d54b2f473f617b9ddcaf9d57690fc19271cd8e9358dbe4725acdd60"} Apr 16 09:08:48.889380 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:48.889060 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mmdlf/must-gather-jgp45" event={"ID":"de9d4f65-5da6-487a-89bb-6bf7a1236a89","Type":"ContainerStarted","Data":"b7715d575b85aabdfd7904d5e540a945783cf0f849429a1b84092f58ee5b68ac"} Apr 16 09:08:49.896069 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:49.896037 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mmdlf/must-gather-jgp45" event={"ID":"de9d4f65-5da6-487a-89bb-6bf7a1236a89","Type":"ContainerStarted","Data":"1488bbc98865e133a60944db9202444f2f324811c43cce51085851e1ad4bf9c3"} Apr 16 09:08:49.910131 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:49.910066 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-mmdlf/must-gather-jgp45" podStartSLOduration=2.10338262 podStartE2EDuration="2.910047219s" podCreationTimestamp="2026-04-16 09:08:47 +0000 UTC" firstStartedPulling="2026-04-16 09:08:47.95369133 +0000 UTC m=+1776.322675366" lastFinishedPulling="2026-04-16 09:08:48.760355924 +0000 UTC m=+1777.129339965" observedRunningTime="2026-04-16 09:08:49.909110318 +0000 UTC m=+1778.278094402" watchObservedRunningTime="2026-04-16 09:08:49.910047219 +0000 UTC m=+1778.279031279" Apr 16 09:08:50.349824 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:50.349795 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zt4sj_4aa4e936-5865-4143-8d38-d2b97d7514b4/global-pull-secret-syncer/0.log" Apr 16 09:08:50.394103 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:50.394066 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6k8v5_8e7b065b-9f25-4b3b-aa65-785e10aec00d/konnectivity-agent/0.log" Apr 16 09:08:50.533563 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:50.533538 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-84.ec2.internal_267abad74bc5a64078f92c3f8060b88d/haproxy/0.log" Apr 16 09:08:53.927376 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:53.927350 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg_7e215abc-0b28-49ee-b7c1-c5c17879ebf1/extract/0.log" Apr 16 09:08:53.954817 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:53.954788 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg_7e215abc-0b28-49ee-b7c1-c5c17879ebf1/util/0.log" Apr 16 09:08:53.985013 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:53.984967 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759q4cfg_7e215abc-0b28-49ee-b7c1-c5c17879ebf1/pull/0.log" Apr 16 09:08:54.012142 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:54.012103 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg_002fee38-9c5e-4e0d-b3b9-15b1e66d3b62/extract/0.log" Apr 16 09:08:54.043157 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:54.043063 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg_002fee38-9c5e-4e0d-b3b9-15b1e66d3b62/util/0.log" Apr 16 09:08:54.067479 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:54.067445 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d69wg_002fee38-9c5e-4e0d-b3b9-15b1e66d3b62/pull/0.log" Apr 16 09:08:54.098234 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:54.098188 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn_2cfad847-f015-4190-8658-10459885802b/extract/0.log" Apr 16 09:08:54.120171 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:54.120147 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn_2cfad847-f015-4190-8658-10459885802b/util/0.log" Apr 16 09:08:54.144593 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:54.144566 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fbktn_2cfad847-f015-4190-8658-10459885802b/pull/0.log" Apr 16 09:08:54.179717 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:54.179604 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv_b1978bda-f153-481b-84bc-5457e58753c2/extract/0.log" Apr 16 09:08:54.209402 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:54.209375 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv_b1978bda-f153-481b-84bc-5457e58753c2/util/0.log" Apr 16 09:08:54.236000 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:54.235974 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qjjcv_b1978bda-f153-481b-84bc-5457e58753c2/pull/0.log" Apr 16 09:08:54.587390 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:54.587362 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-dnfkq_c910e87f-130e-4e62-9adb-7e289d66c95e/manager/0.log" Apr 16 09:08:54.625111 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:54.625086 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-9gs8x_cb3721bd-863e-4a70-8ef3-f341b3306113/kuadrant-console-plugin/0.log" Apr 16 09:08:54.872877 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:54.872796 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-m75dv_3df9c4f5-9aca-43b5-94de-700cbec097b3/limitador/0.log" Apr 16 09:08:56.552189 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:56.552113 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6667474d89-qgz8h_8af06da5-de2c-48a0-8894-9fde08d3085a/cluster-monitoring-operator/0.log" Apr 16 09:08:56.580409 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:56.580350 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7479c89684-sbzrt_04cf4d4f-3397-4603-bb37-6ce19013885c/kube-state-metrics/0.log" Apr 16 09:08:56.603494 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:56.603428 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7479c89684-sbzrt_04cf4d4f-3397-4603-bb37-6ce19013885c/kube-rbac-proxy-main/0.log" Apr 16 09:08:56.632131 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:56.632078 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7479c89684-sbzrt_04cf4d4f-3397-4603-bb37-6ce19013885c/kube-rbac-proxy-self/0.log" Apr 16 09:08:56.868867 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:56.868758 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z79cw_aeba0bf6-c3d1-48b1-a406-a3cff57b6544/node-exporter/0.log" Apr 16 09:08:56.896193 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:56.896128 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z79cw_aeba0bf6-c3d1-48b1-a406-a3cff57b6544/kube-rbac-proxy/0.log" Apr 16 09:08:56.922090 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:56.921987 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z79cw_aeba0bf6-c3d1-48b1-a406-a3cff57b6544/init-textfile/0.log" Apr 16 09:08:56.951721 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:56.951694 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-5669946b84-tgwn7_170f730a-ba17-492c-94d6-cc2f6c0d9177/kube-rbac-proxy-main/0.log" Apr 16 09:08:56.979370 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:56.979340 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-5669946b84-tgwn7_170f730a-ba17-492c-94d6-cc2f6c0d9177/kube-rbac-proxy-self/0.log" Apr 16 09:08:57.002382 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:57.002353 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-5669946b84-tgwn7_170f730a-ba17-492c-94d6-cc2f6c0d9177/openshift-state-metrics/0.log" Apr 16 09:08:57.211634 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:57.211514 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-78f957474d-n59rr_8516f8a5-573f-41fb-bc6c-7e4880f69b09/prometheus-operator/0.log" Apr 16 09:08:57.234120 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:57.234096 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-78f957474d-n59rr_8516f8a5-573f-41fb-bc6c-7e4880f69b09/kube-rbac-proxy/0.log" Apr 16 09:08:57.258890 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:57.258861 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-9cb97cd87-jldxt_5b0fe24d-1c68-4a6a-ae1a-ae2f6da5b957/prometheus-operator-admission-webhook/0.log" Apr 16 09:08:57.290584 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:57.290551 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-749764bc74-49p74_bc059993-986e-4758-8e80-fa0a512bc2d4/telemeter-client/0.log" Apr 16 09:08:57.319862 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:57.319831 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-749764bc74-49p74_bc059993-986e-4758-8e80-fa0a512bc2d4/reload/0.log" Apr 16 09:08:57.341567 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:57.341536 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-749764bc74-49p74_bc059993-986e-4758-8e80-fa0a512bc2d4/kube-rbac-proxy/0.log" Apr 16 09:08:57.370419 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:57.370392 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5658999486-4m4xg_6929b02b-d393-44b8-a0a4-74c8eef0ced4/thanos-query/0.log" Apr 16 09:08:57.409602 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:57.409574 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5658999486-4m4xg_6929b02b-d393-44b8-a0a4-74c8eef0ced4/kube-rbac-proxy-web/0.log" Apr 16 09:08:57.481522 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:57.481489 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5658999486-4m4xg_6929b02b-d393-44b8-a0a4-74c8eef0ced4/kube-rbac-proxy/0.log" Apr 16 09:08:57.513737 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:57.513708 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5658999486-4m4xg_6929b02b-d393-44b8-a0a4-74c8eef0ced4/prom-label-proxy/0.log" Apr 16 09:08:57.536886 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:57.536506 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5658999486-4m4xg_6929b02b-d393-44b8-a0a4-74c8eef0ced4/kube-rbac-proxy-rules/0.log" Apr 16 09:08:57.558711 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:57.558683 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5658999486-4m4xg_6929b02b-d393-44b8-a0a4-74c8eef0ced4/kube-rbac-proxy-metrics/0.log" Apr 16 09:08:58.570460 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:58.570429 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5cb6cf4cb4-s5sxt_66b6340c-4dc1-4115-8563-ad602e61e806/networking-console-plugin/0.log" Apr 16 09:08:58.716395 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:58.716355 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mmdlf/perf-node-gather-daemonset-xnlhm"] Apr 16 09:08:58.721900 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:58.721876 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xnlhm" Apr 16 09:08:58.727213 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:58.727176 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mmdlf/perf-node-gather-daemonset-xnlhm"] Apr 16 09:08:58.878202 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:58.878103 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/129eaafa-51ab-40c4-b94f-d6c1e77650a2-podres\") pod \"perf-node-gather-daemonset-xnlhm\" (UID: \"129eaafa-51ab-40c4-b94f-d6c1e77650a2\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xnlhm" Apr 16 09:08:58.878373 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:58.878211 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/129eaafa-51ab-40c4-b94f-d6c1e77650a2-proc\") pod \"perf-node-gather-daemonset-xnlhm\" (UID: \"129eaafa-51ab-40c4-b94f-d6c1e77650a2\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xnlhm" Apr 16 09:08:58.878373 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:58.878240 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/129eaafa-51ab-40c4-b94f-d6c1e77650a2-lib-modules\") pod \"perf-node-gather-daemonset-xnlhm\" (UID: \"129eaafa-51ab-40c4-b94f-d6c1e77650a2\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xnlhm" Apr 16 09:08:58.878373 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:58.878273 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj8j2\" (UniqueName: \"kubernetes.io/projected/129eaafa-51ab-40c4-b94f-d6c1e77650a2-kube-api-access-nj8j2\") pod \"perf-node-gather-daemonset-xnlhm\" (UID: \"129eaafa-51ab-40c4-b94f-d6c1e77650a2\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xnlhm" Apr 16 09:08:58.878373 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:58.878301 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/129eaafa-51ab-40c4-b94f-d6c1e77650a2-sys\") pod \"perf-node-gather-daemonset-xnlhm\" (UID: \"129eaafa-51ab-40c4-b94f-d6c1e77650a2\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xnlhm" Apr 16 09:08:58.978889 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:58.978848 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/129eaafa-51ab-40c4-b94f-d6c1e77650a2-proc\") pod \"perf-node-gather-daemonset-xnlhm\" (UID: \"129eaafa-51ab-40c4-b94f-d6c1e77650a2\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xnlhm" Apr 16 09:08:58.979058 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:58.978900 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/129eaafa-51ab-40c4-b94f-d6c1e77650a2-lib-modules\") pod \"perf-node-gather-daemonset-xnlhm\" (UID: \"129eaafa-51ab-40c4-b94f-d6c1e77650a2\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xnlhm" Apr 16 09:08:58.979135 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:58.978961 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nj8j2\" (UniqueName: \"kubernetes.io/projected/129eaafa-51ab-40c4-b94f-d6c1e77650a2-kube-api-access-nj8j2\") pod \"perf-node-gather-daemonset-xnlhm\" (UID: \"129eaafa-51ab-40c4-b94f-d6c1e77650a2\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xnlhm" Apr 16 09:08:58.979200 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:58.979153 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/129eaafa-51ab-40c4-b94f-d6c1e77650a2-sys\") pod \"perf-node-gather-daemonset-xnlhm\" (UID: \"129eaafa-51ab-40c4-b94f-d6c1e77650a2\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xnlhm" Apr 16 09:08:58.979200 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:58.978983 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/129eaafa-51ab-40c4-b94f-d6c1e77650a2-proc\") pod \"perf-node-gather-daemonset-xnlhm\" (UID: \"129eaafa-51ab-40c4-b94f-d6c1e77650a2\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xnlhm" Apr 16 09:08:58.979299 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:58.979103 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/129eaafa-51ab-40c4-b94f-d6c1e77650a2-lib-modules\") pod \"perf-node-gather-daemonset-xnlhm\" (UID: \"129eaafa-51ab-40c4-b94f-d6c1e77650a2\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xnlhm" Apr 16 09:08:58.979355 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:58.979313 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/129eaafa-51ab-40c4-b94f-d6c1e77650a2-sys\") pod \"perf-node-gather-daemonset-xnlhm\" (UID: \"129eaafa-51ab-40c4-b94f-d6c1e77650a2\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xnlhm" Apr 16 09:08:58.979462 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:58.979437 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/129eaafa-51ab-40c4-b94f-d6c1e77650a2-podres\") pod \"perf-node-gather-daemonset-xnlhm\" (UID: \"129eaafa-51ab-40c4-b94f-d6c1e77650a2\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xnlhm" Apr 16 09:08:58.979630 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:58.979611 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/129eaafa-51ab-40c4-b94f-d6c1e77650a2-podres\") pod \"perf-node-gather-daemonset-xnlhm\" (UID: \"129eaafa-51ab-40c4-b94f-d6c1e77650a2\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xnlhm" Apr 16 09:08:58.986871 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:58.986847 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj8j2\" (UniqueName: \"kubernetes.io/projected/129eaafa-51ab-40c4-b94f-d6c1e77650a2-kube-api-access-nj8j2\") pod \"perf-node-gather-daemonset-xnlhm\" (UID: \"129eaafa-51ab-40c4-b94f-d6c1e77650a2\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xnlhm" Apr 16 09:08:59.035953 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:59.035918 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xnlhm" Apr 16 09:08:59.400356 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:59.400322 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mmdlf/perf-node-gather-daemonset-xnlhm"] Apr 16 09:08:59.401715 ip-10-0-139-84 kubenswrapper[2578]: W0416 09:08:59.401674 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod129eaafa_51ab_40c4_b94f_d6c1e77650a2.slice/crio-4a3b27ebf101c43a6cc487d7e97dff12e04aa5dbadec9fd0c489d2e2d2cd0452 WatchSource:0}: Error finding container 4a3b27ebf101c43a6cc487d7e97dff12e04aa5dbadec9fd0c489d2e2d2cd0452: Status 404 returned error can't find the container with id 4a3b27ebf101c43a6cc487d7e97dff12e04aa5dbadec9fd0c489d2e2d2cd0452 Apr 16 09:08:59.952622 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:59.952587 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xnlhm" event={"ID":"129eaafa-51ab-40c4-b94f-d6c1e77650a2","Type":"ContainerStarted","Data":"33d4f4c320bd0e04aef674569fa346d7168e0a528f6c869a92a02e79e1aeafe2"} Apr 16 09:08:59.952622 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:59.952624 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xnlhm" event={"ID":"129eaafa-51ab-40c4-b94f-d6c1e77650a2","Type":"ContainerStarted","Data":"4a3b27ebf101c43a6cc487d7e97dff12e04aa5dbadec9fd0c489d2e2d2cd0452"} Apr 16 09:08:59.953184 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:59.952707 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xnlhm" Apr 16 09:08:59.970326 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:08:59.970267 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xnlhm" podStartSLOduration=1.970247044 podStartE2EDuration="1.970247044s" podCreationTimestamp="2026-04-16 09:08:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 09:08:59.968404442 +0000 UTC m=+1788.337388501" watchObservedRunningTime="2026-04-16 09:08:59.970247044 +0000 UTC m=+1788.339231103" Apr 16 09:09:00.118954 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:00.118921 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-7d955d5dd4-bff4c_28d97c1a-528d-4451-8bc1-c2f47b39d660/volume-data-source-validator/0.log" Apr 16 09:09:00.897022 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:00.896996 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rqjhz_b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8/dns/0.log" Apr 16 09:09:00.918383 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:00.918357 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rqjhz_b1cd8102-c016-4e4b-96ee-2f3e5b0ec7b8/kube-rbac-proxy/0.log" Apr 16 09:09:00.988136 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:00.988107 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qxxqp_69ff05e4-5126-420e-ba65-9f88a7fec110/dns-node-resolver/0.log" Apr 16 09:09:01.486829 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:01.486807 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7rgcb_17b0a66e-5c6b-4926-b67d-e916dd6009d3/node-ca/0.log" Apr 16 09:09:02.499984 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:02.499960 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d5746998b-qfmv2_efa00193-20de-451e-a822-40ea9746a0eb/kube-auth-proxy/0.log" Apr 16 09:09:02.603658 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:02.603630 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54fdbc7b78-dxx97_f45d6895-a6e2-4531-ac1e-6e8c46c51012/router/0.log" Apr 16 09:09:03.090759 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:03.090733 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nlp9v_6136520a-182b-44c5-837e-1dd3dc6af3f7/serve-healthcheck-canary/0.log" Apr 16 09:09:03.711386 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:03.711359 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rcsp2_f2fa2681-bd99-4a1a-b082-7c82172d5428/kube-rbac-proxy/0.log" Apr 16 09:09:03.732455 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:03.732428 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rcsp2_f2fa2681-bd99-4a1a-b082-7c82172d5428/exporter/0.log" Apr 16 09:09:03.756388 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:03.756359 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rcsp2_f2fa2681-bd99-4a1a-b082-7c82172d5428/extractor/0.log" Apr 16 09:09:05.751597 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:05.751563 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-569944d57d-rk86c_5b4974c7-139a-43f3-b681-f42333201e5f/manager/0.log" Apr 16 09:09:05.968634 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:05.968608 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xnlhm" Apr 16 09:09:06.941687 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:06.941658 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7cbc7f8cc-dd79v_b7b083bc-b402-45b5-9d6d-a3fbb0cd96dd/manager/0.log" Apr 16 09:09:06.993016 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:06.992983 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-bfc7f696d-xkx66_411dd942-5ef1-47ef-8412-c218f0bcae6b/openshift-lws-operator/0.log" Apr 16 09:09:11.258003 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:11.257968 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-64d4d94569-zr5wc_b86396a6-050a-4fc8-87a7-3185d62b983c/migrator/0.log" Apr 16 09:09:11.279862 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:11.279834 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-64d4d94569-zr5wc_b86396a6-050a-4fc8-87a7-3185d62b983c/graceful-termination/0.log" Apr 16 09:09:12.707357 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:12.707320 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7kxqz_d38f82dc-5ceb-4b6a-b918-e2b0085a9949/kube-multus-additional-cni-plugins/0.log" Apr 16 09:09:12.733645 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:12.733616 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7kxqz_d38f82dc-5ceb-4b6a-b918-e2b0085a9949/egress-router-binary-copy/0.log" Apr 16 09:09:12.757777 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:12.757712 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7kxqz_d38f82dc-5ceb-4b6a-b918-e2b0085a9949/cni-plugins/0.log" Apr 16 09:09:12.778688 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:12.778663 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7kxqz_d38f82dc-5ceb-4b6a-b918-e2b0085a9949/bond-cni-plugin/0.log" Apr 16 09:09:12.799661 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:12.799640 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7kxqz_d38f82dc-5ceb-4b6a-b918-e2b0085a9949/routeoverride-cni/0.log" Apr 16 09:09:12.822142 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:12.822123 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7kxqz_d38f82dc-5ceb-4b6a-b918-e2b0085a9949/whereabouts-cni-bincopy/0.log" Apr 16 09:09:12.847855 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:12.847823 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7kxqz_d38f82dc-5ceb-4b6a-b918-e2b0085a9949/whereabouts-cni/0.log" Apr 16 09:09:13.221559 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:13.221527 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-f9c2h_f2f2b361-d096-4c34-8cc1-5cfe69cdfe03/kube-multus/0.log" Apr 16 09:09:13.304918 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:13.304874 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fxtnv_57f278d8-41a1-4600-a089-04d841c7d446/network-metrics-daemon/0.log" Apr 16 09:09:13.325498 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:13.325474 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fxtnv_57f278d8-41a1-4600-a089-04d841c7d446/kube-rbac-proxy/0.log" Apr 16 09:09:14.246179 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:14.246156 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6jwz6_0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791/ovn-controller/0.log" Apr 16 09:09:14.275017 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:14.274989 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6jwz6_0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791/ovn-acl-logging/0.log" Apr 16 09:09:14.295573 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:14.295538 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6jwz6_0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791/kube-rbac-proxy-node/0.log" Apr 16 09:09:14.318959 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:14.318931 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6jwz6_0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791/kube-rbac-proxy-ovn-metrics/0.log" Apr 16 09:09:14.339223 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:14.339185 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6jwz6_0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791/northd/0.log" Apr 16 09:09:14.362120 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:14.362097 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6jwz6_0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791/nbdb/0.log" Apr 16 09:09:14.391267 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:14.391235 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6jwz6_0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791/sbdb/0.log" Apr 16 09:09:14.506193 ip-10-0-139-84 kubenswrapper[2578]: I0416 09:09:14.506115 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6jwz6_0b0eb1f1-04e2-4edb-9f6b-fa9cee46f791/ovnkube-controller/0.log"