Apr 22 15:05:15.792321 ip-10-0-131-168 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Apr 22 15:05:15.792331 ip-10-0-131-168 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Apr 22 15:05:15.792339 ip-10-0-131-168 systemd[1]: kubelet.service: Failed with result 'resources'. Apr 22 15:05:15.792642 ip-10-0-131-168 systemd[1]: Failed to start Kubernetes Kubelet. Apr 22 15:05:25.865824 ip-10-0-131-168 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Apr 22 15:05:25.865846 ip-10-0-131-168 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 211659cf18b344af8197d51f20bb4bcf -- Apr 22 15:07:40.811771 ip-10-0-131-168 systemd[1]: Starting Kubernetes Kubelet... Apr 22 15:07:41.276847 ip-10-0-131-168 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Apr 22 15:07:41.276847 ip-10-0-131-168 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Apr 22 15:07:41.276847 ip-10-0-131-168 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Apr 22 15:07:41.276847 ip-10-0-131-168 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Apr 22 15:07:41.276847 ip-10-0-131-168 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Apr 22 15:07:41.279532 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.279430 2577 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Apr 22 15:07:41.283858 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283834 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 22 15:07:41.283858 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283855 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 22 15:07:41.283858 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283859 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 22 15:07:41.283858 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283862 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 22 15:07:41.283858 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283865 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 22 15:07:41.284062 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283869 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 22 15:07:41.284062 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283872 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 22 15:07:41.284062 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283875 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 22 15:07:41.284062 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283878 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 22 15:07:41.284062 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283881 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 22 15:07:41.284062 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283883 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 22 15:07:41.284062 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283886 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 22 15:07:41.284062 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283889 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 22 15:07:41.284062 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283892 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 22 15:07:41.284062 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283895 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 22 15:07:41.284062 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283898 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 22 15:07:41.284062 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283901 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 22 15:07:41.284062 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283905 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 22 15:07:41.284062 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283907 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 22 15:07:41.284062 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283910 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 22 15:07:41.284062 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283912 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 22 15:07:41.284062 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283915 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 22 15:07:41.284062 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283918 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 22 15:07:41.284062 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283921 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 22 15:07:41.284062 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283923 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 22 15:07:41.284557 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283933 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 22 15:07:41.284557 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283936 2577 feature_gate.go:328] unrecognized feature gate: Example Apr 22 15:07:41.284557 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283939 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 22 15:07:41.284557 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283942 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 22 15:07:41.284557 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283944 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 22 15:07:41.284557 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283947 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 22 15:07:41.284557 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283949 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 22 15:07:41.284557 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283953 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 22 15:07:41.284557 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283957 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 22 15:07:41.284557 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283960 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 22 15:07:41.284557 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283963 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 22 15:07:41.284557 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283966 2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 22 15:07:41.284557 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283969 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 22 15:07:41.284557 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283974 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 22 15:07:41.284557 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283978 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 22 15:07:41.284557 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283982 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 22 15:07:41.284557 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283985 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 22 15:07:41.284557 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283987 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 22 15:07:41.284557 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283990 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 22 15:07:41.285097 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283993 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 22 15:07:41.285097 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283995 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 22 15:07:41.285097 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.283998 2577 feature_gate.go:328] unrecognized feature gate: Example2 Apr 22 15:07:41.285097 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284000 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 22 15:07:41.285097 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284003 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 22 15:07:41.285097 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284005 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 22 15:07:41.285097 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284008 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 22 15:07:41.285097 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284010 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 22 15:07:41.285097 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284013 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 22 15:07:41.285097 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284016 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 22 15:07:41.285097 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284018 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 22 15:07:41.285097 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284021 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 22 15:07:41.285097 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284023 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 22 15:07:41.285097 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284026 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 22 15:07:41.285097 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284029 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 22 15:07:41.285097 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284031 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 22 15:07:41.285097 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284034 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 22 15:07:41.285097 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284036 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 22 15:07:41.285097 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284039 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 22 15:07:41.285097 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284043 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 22 15:07:41.285592 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284045 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 22 15:07:41.285592 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284048 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 22 15:07:41.285592 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284051 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 22 15:07:41.285592 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284053 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 22 15:07:41.285592 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284056 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 22 15:07:41.285592 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284058 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 22 15:07:41.285592 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284061 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 22 15:07:41.285592 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284063 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 22 15:07:41.285592 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284066 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 22 15:07:41.285592 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284069 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 22 15:07:41.285592 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284072 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 22 15:07:41.285592 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284074 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 22 15:07:41.285592 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284077 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 22 15:07:41.285592 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284080 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 22 15:07:41.285592 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284083 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 22 15:07:41.285592 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284086 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 22 15:07:41.285592 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284089 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 22 15:07:41.285592 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284091 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 22 15:07:41.285592 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284094 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 22 15:07:41.286045 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284097 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 22 15:07:41.286045 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284099 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 22 15:07:41.286045 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284102 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 22 15:07:41.286045 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284555 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 22 15:07:41.286045 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284561 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 22 15:07:41.286045 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284565 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 22 15:07:41.286045 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284568 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 22 15:07:41.286045 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284571 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 22 15:07:41.286045 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284574 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 22 15:07:41.286045 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284577 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 22 15:07:41.286045 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284580 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 22 15:07:41.286045 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284583 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 22 15:07:41.286045 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284586 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 22 15:07:41.286045 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284590 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 22 15:07:41.286045 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284593 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 22 15:07:41.286045 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284595 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 22 15:07:41.286045 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284598 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 22 15:07:41.286045 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284601 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 22 15:07:41.286045 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284604 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 22 15:07:41.286045 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284607 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 22 15:07:41.286549 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284610 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 22 15:07:41.286549 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284613 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 22 15:07:41.286549 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284616 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 22 15:07:41.286549 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284619 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 22 15:07:41.286549 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284621 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 22 15:07:41.286549 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284624 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 22 15:07:41.286549 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284627 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 22 15:07:41.286549 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284630 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 22 15:07:41.286549 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284633 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 22 15:07:41.286549 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284636 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 22 15:07:41.286549 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284638 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 22 15:07:41.286549 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284641 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 22 15:07:41.286549 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284643 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 22 15:07:41.286549 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284646 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 22 15:07:41.286549 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284650 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 22 15:07:41.286549 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284654 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 22 15:07:41.286549 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284656 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 22 15:07:41.286549 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284660 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 22 15:07:41.286549 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284663 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 22 15:07:41.286549 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284665 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 22 15:07:41.287035 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284668 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 22 15:07:41.287035 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284670 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 22 15:07:41.287035 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284673 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 22 15:07:41.287035 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284676 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 22 15:07:41.287035 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284679 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 22 15:07:41.287035 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284681 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 22 15:07:41.287035 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284684 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 22 15:07:41.287035 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284687 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 22 15:07:41.287035 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284689 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 22 15:07:41.287035 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284692 2577 feature_gate.go:328] unrecognized feature gate: Example2 Apr 22 15:07:41.287035 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284694 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 22 15:07:41.287035 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284697 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 22 15:07:41.287035 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284699 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 22 15:07:41.287035 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284702 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 22 15:07:41.287035 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284705 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 22 15:07:41.287035 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284707 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 22 15:07:41.287035 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284710 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 22 15:07:41.287035 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284712 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 22 15:07:41.287035 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284714 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 22 15:07:41.287507 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284717 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 22 15:07:41.287507 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284720 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 22 15:07:41.287507 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284723 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 22 15:07:41.287507 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284725 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 22 15:07:41.287507 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284728 2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 22 15:07:41.287507 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284730 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 22 15:07:41.287507 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284733 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 22 15:07:41.287507 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284735 2577 feature_gate.go:328] unrecognized feature gate: Example Apr 22 15:07:41.287507 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284738 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 22 15:07:41.287507 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284741 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 22 15:07:41.287507 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284744 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 22 15:07:41.287507 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284747 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 22 15:07:41.287507 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284749 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 22 15:07:41.287507 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284752 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 22 15:07:41.287507 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284754 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 22 15:07:41.287507 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284757 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 22 15:07:41.287507 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284760 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 22 15:07:41.287507 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284763 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 22 15:07:41.287507 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284766 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 22 15:07:41.287507 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284768 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 22 15:07:41.287999 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284772 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 22 15:07:41.287999 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284775 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 22 15:07:41.287999 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284778 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 22 15:07:41.287999 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284781 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 22 15:07:41.287999 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284783 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 22 15:07:41.287999 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284786 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 22 15:07:41.287999 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284789 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 22 15:07:41.287999 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284791 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 22 15:07:41.287999 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284793 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 22 15:07:41.287999 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.284796 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 22 15:07:41.287999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284866 2577 flags.go:64] FLAG: --address="0.0.0.0" Apr 22 15:07:41.287999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284873 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Apr 22 15:07:41.287999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284880 2577 flags.go:64] FLAG: --anonymous-auth="true" Apr 22 15:07:41.287999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284885 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" Apr 22 15:07:41.287999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284890 2577 flags.go:64] FLAG: --authentication-token-webhook="false" Apr 22 15:07:41.287999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284893 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Apr 22 15:07:41.287999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284898 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Apr 22 15:07:41.287999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284902 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Apr 22 15:07:41.287999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284906 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Apr 22 15:07:41.287999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284909 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Apr 22 15:07:41.287999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284917 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Apr 22 15:07:41.288525 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284920 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Apr 22 15:07:41.288525 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284924 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Apr 22 15:07:41.288525 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284927 2577 flags.go:64] FLAG: --cgroup-root="" Apr 22 15:07:41.288525 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284930 2577 flags.go:64] FLAG: --cgroups-per-qos="true" Apr 22 15:07:41.288525 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284933 2577 flags.go:64] FLAG: --client-ca-file="" Apr 22 15:07:41.288525 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284936 2577 flags.go:64] FLAG: --cloud-config="" Apr 22 15:07:41.288525 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284939 2577 flags.go:64] FLAG: --cloud-provider="external" Apr 22 15:07:41.288525 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284942 2577 flags.go:64] FLAG: --cluster-dns="[]" Apr 22 15:07:41.288525 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284947 2577 flags.go:64] FLAG: --cluster-domain="" Apr 22 15:07:41.288525 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284950 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Apr 22 15:07:41.288525 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284954 2577 flags.go:64] FLAG: --config-dir="" Apr 22 15:07:41.288525 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284957 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Apr 22 15:07:41.288525 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284960 2577 flags.go:64] FLAG: --container-log-max-files="5" Apr 22 15:07:41.288525 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284964 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" Apr 22 15:07:41.288525 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284967 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Apr 22 15:07:41.288525 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284970 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Apr 22 15:07:41.288525 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284973 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" Apr 22 15:07:41.288525 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284977 2577 flags.go:64] FLAG: --contention-profiling="false" Apr 22 15:07:41.288525 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284980 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" Apr 22 15:07:41.288525 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284983 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Apr 22 15:07:41.288525 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284986 2577 flags.go:64] FLAG: --cpu-manager-policy="none" Apr 22 15:07:41.288525 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284989 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" Apr 22 15:07:41.288525 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284993 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Apr 22 15:07:41.288525 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284996 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" Apr 22 15:07:41.288525 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.284999 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" Apr 22 15:07:41.289129 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285002 2577 flags.go:64] FLAG: --enable-load-reader="false" Apr 22 15:07:41.289129 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285006 2577 flags.go:64] FLAG: --enable-server="true" Apr 22 15:07:41.289129 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285009 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Apr 22 15:07:41.289129 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285013 2577 flags.go:64] FLAG: --event-burst="100" Apr 22 15:07:41.289129 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285017 2577 flags.go:64] FLAG: --event-qps="50" Apr 22 15:07:41.289129 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285020 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" Apr 22 15:07:41.289129 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285024 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" Apr 22 15:07:41.289129 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285027 2577 flags.go:64] FLAG: --eviction-hard="" Apr 22 15:07:41.289129 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285031 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Apr 22 15:07:41.289129 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285034 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" Apr 22 15:07:41.289129 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285037 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Apr 22 15:07:41.289129 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285040 2577 flags.go:64] FLAG: --eviction-soft="" Apr 22 15:07:41.289129 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285043 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" Apr 22 15:07:41.289129 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285046 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" Apr 22 15:07:41.289129 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285056 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Apr 22 15:07:41.289129 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285061 2577 flags.go:64] FLAG: --experimental-mounter-path="" Apr 22 15:07:41.289129 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285065 2577 flags.go:64] FLAG: --fail-cgroupv1="false" Apr 22 15:07:41.289129 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285069 2577 flags.go:64] FLAG: --fail-swap-on="true" Apr 22 15:07:41.289129 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285072 2577 flags.go:64] FLAG: --feature-gates="" Apr 22 15:07:41.289129 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285076 2577 flags.go:64] FLAG: --file-check-frequency="20s" Apr 22 15:07:41.289129 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285079 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Apr 22 15:07:41.289129 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285083 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Apr 22 15:07:41.289129 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285086 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Apr 22 15:07:41.289129 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285089 2577 flags.go:64] FLAG: --healthz-port="10248" Apr 22 15:07:41.289129 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285093 2577 flags.go:64] FLAG: --help="false" Apr 22 15:07:41.289775 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285096 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-131-168.ec2.internal" Apr 22 15:07:41.289775 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285099 2577 flags.go:64] FLAG: --housekeeping-interval="10s" Apr 22 15:07:41.289775 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285102 2577 flags.go:64] FLAG: --http-check-frequency="20s" Apr 22 15:07:41.289775 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285105 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Apr 22 15:07:41.289775 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285109 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Apr 22 15:07:41.289775 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285112 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" Apr 22 15:07:41.289775 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285115 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" Apr 22 15:07:41.289775 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285118 2577 flags.go:64] FLAG: --image-service-endpoint="" Apr 22 15:07:41.289775 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285121 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" Apr 22 15:07:41.289775 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285124 2577 flags.go:64] FLAG: --kube-api-burst="100" Apr 22 15:07:41.289775 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285127 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Apr 22 15:07:41.289775 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285131 2577 flags.go:64] FLAG: --kube-api-qps="50" Apr 22 15:07:41.289775 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285135 2577 flags.go:64] FLAG: --kube-reserved="" Apr 22 15:07:41.289775 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285138 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" Apr 22 15:07:41.289775 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285141 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Apr 22 15:07:41.289775 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285144 2577 flags.go:64] FLAG: --kubelet-cgroups="" Apr 22 15:07:41.289775 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285147 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Apr 22 15:07:41.289775 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285150 2577 flags.go:64] FLAG: --lock-file="" Apr 22 15:07:41.289775 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285153 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" Apr 22 15:07:41.289775 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285156 2577 flags.go:64] FLAG: --log-flush-frequency="5s" Apr 22 15:07:41.289775 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285159 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" Apr 22 15:07:41.289775 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285164 2577 flags.go:64] FLAG: --log-json-split-stream="false" Apr 22 15:07:41.289775 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285167 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" Apr 22 15:07:41.290320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285170 2577 flags.go:64] FLAG: --log-text-split-stream="false" Apr 22 15:07:41.290320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285173 2577 flags.go:64] FLAG: --logging-format="text" Apr 22 15:07:41.290320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285176 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Apr 22 15:07:41.290320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285180 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" Apr 22 15:07:41.290320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285183 2577 flags.go:64] FLAG: --manifest-url="" Apr 22 15:07:41.290320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285185 2577 flags.go:64] FLAG: --manifest-url-header="" Apr 22 15:07:41.290320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285190 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" Apr 22 15:07:41.290320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285193 2577 flags.go:64] FLAG: --max-open-files="1000000" Apr 22 15:07:41.290320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285197 2577 flags.go:64] FLAG: --max-pods="110" Apr 22 15:07:41.290320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285200 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" Apr 22 15:07:41.290320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285202 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Apr 22 15:07:41.290320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285205 2577 flags.go:64] FLAG: --memory-manager-policy="None" Apr 22 15:07:41.290320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285208 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Apr 22 15:07:41.290320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285212 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Apr 22 15:07:41.290320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285215 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" Apr 22 15:07:41.290320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285218 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Apr 22 15:07:41.290320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285226 2577 flags.go:64] FLAG: --node-status-max-images="50" Apr 22 15:07:41.290320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285229 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" Apr 22 15:07:41.290320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285232 2577 flags.go:64] FLAG: --oom-score-adj="-999" Apr 22 15:07:41.290320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285238 2577 flags.go:64] FLAG: --pod-cidr="" Apr 22 15:07:41.290320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285241 2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c8cfe89231412ff3ee8cb6207fa0be33cad0f08e88c9c0f1e9f7e8c6f14d6715" Apr 22 15:07:41.290320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285246 2577 flags.go:64] FLAG: --pod-manifest-path="" Apr 22 15:07:41.290320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285249 2577 flags.go:64] FLAG: --pod-max-pids="-1" Apr 22 15:07:41.290320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285252 2577 flags.go:64] FLAG: --pods-per-core="0" Apr 22 15:07:41.290919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285256 2577 flags.go:64] FLAG: --port="10250" Apr 22 15:07:41.290919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285259 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" Apr 22 15:07:41.290919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285262 2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07066cb087cd16b35" Apr 22 15:07:41.290919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285265 2577 flags.go:64] FLAG: --qos-reserved="" Apr 22 15:07:41.290919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285268 2577 flags.go:64] FLAG: --read-only-port="10255" Apr 22 15:07:41.290919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285271 2577 flags.go:64] FLAG: --register-node="true" Apr 22 15:07:41.290919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285274 2577 flags.go:64] FLAG: --register-schedulable="true" Apr 22 15:07:41.290919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285277 2577 flags.go:64] FLAG: --register-with-taints="" Apr 22 15:07:41.290919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285281 2577 flags.go:64] FLAG: --registry-burst="10" Apr 22 15:07:41.290919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285284 2577 flags.go:64] FLAG: --registry-qps="5" Apr 22 15:07:41.290919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285287 2577 flags.go:64] FLAG: --reserved-cpus="" Apr 22 15:07:41.290919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285290 2577 flags.go:64] FLAG: --reserved-memory="" Apr 22 15:07:41.290919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285293 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Apr 22 15:07:41.290919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285297 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Apr 22 15:07:41.290919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285300 2577 flags.go:64] FLAG: --rotate-certificates="false" Apr 22 15:07:41.290919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285303 2577 flags.go:64] FLAG: --rotate-server-certificates="false" Apr 22 15:07:41.290919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285306 2577 flags.go:64] FLAG: --runonce="false" Apr 22 15:07:41.290919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285309 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Apr 22 15:07:41.290919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285312 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Apr 22 15:07:41.290919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285315 2577 flags.go:64] FLAG: --seccomp-default="false" Apr 22 15:07:41.290919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285318 2577 flags.go:64] FLAG: --serialize-image-pulls="true" Apr 22 15:07:41.290919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285320 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Apr 22 15:07:41.290919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285323 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" Apr 22 15:07:41.290919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285339 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Apr 22 15:07:41.290919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285342 2577 flags.go:64] FLAG: --storage-driver-password="root" Apr 22 15:07:41.290919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285345 2577 flags.go:64] FLAG: --storage-driver-secure="false" Apr 22 15:07:41.291585 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285348 2577 flags.go:64] FLAG: --storage-driver-table="stats" Apr 22 15:07:41.291585 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285351 2577 flags.go:64] FLAG: --storage-driver-user="root" Apr 22 15:07:41.291585 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285356 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Apr 22 15:07:41.291585 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285359 2577 flags.go:64] FLAG: --sync-frequency="1m0s" Apr 22 15:07:41.291585 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285363 2577 flags.go:64] FLAG: --system-cgroups="" Apr 22 15:07:41.291585 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285365 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Apr 22 15:07:41.291585 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285371 2577 flags.go:64] FLAG: --system-reserved-cgroup="" Apr 22 15:07:41.291585 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285374 2577 flags.go:64] FLAG: --tls-cert-file="" Apr 22 15:07:41.291585 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285377 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" Apr 22 15:07:41.291585 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285382 2577 flags.go:64] FLAG: --tls-min-version="" Apr 22 15:07:41.291585 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285385 2577 flags.go:64] FLAG: --tls-private-key-file="" Apr 22 15:07:41.291585 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285388 2577 flags.go:64] FLAG: --topology-manager-policy="none" Apr 22 15:07:41.291585 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285391 2577 flags.go:64] FLAG: --topology-manager-policy-options="" Apr 22 15:07:41.291585 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285394 2577 flags.go:64] FLAG: --topology-manager-scope="container" Apr 22 15:07:41.291585 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285397 2577 flags.go:64] FLAG: --v="2" Apr 22 15:07:41.291585 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285401 2577 flags.go:64] FLAG: --version="false" Apr 22 15:07:41.291585 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285405 2577 flags.go:64] FLAG: --vmodule="" Apr 22 15:07:41.291585 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285410 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Apr 22 15:07:41.291585 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.285413 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Apr 22 15:07:41.291585 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285504 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 22 15:07:41.291585 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285507 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 22 15:07:41.291585 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285510 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 22 15:07:41.291585 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285513 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 22 15:07:41.291585 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285516 2577 feature_gate.go:328] unrecognized feature gate: Example Apr 22 15:07:41.292143 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285519 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 22 15:07:41.292143 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285522 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 22 15:07:41.292143 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285524 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 22 15:07:41.292143 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285527 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 22 15:07:41.292143 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285530 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 22 15:07:41.292143 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285532 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 22 15:07:41.292143 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285535 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 22 15:07:41.292143 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285538 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 22 15:07:41.292143 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285540 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 22 15:07:41.292143 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285559 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 22 15:07:41.292143 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285565 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 22 15:07:41.292143 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285568 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 22 15:07:41.292143 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285572 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 22 15:07:41.292143 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285575 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 22 15:07:41.292143 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285578 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 22 15:07:41.292143 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285581 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 22 15:07:41.292143 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285584 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 22 15:07:41.292143 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285587 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 22 15:07:41.292143 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285590 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 22 15:07:41.292641 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285593 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 22 15:07:41.292641 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285598 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 22 15:07:41.292641 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285601 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 22 15:07:41.292641 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285604 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 22 15:07:41.292641 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285606 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 22 15:07:41.292641 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285609 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 22 15:07:41.292641 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285611 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 22 15:07:41.292641 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285614 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 22 15:07:41.292641 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285616 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 22 15:07:41.292641 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285619 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 22 15:07:41.292641 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285622 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 22 15:07:41.292641 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285625 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 22 15:07:41.292641 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285627 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 22 15:07:41.292641 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285630 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 22 15:07:41.292641 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285632 2577 feature_gate.go:328] unrecognized feature gate: Example2 Apr 22 15:07:41.292641 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285634 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 22 15:07:41.292641 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285637 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 22 15:07:41.292641 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285639 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 22 15:07:41.292641 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285643 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 22 15:07:41.293129 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285645 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 22 15:07:41.293129 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285648 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 22 15:07:41.293129 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285650 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 22 15:07:41.293129 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285653 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 22 15:07:41.293129 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285657 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 22 15:07:41.293129 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285660 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 22 15:07:41.293129 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285663 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 22 15:07:41.293129 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285678 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 22 15:07:41.293129 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285682 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 22 15:07:41.293129 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285685 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 22 15:07:41.293129 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285688 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 22 15:07:41.293129 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285691 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 22 15:07:41.293129 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285694 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 22 15:07:41.293129 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285697 2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 22 15:07:41.293129 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285700 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 22 15:07:41.293129 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285703 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 22 15:07:41.293129 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285705 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 22 15:07:41.293129 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285708 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 22 15:07:41.293129 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285711 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 22 15:07:41.293129 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285713 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 22 15:07:41.293687 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285716 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 22 15:07:41.293687 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285719 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 22 15:07:41.293687 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285721 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 22 15:07:41.293687 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285724 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 22 15:07:41.293687 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285727 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 22 15:07:41.293687 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285730 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 22 15:07:41.293687 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285732 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 22 15:07:41.293687 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285735 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 22 15:07:41.293687 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285737 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 22 15:07:41.293687 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285740 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 22 15:07:41.293687 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285744 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 22 15:07:41.293687 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285749 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 22 15:07:41.293687 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285752 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 22 15:07:41.293687 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285755 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 22 15:07:41.293687 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285757 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 22 15:07:41.293687 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285760 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 22 15:07:41.293687 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285764 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 22 15:07:41.293687 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285766 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 22 15:07:41.293687 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285769 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 22 15:07:41.294159 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285772 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 22 15:07:41.294159 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285775 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 22 15:07:41.294159 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285777 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 22 15:07:41.294159 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.285780 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 22 15:07:41.294159 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.286537 2577 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Apr 22 15:07:41.294292 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.294048 2577 server.go:530] "Kubelet version" kubeletVersion="v1.33.9" Apr 22 15:07:41.294292 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.294173 2577 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Apr 22 15:07:41.294292 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294242 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 22 15:07:41.294292 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294249 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 22 15:07:41.294292 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294252 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 22 15:07:41.294292 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294256 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 22 15:07:41.294292 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294259 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 22 15:07:41.294292 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294262 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 22 15:07:41.294292 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294265 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 22 15:07:41.294292 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294267 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 22 15:07:41.294292 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294271 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 22 15:07:41.294292 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294273 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 22 15:07:41.294292 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294276 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 22 15:07:41.294292 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294279 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 22 15:07:41.294292 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294282 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 22 15:07:41.294292 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294284 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 22 15:07:41.294292 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294287 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 22 15:07:41.294292 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294289 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 22 15:07:41.294292 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294292 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 22 15:07:41.294292 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294295 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 22 15:07:41.294801 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294297 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 22 15:07:41.294801 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294301 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 22 15:07:41.294801 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294305 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 22 15:07:41.294801 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294309 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 22 15:07:41.294801 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294312 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 22 15:07:41.294801 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294315 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 22 15:07:41.294801 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294318 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 22 15:07:41.294801 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294321 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 22 15:07:41.294801 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294324 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 22 15:07:41.294801 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294342 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 22 15:07:41.294801 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294344 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 22 15:07:41.294801 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294347 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 22 15:07:41.294801 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294351 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 22 15:07:41.294801 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294354 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 22 15:07:41.294801 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294356 2577 feature_gate.go:328] unrecognized feature gate: Example2 Apr 22 15:07:41.294801 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294359 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 22 15:07:41.294801 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294362 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 22 15:07:41.294801 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294365 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 22 15:07:41.294801 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294368 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 22 15:07:41.295273 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294371 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 22 15:07:41.295273 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294373 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 22 15:07:41.295273 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294376 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 22 15:07:41.295273 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294378 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 22 15:07:41.295273 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294381 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 22 15:07:41.295273 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294383 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 22 15:07:41.295273 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294386 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 22 15:07:41.295273 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294388 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 22 15:07:41.295273 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294391 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 22 15:07:41.295273 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294393 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 22 15:07:41.295273 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294396 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 22 15:07:41.295273 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294398 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 22 15:07:41.295273 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294400 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 22 15:07:41.295273 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294403 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 22 15:07:41.295273 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294408 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 22 15:07:41.295273 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294412 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 22 15:07:41.295273 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294415 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 22 15:07:41.295273 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294418 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 22 15:07:41.295273 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294420 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 22 15:07:41.295273 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294423 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 22 15:07:41.295864 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294426 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 22 15:07:41.295864 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294429 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 22 15:07:41.295864 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294432 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 22 15:07:41.295864 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294434 2577 feature_gate.go:328] unrecognized feature gate: Example Apr 22 15:07:41.295864 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294437 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 22 15:07:41.295864 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294439 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 22 15:07:41.295864 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294442 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 22 15:07:41.295864 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294444 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 22 15:07:41.295864 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294447 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 22 15:07:41.295864 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294450 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 22 15:07:41.295864 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294452 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 22 15:07:41.295864 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294455 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 22 15:07:41.295864 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294458 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 22 15:07:41.295864 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294460 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 22 15:07:41.295864 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294462 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 22 15:07:41.295864 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294465 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 22 15:07:41.295864 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294467 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 22 15:07:41.295864 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294470 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 22 15:07:41.295864 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294472 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 22 15:07:41.295864 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294475 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 22 15:07:41.296370 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294477 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 22 15:07:41.296370 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294479 2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 22 15:07:41.296370 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294482 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 22 15:07:41.296370 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294484 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 22 15:07:41.296370 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294487 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 22 15:07:41.296370 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294490 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 22 15:07:41.296370 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294493 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 22 15:07:41.296370 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294496 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 22 15:07:41.296370 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294498 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 22 15:07:41.296370 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.294504 2577 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Apr 22 15:07:41.296370 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294604 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 22 15:07:41.296370 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294609 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 22 15:07:41.296370 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294612 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 22 15:07:41.296370 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294615 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 22 15:07:41.296370 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294619 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 22 15:07:41.296370 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294621 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 22 15:07:41.296770 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294624 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 22 15:07:41.296770 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294626 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 22 15:07:41.296770 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294630 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 22 15:07:41.296770 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294634 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 22 15:07:41.296770 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294637 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 22 15:07:41.296770 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294640 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 22 15:07:41.296770 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294642 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 22 15:07:41.296770 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294645 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 22 15:07:41.296770 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294648 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 22 15:07:41.296770 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294650 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 22 15:07:41.296770 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294652 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 22 15:07:41.296770 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294655 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 22 15:07:41.296770 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294658 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 22 15:07:41.296770 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294660 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 22 15:07:41.296770 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294663 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 22 15:07:41.296770 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294665 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 22 15:07:41.296770 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294668 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 22 15:07:41.296770 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294670 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 22 15:07:41.296770 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294673 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 22 15:07:41.297242 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294675 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 22 15:07:41.297242 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294678 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 22 15:07:41.297242 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294680 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 22 15:07:41.297242 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294683 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 22 15:07:41.297242 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294686 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 22 15:07:41.297242 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294688 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 22 15:07:41.297242 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294691 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 22 15:07:41.297242 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294693 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 22 15:07:41.297242 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294696 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 22 15:07:41.297242 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294698 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 22 15:07:41.297242 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294701 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 22 15:07:41.297242 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294704 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 22 15:07:41.297242 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294706 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 22 15:07:41.297242 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294709 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 22 15:07:41.297242 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294712 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 22 15:07:41.297242 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294714 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 22 15:07:41.297242 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294717 2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 22 15:07:41.297242 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294719 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 22 15:07:41.297242 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294722 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 22 15:07:41.297734 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294724 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 22 15:07:41.297734 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294727 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 22 15:07:41.297734 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294729 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 22 15:07:41.297734 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294731 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 22 15:07:41.297734 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294734 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 22 15:07:41.297734 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294736 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 22 15:07:41.297734 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294739 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 22 15:07:41.297734 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294741 2577 feature_gate.go:328] unrecognized feature gate: Example Apr 22 15:07:41.297734 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294744 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 22 15:07:41.297734 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294746 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 22 15:07:41.297734 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294749 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 22 15:07:41.297734 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294751 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 22 15:07:41.297734 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294753 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 22 15:07:41.297734 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294757 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 22 15:07:41.297734 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294760 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 22 15:07:41.297734 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294763 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 22 15:07:41.297734 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294766 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 22 15:07:41.297734 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294769 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 22 15:07:41.297734 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294771 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 22 15:07:41.297734 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294774 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 22 15:07:41.298216 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294776 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 22 15:07:41.298216 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294779 2577 feature_gate.go:328] unrecognized feature gate: Example2 Apr 22 15:07:41.298216 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294781 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 22 15:07:41.298216 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294783 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 22 15:07:41.298216 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294786 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 22 15:07:41.298216 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294788 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 22 15:07:41.298216 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294791 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 22 15:07:41.298216 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294794 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 22 15:07:41.298216 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294796 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 22 15:07:41.298216 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294798 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 22 15:07:41.298216 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294801 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 22 15:07:41.298216 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294803 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 22 15:07:41.298216 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294806 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 22 15:07:41.298216 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294809 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 22 15:07:41.298216 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294811 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 22 15:07:41.298216 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294814 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 22 15:07:41.298216 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294817 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 22 15:07:41.298216 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294819 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 22 15:07:41.298216 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294822 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 22 15:07:41.298216 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294825 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 22 15:07:41.298728 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294827 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 22 15:07:41.298728 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:41.294830 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 22 15:07:41.298728 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.294835 2577 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Apr 22 15:07:41.298728 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.295580 2577 server.go:962] "Client rotation is on, will bootstrap in background" Apr 22 15:07:41.300602 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.300585 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Apr 22 15:07:41.301556 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.301544 2577 server.go:1019] "Starting client certificate rotation" Apr 22 15:07:41.301663 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.301644 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Apr 22 15:07:41.301706 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.301694 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Apr 22 15:07:41.329173 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.329143 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Apr 22 15:07:41.334225 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.334196 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Apr 22 15:07:41.346877 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.346854 2577 log.go:25] "Validated CRI v1 runtime API" Apr 22 15:07:41.352462 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.352445 2577 log.go:25] "Validated CRI v1 image API" Apr 22 15:07:41.353851 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.353833 2577 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Apr 22 15:07:41.356639 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.356617 2577 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Apr 22 15:07:41.357365 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.357321 2577 fs.go:135] Filesystem UUIDs: map[679988e6-8026-4a0c-9ad8-250f8544aa81:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 913ec933-136a-483b-bb99-417840e41d88:/dev/nvme0n1p3] Apr 22 15:07:41.357412 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.357367 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Apr 22 15:07:41.362317 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.362188 2577 manager.go:217] Machine: {Timestamp:2026-04-22 15:07:41.36107978 +0000 UTC m=+0.417817855 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099264 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e58ca596ef59f6f7967f2b410a500 SystemUUID:ec2e58ca-596e-f59f-6f79-67f2b410a500 BootID:211659cf-18b3-44af-8197-d51f20bb4bcf Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:47:9e:f8:c1:6d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:47:9e:f8:c1:6d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0a:9d:e2:86:2d:87 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Apr 22 15:07:41.362317 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.362311 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Apr 22 15:07:41.362475 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.362463 2577 manager.go:233] Version: {KernelVersion:5.14.0-570.107.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260414-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Apr 22 15:07:41.364218 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.364191 2577 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Apr 22 15:07:41.364384 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.364220 2577 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-168.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Apr 22 15:07:41.364428 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.364396 2577 topology_manager.go:138] "Creating topology manager with none policy" Apr 22 15:07:41.364428 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.364406 2577 container_manager_linux.go:306] "Creating device plugin manager" Apr 22 15:07:41.364428 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.364418 2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Apr 22 15:07:41.364513 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.364429 2577 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Apr 22 15:07:41.365378 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.365368 2577 state_mem.go:36] "Initialized new in-memory state store" Apr 22 15:07:41.365486 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.365478 2577 server.go:1267] "Using root directory" path="/var/lib/kubelet" Apr 22 15:07:41.368062 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.368052 2577 kubelet.go:491] "Attempting to sync node with API server" Apr 22 15:07:41.368098 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.368066 2577 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Apr 22 15:07:41.368098 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.368080 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Apr 22 15:07:41.368098 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.368090 2577 kubelet.go:397] "Adding apiserver pod source" Apr 22 15:07:41.368188 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.368099 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Apr 22 15:07:41.369253 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.369240 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Apr 22 15:07:41.369297 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.369259 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Apr 22 15:07:41.373503 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.373478 2577 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.10-2.rhaos4.20.gita4d0894.el9" apiVersion="v1" Apr 22 15:07:41.375286 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.375272 2577 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Apr 22 15:07:41.377239 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.377227 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Apr 22 15:07:41.377282 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.377246 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Apr 22 15:07:41.377282 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.377253 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Apr 22 15:07:41.377282 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.377258 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Apr 22 15:07:41.377282 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.377265 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Apr 22 15:07:41.377282 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.377274 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Apr 22 15:07:41.377282 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.377282 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Apr 22 15:07:41.377463 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.377288 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Apr 22 15:07:41.377463 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.377296 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Apr 22 15:07:41.377463 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.377302 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Apr 22 15:07:41.377463 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.377311 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Apr 22 15:07:41.377463 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.377320 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Apr 22 15:07:41.379450 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.379436 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Apr 22 15:07:41.379450 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.379451 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Apr 22 15:07:41.381997 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.381977 2577 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-168.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Apr 22 15:07:41.382094 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:41.381989 2577 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Apr 22 15:07:41.382094 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:41.382067 2577 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-168.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Apr 22 15:07:41.383174 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.383161 2577 watchdog_linux.go:99] "Systemd watchdog is not enabled" Apr 22 15:07:41.383233 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.383224 2577 server.go:1295] "Started kubelet" Apr 22 15:07:41.383361 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.383309 2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Apr 22 15:07:41.383445 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.383315 2577 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Apr 22 15:07:41.383445 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.383420 2577 server_v1.go:47] "podresources" method="list" useActivePods=true Apr 22 15:07:41.384050 ip-10-0-131-168 systemd[1]: Started Kubernetes Kubelet. Apr 22 15:07:41.384880 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.384809 2577 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Apr 22 15:07:41.386429 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.386413 2577 server.go:317] "Adding debug handlers to kubelet server" Apr 22 15:07:41.389951 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:41.388956 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-168.ec2.internal.18a8b64536a149a6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-168.ec2.internal,UID:ip-10-0-131-168.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-168.ec2.internal,},FirstTimestamp:2026-04-22 15:07:41.383199142 +0000 UTC m=+0.439937218,LastTimestamp:2026-04-22 15:07:41.383199142 +0000 UTC m=+0.439937218,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-168.ec2.internal,}" Apr 22 15:07:41.390975 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.390957 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Apr 22 15:07:41.391491 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.391472 2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Apr 22 15:07:41.392400 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.392377 2577 volume_manager.go:295] "The desired_state_of_world populator starts" Apr 22 15:07:41.392400 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.392402 2577 volume_manager.go:297] "Starting Kubelet Volume Manager" Apr 22 15:07:41.392532 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.392447 2577 factory.go:55] Registering systemd factory Apr 22 15:07:41.392532 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.392464 2577 factory.go:223] Registration of the systemd container factory successfully Apr 22 15:07:41.392532 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.392375 2577 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Apr 22 15:07:41.392662 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.392569 2577 reconstruct.go:97] "Volume reconstruction finished" Apr 22 15:07:41.392662 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.392579 2577 reconciler.go:26] "Reconciler: start to sync state" Apr 22 15:07:41.392744 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:41.392660 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-168.ec2.internal\" not found" Apr 22 15:07:41.392935 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.392920 2577 factory.go:153] Registering CRI-O factory Apr 22 15:07:41.392995 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.392939 2577 factory.go:223] Registration of the crio container factory successfully Apr 22 15:07:41.392995 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.392981 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Apr 22 15:07:41.393103 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.393001 2577 factory.go:103] Registering Raw factory Apr 22 15:07:41.393103 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.393018 2577 manager.go:1196] Started watching for new ooms in manager Apr 22 15:07:41.393448 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.393432 2577 manager.go:319] Starting recovery of all containers Apr 22 15:07:41.394736 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:41.394633 2577 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Apr 22 15:07:41.402910 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:41.402865 2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-168.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Apr 22 15:07:41.403043 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:41.403019 2577 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Apr 22 15:07:41.404279 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.404255 2577 manager.go:324] Recovery completed Apr 22 15:07:41.406381 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:41.406360 2577 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Apr 22 15:07:41.409292 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.409278 2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 22 15:07:41.411746 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.411729 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-44psn" Apr 22 15:07:41.411930 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.411917 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-168.ec2.internal" event="NodeHasSufficientMemory" Apr 22 15:07:41.411978 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.411946 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-168.ec2.internal" event="NodeHasNoDiskPressure" Apr 22 15:07:41.411978 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.411960 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-168.ec2.internal" event="NodeHasSufficientPID" Apr 22 15:07:41.412444 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.412431 2577 cpu_manager.go:222] "Starting CPU manager" policy="none" Apr 22 15:07:41.412491 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.412444 2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Apr 22 15:07:41.412491 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.412459 2577 state_mem.go:36] "Initialized new in-memory state store" Apr 22 15:07:41.414419 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:41.414354 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-168.ec2.internal.18a8b6453857b929 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-168.ec2.internal,UID:ip-10-0-131-168.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-168.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-168.ec2.internal,},FirstTimestamp:2026-04-22 15:07:41.411932457 +0000 UTC m=+0.468670533,LastTimestamp:2026-04-22 15:07:41.411932457 +0000 UTC m=+0.468670533,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-168.ec2.internal,}" Apr 22 15:07:41.415399 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.415387 2577 policy_none.go:49] "None policy: Start" Apr 22 15:07:41.415447 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.415404 2577 memory_manager.go:186] "Starting memorymanager" policy="None" Apr 22 15:07:41.415447 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.415414 2577 state_mem.go:35] "Initializing new in-memory state store" Apr 22 15:07:41.417460 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.417444 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-44psn" Apr 22 15:07:41.458992 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.458973 2577 manager.go:341] "Starting Device Plugin manager" Apr 22 15:07:41.479262 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:41.459014 2577 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Apr 22 15:07:41.479262 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.459032 2577 server.go:85] "Starting device plugin registration server" Apr 22 15:07:41.479262 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.459275 2577 eviction_manager.go:189] "Eviction manager: starting control loop" Apr 22 15:07:41.479262 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.459285 2577 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Apr 22 15:07:41.479262 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.459410 2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Apr 22 15:07:41.479262 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.459476 2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Apr 22 15:07:41.479262 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.459484 2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Apr 22 15:07:41.479262 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:41.460244 2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Apr 22 15:07:41.479262 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:41.460281 2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-168.ec2.internal\" not found" Apr 22 15:07:41.500852 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.500820 2577 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Apr 22 15:07:41.502137 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.502119 2577 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Apr 22 15:07:41.502253 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.502148 2577 status_manager.go:230] "Starting to sync pod status with apiserver" Apr 22 15:07:41.502253 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.502178 2577 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Apr 22 15:07:41.502253 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.502185 2577 kubelet.go:2451] "Starting kubelet main sync loop" Apr 22 15:07:41.502425 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:41.502276 2577 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Apr 22 15:07:41.506486 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.506469 2577 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Apr 22 15:07:41.560408 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.560316 2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 22 15:07:41.561307 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.561292 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-168.ec2.internal" event="NodeHasSufficientMemory" Apr 22 15:07:41.561411 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.561322 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-168.ec2.internal" event="NodeHasNoDiskPressure" Apr 22 15:07:41.561411 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.561351 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-168.ec2.internal" event="NodeHasSufficientPID" Apr 22 15:07:41.561411 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.561376 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-168.ec2.internal" Apr 22 15:07:41.570441 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.570420 2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-168.ec2.internal" Apr 22 15:07:41.570529 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:41.570444 2577 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-168.ec2.internal\": node \"ip-10-0-131-168.ec2.internal\" not found" Apr 22 15:07:41.592057 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:41.592032 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-168.ec2.internal\" not found" Apr 22 15:07:41.603817 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.603796 2577 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-168.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-168.ec2.internal"] Apr 22 15:07:41.603873 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.603857 2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 22 15:07:41.604746 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.604734 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-168.ec2.internal" event="NodeHasSufficientMemory" Apr 22 15:07:41.604798 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.604760 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-168.ec2.internal" event="NodeHasNoDiskPressure" Apr 22 15:07:41.604798 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.604770 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-168.ec2.internal" event="NodeHasSufficientPID" Apr 22 15:07:41.605897 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.605886 2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 22 15:07:41.606039 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.606024 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-168.ec2.internal" Apr 22 15:07:41.606090 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.606054 2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 22 15:07:41.606618 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.606600 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-168.ec2.internal" event="NodeHasSufficientMemory" Apr 22 15:07:41.606704 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.606620 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-168.ec2.internal" event="NodeHasSufficientMemory" Apr 22 15:07:41.606704 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.606642 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-168.ec2.internal" event="NodeHasNoDiskPressure" Apr 22 15:07:41.606704 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.606651 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-168.ec2.internal" event="NodeHasSufficientPID" Apr 22 15:07:41.606704 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.606624 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-168.ec2.internal" event="NodeHasNoDiskPressure" Apr 22 15:07:41.606837 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.606713 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-168.ec2.internal" event="NodeHasSufficientPID" Apr 22 15:07:41.607982 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.607966 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-168.ec2.internal" Apr 22 15:07:41.608054 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.607991 2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 22 15:07:41.608624 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.608611 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-168.ec2.internal" event="NodeHasSufficientMemory" Apr 22 15:07:41.608687 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.608636 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-168.ec2.internal" event="NodeHasNoDiskPressure" Apr 22 15:07:41.608687 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.608647 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-168.ec2.internal" event="NodeHasSufficientPID" Apr 22 15:07:41.644929 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:41.644907 2577 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-168.ec2.internal\" not found" node="ip-10-0-131-168.ec2.internal" Apr 22 15:07:41.649081 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:41.649064 2577 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-168.ec2.internal\" not found" node="ip-10-0-131-168.ec2.internal" Apr 22 15:07:41.692112 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:41.692076 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-168.ec2.internal\" not found" Apr 22 15:07:41.792199 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:41.792166 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-168.ec2.internal\" not found" Apr 22 15:07:41.793371 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.793354 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ccd5fe942c1851cfa510f628904b2dc3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-168.ec2.internal\" (UID: \"ccd5fe942c1851cfa510f628904b2dc3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-168.ec2.internal" Apr 22 15:07:41.793416 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.793391 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ccd5fe942c1851cfa510f628904b2dc3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-168.ec2.internal\" (UID: \"ccd5fe942c1851cfa510f628904b2dc3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-168.ec2.internal" Apr 22 15:07:41.793450 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.793436 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5e64b9b4046044f766b93ef1fd3f3a00-config\") pod \"kube-apiserver-proxy-ip-10-0-131-168.ec2.internal\" (UID: \"5e64b9b4046044f766b93ef1fd3f3a00\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-168.ec2.internal" Apr 22 15:07:41.892820 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:41.892763 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-168.ec2.internal\" not found" Apr 22 15:07:41.893882 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.893866 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ccd5fe942c1851cfa510f628904b2dc3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-168.ec2.internal\" (UID: \"ccd5fe942c1851cfa510f628904b2dc3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-168.ec2.internal" Apr 22 15:07:41.893925 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.893891 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ccd5fe942c1851cfa510f628904b2dc3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-168.ec2.internal\" (UID: \"ccd5fe942c1851cfa510f628904b2dc3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-168.ec2.internal" Apr 22 15:07:41.893925 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.893908 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5e64b9b4046044f766b93ef1fd3f3a00-config\") pod \"kube-apiserver-proxy-ip-10-0-131-168.ec2.internal\" (UID: \"5e64b9b4046044f766b93ef1fd3f3a00\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-168.ec2.internal" Apr 22 15:07:41.893984 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.893945 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5e64b9b4046044f766b93ef1fd3f3a00-config\") pod \"kube-apiserver-proxy-ip-10-0-131-168.ec2.internal\" (UID: \"5e64b9b4046044f766b93ef1fd3f3a00\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-168.ec2.internal" Apr 22 15:07:41.893984 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.893963 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ccd5fe942c1851cfa510f628904b2dc3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-168.ec2.internal\" (UID: \"ccd5fe942c1851cfa510f628904b2dc3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-168.ec2.internal" Apr 22 15:07:41.894084 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.893980 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ccd5fe942c1851cfa510f628904b2dc3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-168.ec2.internal\" (UID: \"ccd5fe942c1851cfa510f628904b2dc3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-168.ec2.internal" Apr 22 15:07:41.947048 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.947013 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-168.ec2.internal" Apr 22 15:07:41.951750 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:41.951733 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-168.ec2.internal" Apr 22 15:07:41.993552 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:41.993515 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-168.ec2.internal\" not found" Apr 22 15:07:42.093943 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:42.093917 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-168.ec2.internal\" not found" Apr 22 15:07:42.194476 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:42.194391 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-168.ec2.internal\" not found" Apr 22 15:07:42.278479 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.278447 2577 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Apr 22 15:07:42.292264 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.292226 2577 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-168.ec2.internal" Apr 22 15:07:42.301468 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.301445 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Apr 22 15:07:42.301636 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.301618 2577 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Apr 22 15:07:42.301636 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.301628 2577 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Apr 22 15:07:42.301719 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:42.301644 2577 kubelet.go:3342] "Failed creating a mirror pod" err="Post \"https://af4be2b8e1ca6421f9c4b67f94b5d72b-81f45104c50f3fdf.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/openshift-machine-config-operator/pods\": read tcp 10.0.131.168:43700->32.195.248.248:6443: use of closed network connection" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-168.ec2.internal" Apr 22 15:07:42.301719 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.301670 2577 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-168.ec2.internal" Apr 22 15:07:42.321718 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.321694 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Apr 22 15:07:42.369054 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.369018 2577 apiserver.go:52] "Watching apiserver" Apr 22 15:07:42.377276 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.377252 2577 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Apr 22 15:07:42.377633 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.377608 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-mzm96","kube-system/kube-apiserver-proxy-ip-10-0-131-168.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2","openshift-image-registry/node-ca-g8wmb","openshift-multus/multus-additional-cni-plugins-2jf6l","openshift-network-operator/iptables-alerter-rrrjp","openshift-cluster-node-tuning-operator/tuned-hz7vw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-168.ec2.internal","openshift-multus/multus-h5xww","openshift-multus/network-metrics-daemon-z9ckr","openshift-network-diagnostics/network-check-target-8lfsw","openshift-ovn-kubernetes/ovnkube-node-s6qsb"] Apr 22 15:07:42.380237 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.380216 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mzm96" Apr 22 15:07:42.381456 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.381435 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" Apr 22 15:07:42.381609 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.381587 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g8wmb" Apr 22 15:07:42.383426 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.383400 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Apr 22 15:07:42.383536 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.383497 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Apr 22 15:07:42.383728 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.383697 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7qqm6\"" Apr 22 15:07:42.384264 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.384246 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Apr 22 15:07:42.384385 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.384341 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6mf7c\"" Apr 22 15:07:42.384558 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.384453 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Apr 22 15:07:42.385487 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.385068 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Apr 22 15:07:42.385487 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.385138 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Apr 22 15:07:42.385487 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.385264 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rrf8w\"" Apr 22 15:07:42.385487 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.385068 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Apr 22 15:07:42.385487 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.385416 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Apr 22 15:07:42.385762 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.385505 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rrrjp" Apr 22 15:07:42.386867 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.386834 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2jf6l" Apr 22 15:07:42.386948 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.386923 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.387929 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.387911 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kzl22\"" Apr 22 15:07:42.388340 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.388313 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.388423 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.388366 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Apr 22 15:07:42.388877 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.388861 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Apr 22 15:07:42.389240 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.389192 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Apr 22 15:07:42.389320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.389303 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Apr 22 15:07:42.389399 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.389315 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Apr 22 15:07:42.389532 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.389514 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:07:42.389595 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.389559 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Apr 22 15:07:42.389648 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:42.389588 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z9ckr" podUID="487f23b6-9470-41e9-abbd-295b6d317b10" Apr 22 15:07:42.389648 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.389602 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Apr 22 15:07:42.389648 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.389632 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Apr 22 15:07:42.389786 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.389712 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qjp2l\"" Apr 22 15:07:42.389917 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.389899 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Apr 22 15:07:42.389970 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.389946 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Apr 22 15:07:42.390252 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.390236 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zdh5s\"" Apr 22 15:07:42.390320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.390256 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Apr 22 15:07:42.390408 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.390390 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-k5vfw\"" Apr 22 15:07:42.390978 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.390964 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:07:42.391126 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:42.391101 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lfsw" podUID="68a481e1-e027-433e-84ea-cb8d42406d8f" Apr 22 15:07:42.391126 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.391110 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Apr 22 15:07:42.392563 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.392548 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.393550 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.393533 2577 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Apr 22 15:07:42.394642 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.394625 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Apr 22 15:07:42.394740 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.394698 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Apr 22 15:07:42.395092 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.395077 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Apr 22 15:07:42.395171 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.395141 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Apr 22 15:07:42.395224 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.395188 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4dwgj\"" Apr 22 15:07:42.395517 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.395500 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Apr 22 15:07:42.395597 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.395541 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Apr 22 15:07:42.395712 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.395694 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6b9559b7-6b5b-4ecf-9655-4379447b15c7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2jf6l\" (UID: \"6b9559b7-6b5b-4ecf-9655-4379447b15c7\") " pod="openshift-multus/multus-additional-cni-plugins-2jf6l" Apr 22 15:07:42.395753 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.395731 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/487f23b6-9470-41e9-abbd-295b6d317b10-metrics-certs\") pod \"network-metrics-daemon-z9ckr\" (UID: \"487f23b6-9470-41e9-abbd-295b6d317b10\") " pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:07:42.395785 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.395766 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-etc-sysctl-d\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.395817 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.395791 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/be504427-e741-4041-98dc-92cc1db1293a-etc-tuned\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.395856 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.395842 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-run-systemd\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.395910 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.395889 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-ovn-node-metrics-cert\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.395954 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.395938 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0510159c-4726-4872-8eb0-5d435faa0b32-sys-fs\") pod \"aws-ebs-csi-driver-node-vmfs2\" (UID: \"0510159c-4726-4872-8eb0-5d435faa0b32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" Apr 22 15:07:42.395991 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.395964 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6b9559b7-6b5b-4ecf-9655-4379447b15c7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2jf6l\" (UID: \"6b9559b7-6b5b-4ecf-9655-4379447b15c7\") " pod="openshift-multus/multus-additional-cni-plugins-2jf6l" Apr 22 15:07:42.396039 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.395990 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvk42\" (UniqueName: \"kubernetes.io/projected/6b9559b7-6b5b-4ecf-9655-4379447b15c7-kube-api-access-kvk42\") pod \"multus-additional-cni-plugins-2jf6l\" (UID: \"6b9559b7-6b5b-4ecf-9655-4379447b15c7\") " pod="openshift-multus/multus-additional-cni-plugins-2jf6l" Apr 22 15:07:42.396039 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396013 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-etc-modprobe-d\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.396137 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396057 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-etc-kubernetes\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.396137 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396103 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-run\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.396233 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396133 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-log-socket\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.396233 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396158 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjdwk\" (UniqueName: \"kubernetes.io/projected/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-kube-api-access-tjdwk\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.396233 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396186 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-host-var-lib-cni-multus\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.396233 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396211 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.396429 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396236 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9dc3cd64-355a-4b4b-a5e1-5d165978d8a3-serviceca\") pod \"node-ca-g8wmb\" (UID: \"9dc3cd64-355a-4b4b-a5e1-5d165978d8a3\") " pod="openshift-image-registry/node-ca-g8wmb" Apr 22 15:07:42.396429 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396259 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0510159c-4726-4872-8eb0-5d435faa0b32-socket-dir\") pod \"aws-ebs-csi-driver-node-vmfs2\" (UID: \"0510159c-4726-4872-8eb0-5d435faa0b32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" Apr 22 15:07:42.396429 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396354 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rdnm\" (UniqueName: \"kubernetes.io/projected/487f23b6-9470-41e9-abbd-295b6d317b10-kube-api-access-5rdnm\") pod \"network-metrics-daemon-z9ckr\" (UID: \"487f23b6-9470-41e9-abbd-295b6d317b10\") " pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:07:42.396429 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396384 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-etc-systemd\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.396543 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396438 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-host-slash\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.396543 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396484 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0510159c-4726-4872-8eb0-5d435faa0b32-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vmfs2\" (UID: \"0510159c-4726-4872-8eb0-5d435faa0b32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" Apr 22 15:07:42.396543 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396512 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8skwh\" (UniqueName: \"kubernetes.io/projected/9dc3cd64-355a-4b4b-a5e1-5d165978d8a3-kube-api-access-8skwh\") pod \"node-ca-g8wmb\" (UID: \"9dc3cd64-355a-4b4b-a5e1-5d165978d8a3\") " pod="openshift-image-registry/node-ca-g8wmb" Apr 22 15:07:42.396634 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396544 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b9559b7-6b5b-4ecf-9655-4379447b15c7-system-cni-dir\") pod \"multus-additional-cni-plugins-2jf6l\" (UID: \"6b9559b7-6b5b-4ecf-9655-4379447b15c7\") " pod="openshift-multus/multus-additional-cni-plugins-2jf6l" Apr 22 15:07:42.396634 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396574 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-multus-socket-dir-parent\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.396634 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396621 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-etc-sysconfig\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.396744 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396647 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ab8e8378-2063-4f8e-bb2a-cc3cd4ca56d1-konnectivity-ca\") pod \"konnectivity-agent-mzm96\" (UID: \"ab8e8378-2063-4f8e-bb2a-cc3cd4ca56d1\") " pod="kube-system/konnectivity-agent-mzm96" Apr 22 15:07:42.396744 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396670 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0510159c-4726-4872-8eb0-5d435faa0b32-registration-dir\") pod \"aws-ebs-csi-driver-node-vmfs2\" (UID: \"0510159c-4726-4872-8eb0-5d435faa0b32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" Apr 22 15:07:42.396744 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396691 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9dc3cd64-355a-4b4b-a5e1-5d165978d8a3-host\") pod \"node-ca-g8wmb\" (UID: \"9dc3cd64-355a-4b4b-a5e1-5d165978d8a3\") " pod="openshift-image-registry/node-ca-g8wmb" Apr 22 15:07:42.396744 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396712 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6b9559b7-6b5b-4ecf-9655-4379447b15c7-cni-binary-copy\") pod \"multus-additional-cni-plugins-2jf6l\" (UID: \"6b9559b7-6b5b-4ecf-9655-4379447b15c7\") " pod="openshift-multus/multus-additional-cni-plugins-2jf6l" Apr 22 15:07:42.396744 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396735 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-var-lib-openvswitch\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.396911 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396782 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-etc-openvswitch\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.396911 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396809 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-node-log\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.396911 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396826 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6b9559b7-6b5b-4ecf-9655-4379447b15c7-cnibin\") pod \"multus-additional-cni-plugins-2jf6l\" (UID: \"6b9559b7-6b5b-4ecf-9655-4379447b15c7\") " pod="openshift-multus/multus-additional-cni-plugins-2jf6l" Apr 22 15:07:42.396911 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396841 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-multus-cni-dir\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.396911 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396861 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-hostroot\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.396911 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396876 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74hl5\" (UniqueName: \"kubernetes.io/projected/727c0127-0608-49d1-a2a3-aa2d700b9898-kube-api-access-74hl5\") pod \"iptables-alerter-rrrjp\" (UID: \"727c0127-0608-49d1-a2a3-aa2d700b9898\") " pod="openshift-network-operator/iptables-alerter-rrrjp" Apr 22 15:07:42.396911 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396892 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-sys\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.397183 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396943 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9n55\" (UniqueName: \"kubernetes.io/projected/68a481e1-e027-433e-84ea-cb8d42406d8f-kube-api-access-w9n55\") pod \"network-check-target-8lfsw\" (UID: \"68a481e1-e027-433e-84ea-cb8d42406d8f\") " pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:07:42.397183 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.396991 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-host-run-netns\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.397183 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397018 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-host-cni-netd\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.397183 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397045 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj8f4\" (UniqueName: \"kubernetes.io/projected/0510159c-4726-4872-8eb0-5d435faa0b32-kube-api-access-xj8f4\") pod \"aws-ebs-csi-driver-node-vmfs2\" (UID: \"0510159c-4726-4872-8eb0-5d435faa0b32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" Apr 22 15:07:42.397183 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397069 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-system-cni-dir\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.397183 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397110 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-cnibin\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.397183 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397141 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-os-release\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.397183 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397166 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-host-run-k8s-cni-cncf-io\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.397473 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397190 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-env-overrides\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.397473 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397216 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ab8e8378-2063-4f8e-bb2a-cc3cd4ca56d1-agent-certs\") pod \"konnectivity-agent-mzm96\" (UID: \"ab8e8378-2063-4f8e-bb2a-cc3cd4ca56d1\") " pod="kube-system/konnectivity-agent-mzm96" Apr 22 15:07:42.397473 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397241 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6b9559b7-6b5b-4ecf-9655-4379447b15c7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2jf6l\" (UID: \"6b9559b7-6b5b-4ecf-9655-4379447b15c7\") " pod="openshift-multus/multus-additional-cni-plugins-2jf6l" Apr 22 15:07:42.397473 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397266 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-multus-conf-dir\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.397473 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397291 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/727c0127-0608-49d1-a2a3-aa2d700b9898-iptables-alerter-script\") pod \"iptables-alerter-rrrjp\" (UID: \"727c0127-0608-49d1-a2a3-aa2d700b9898\") " pod="openshift-network-operator/iptables-alerter-rrrjp" Apr 22 15:07:42.397473 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397344 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-systemd-units\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.397473 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397382 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0510159c-4726-4872-8eb0-5d435faa0b32-device-dir\") pod \"aws-ebs-csi-driver-node-vmfs2\" (UID: \"0510159c-4726-4872-8eb0-5d435faa0b32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" Apr 22 15:07:42.397473 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397404 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0510159c-4726-4872-8eb0-5d435faa0b32-etc-selinux\") pod \"aws-ebs-csi-driver-node-vmfs2\" (UID: \"0510159c-4726-4872-8eb0-5d435faa0b32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" Apr 22 15:07:42.397473 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397435 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-lib-modules\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.397473 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397472 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-run-openvswitch\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.397901 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397521 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-host-run-ovn-kubernetes\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.397901 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397574 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-host-run-netns\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.397901 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397615 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-host-run-multus-certs\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.397901 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397650 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvrrn\" (UniqueName: \"kubernetes.io/projected/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-kube-api-access-nvrrn\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.397901 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397684 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/727c0127-0608-49d1-a2a3-aa2d700b9898-host-slash\") pod \"iptables-alerter-rrrjp\" (UID: \"727c0127-0608-49d1-a2a3-aa2d700b9898\") " pod="openshift-network-operator/iptables-alerter-rrrjp" Apr 22 15:07:42.397901 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397740 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-etc-sysctl-conf\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.397901 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397784 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cfcb\" (UniqueName: \"kubernetes.io/projected/be504427-e741-4041-98dc-92cc1db1293a-kube-api-access-5cfcb\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.397901 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397811 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-ovnkube-script-lib\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.397901 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397833 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-cni-binary-copy\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.397901 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397862 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-host-var-lib-cni-bin\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.398235 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397900 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-etc-kubernetes\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.398235 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397941 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-var-lib-kubelet\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.398235 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.397989 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-ovnkube-config\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.398235 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.398041 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-host-var-lib-kubelet\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.398235 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.398084 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-multus-daemon-config\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.398235 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.398112 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-run-ovn\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.398235 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.398153 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-host-cni-bin\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.398235 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.398181 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-host\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.398235 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.398211 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/be504427-e741-4041-98dc-92cc1db1293a-tmp\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.398235 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.398233 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-host-kubelet\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.398724 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.398255 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6b9559b7-6b5b-4ecf-9655-4379447b15c7-os-release\") pod \"multus-additional-cni-plugins-2jf6l\" (UID: \"6b9559b7-6b5b-4ecf-9655-4379447b15c7\") " pod="openshift-multus/multus-additional-cni-plugins-2jf6l" Apr 22 15:07:42.409608 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.409588 2577 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Apr 22 15:07:42.417693 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:42.417524 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccd5fe942c1851cfa510f628904b2dc3.slice/crio-374e5265f940fe628c0d058d21de9cc82bfed5e42d4ea62a6e1fcdd95d63dcac WatchSource:0}: Error finding container 374e5265f940fe628c0d058d21de9cc82bfed5e42d4ea62a6e1fcdd95d63dcac: Status 404 returned error can't find the container with id 374e5265f940fe628c0d058d21de9cc82bfed5e42d4ea62a6e1fcdd95d63dcac Apr 22 15:07:42.418047 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:42.418029 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e64b9b4046044f766b93ef1fd3f3a00.slice/crio-d7373b1249a6cbec69666beff4a7ea7abc6fdbf2d0c4b1ada511941377c07037 WatchSource:0}: Error finding container d7373b1249a6cbec69666beff4a7ea7abc6fdbf2d0c4b1ada511941377c07037: Status 404 returned error can't find the container with id d7373b1249a6cbec69666beff4a7ea7abc6fdbf2d0c4b1ada511941377c07037 Apr 22 15:07:42.419988 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.419951 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-04-21 15:02:41 +0000 UTC" deadline="2027-09-22 01:38:15.596688039 +0000 UTC" Apr 22 15:07:42.419988 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.419984 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12418h30m33.176706963s" Apr 22 15:07:42.423356 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.423318 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 22 15:07:42.460715 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.460693 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dc44t" Apr 22 15:07:42.469265 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.469237 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dc44t" Apr 22 15:07:42.498576 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.498544 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6b9559b7-6b5b-4ecf-9655-4379447b15c7-os-release\") pod \"multus-additional-cni-plugins-2jf6l\" (UID: \"6b9559b7-6b5b-4ecf-9655-4379447b15c7\") " pod="openshift-multus/multus-additional-cni-plugins-2jf6l" Apr 22 15:07:42.498576 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.498575 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6b9559b7-6b5b-4ecf-9655-4379447b15c7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2jf6l\" (UID: \"6b9559b7-6b5b-4ecf-9655-4379447b15c7\") " pod="openshift-multus/multus-additional-cni-plugins-2jf6l" Apr 22 15:07:42.499453 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.498597 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/487f23b6-9470-41e9-abbd-295b6d317b10-metrics-certs\") pod \"network-metrics-daemon-z9ckr\" (UID: \"487f23b6-9470-41e9-abbd-295b6d317b10\") " pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:07:42.499453 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.498614 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-etc-sysctl-d\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.499453 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.498628 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/be504427-e741-4041-98dc-92cc1db1293a-etc-tuned\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.499453 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.498643 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-run-systemd\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.499453 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.498664 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-ovn-node-metrics-cert\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.499453 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.498682 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6b9559b7-6b5b-4ecf-9655-4379447b15c7-os-release\") pod \"multus-additional-cni-plugins-2jf6l\" (UID: \"6b9559b7-6b5b-4ecf-9655-4379447b15c7\") " pod="openshift-multus/multus-additional-cni-plugins-2jf6l" Apr 22 15:07:42.499453 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.498687 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0510159c-4726-4872-8eb0-5d435faa0b32-sys-fs\") pod \"aws-ebs-csi-driver-node-vmfs2\" (UID: \"0510159c-4726-4872-8eb0-5d435faa0b32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" Apr 22 15:07:42.499453 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.498737 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0510159c-4726-4872-8eb0-5d435faa0b32-sys-fs\") pod \"aws-ebs-csi-driver-node-vmfs2\" (UID: \"0510159c-4726-4872-8eb0-5d435faa0b32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" Apr 22 15:07:42.499453 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:42.498742 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 15:07:42.499453 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.498767 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6b9559b7-6b5b-4ecf-9655-4379447b15c7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2jf6l\" (UID: \"6b9559b7-6b5b-4ecf-9655-4379447b15c7\") " pod="openshift-multus/multus-additional-cni-plugins-2jf6l" Apr 22 15:07:42.499453 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.498782 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-etc-sysctl-d\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.499453 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.498743 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6b9559b7-6b5b-4ecf-9655-4379447b15c7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2jf6l\" (UID: \"6b9559b7-6b5b-4ecf-9655-4379447b15c7\") " pod="openshift-multus/multus-additional-cni-plugins-2jf6l" Apr 22 15:07:42.499453 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.498739 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-run-systemd\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.499453 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:42.498854 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/487f23b6-9470-41e9-abbd-295b6d317b10-metrics-certs podName:487f23b6-9470-41e9-abbd-295b6d317b10 nodeName:}" failed. No retries permitted until 2026-04-22 15:07:42.998808304 +0000 UTC m=+2.055546384 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/487f23b6-9470-41e9-abbd-295b6d317b10-metrics-certs") pod "network-metrics-daemon-z9ckr" (UID: "487f23b6-9470-41e9-abbd-295b6d317b10") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 15:07:42.499453 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.498910 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kvk42\" (UniqueName: \"kubernetes.io/projected/6b9559b7-6b5b-4ecf-9655-4379447b15c7-kube-api-access-kvk42\") pod \"multus-additional-cni-plugins-2jf6l\" (UID: \"6b9559b7-6b5b-4ecf-9655-4379447b15c7\") " pod="openshift-multus/multus-additional-cni-plugins-2jf6l" Apr 22 15:07:42.499453 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.498940 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-etc-modprobe-d\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.499453 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.498965 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-etc-kubernetes\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.500322 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499000 2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Apr 22 15:07:42.500322 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499010 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-run\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.500322 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499068 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-etc-kubernetes\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.500322 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499070 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-log-socket\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.500322 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499097 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-run\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.500322 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499102 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-log-socket\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.500322 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499118 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tjdwk\" (UniqueName: \"kubernetes.io/projected/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-kube-api-access-tjdwk\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.500322 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499099 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-etc-modprobe-d\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.500322 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499145 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-host-var-lib-cni-multus\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.500322 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499176 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-host-var-lib-cni-multus\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.500322 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499193 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.500322 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499227 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9dc3cd64-355a-4b4b-a5e1-5d165978d8a3-serviceca\") pod \"node-ca-g8wmb\" (UID: \"9dc3cd64-355a-4b4b-a5e1-5d165978d8a3\") " pod="openshift-image-registry/node-ca-g8wmb" Apr 22 15:07:42.500322 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499273 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0510159c-4726-4872-8eb0-5d435faa0b32-socket-dir\") pod \"aws-ebs-csi-driver-node-vmfs2\" (UID: \"0510159c-4726-4872-8eb0-5d435faa0b32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" Apr 22 15:07:42.500322 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499294 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6b9559b7-6b5b-4ecf-9655-4379447b15c7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2jf6l\" (UID: \"6b9559b7-6b5b-4ecf-9655-4379447b15c7\") " pod="openshift-multus/multus-additional-cni-plugins-2jf6l" Apr 22 15:07:42.500322 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499338 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5rdnm\" (UniqueName: \"kubernetes.io/projected/487f23b6-9470-41e9-abbd-295b6d317b10-kube-api-access-5rdnm\") pod \"network-metrics-daemon-z9ckr\" (UID: \"487f23b6-9470-41e9-abbd-295b6d317b10\") " pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:07:42.500322 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499367 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-etc-systemd\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.500322 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499397 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-host-slash\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.500322 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499422 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0510159c-4726-4872-8eb0-5d435faa0b32-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vmfs2\" (UID: \"0510159c-4726-4872-8eb0-5d435faa0b32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" Apr 22 15:07:42.501300 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499446 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8skwh\" (UniqueName: \"kubernetes.io/projected/9dc3cd64-355a-4b4b-a5e1-5d165978d8a3-kube-api-access-8skwh\") pod \"node-ca-g8wmb\" (UID: \"9dc3cd64-355a-4b4b-a5e1-5d165978d8a3\") " pod="openshift-image-registry/node-ca-g8wmb" Apr 22 15:07:42.501300 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499447 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0510159c-4726-4872-8eb0-5d435faa0b32-socket-dir\") pod \"aws-ebs-csi-driver-node-vmfs2\" (UID: \"0510159c-4726-4872-8eb0-5d435faa0b32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" Apr 22 15:07:42.501300 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499482 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b9559b7-6b5b-4ecf-9655-4379447b15c7-system-cni-dir\") pod \"multus-additional-cni-plugins-2jf6l\" (UID: \"6b9559b7-6b5b-4ecf-9655-4379447b15c7\") " pod="openshift-multus/multus-additional-cni-plugins-2jf6l" Apr 22 15:07:42.501300 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499509 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-multus-socket-dir-parent\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.501300 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499512 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.501300 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499532 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-etc-sysconfig\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.501300 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499556 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b9559b7-6b5b-4ecf-9655-4379447b15c7-system-cni-dir\") pod \"multus-additional-cni-plugins-2jf6l\" (UID: \"6b9559b7-6b5b-4ecf-9655-4379447b15c7\") " pod="openshift-multus/multus-additional-cni-plugins-2jf6l" Apr 22 15:07:42.501300 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499560 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ab8e8378-2063-4f8e-bb2a-cc3cd4ca56d1-konnectivity-ca\") pod \"konnectivity-agent-mzm96\" (UID: \"ab8e8378-2063-4f8e-bb2a-cc3cd4ca56d1\") " pod="kube-system/konnectivity-agent-mzm96" Apr 22 15:07:42.501300 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499584 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0510159c-4726-4872-8eb0-5d435faa0b32-registration-dir\") pod \"aws-ebs-csi-driver-node-vmfs2\" (UID: \"0510159c-4726-4872-8eb0-5d435faa0b32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" Apr 22 15:07:42.501300 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499591 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-multus-socket-dir-parent\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.501300 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499607 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9dc3cd64-355a-4b4b-a5e1-5d165978d8a3-host\") pod \"node-ca-g8wmb\" (UID: \"9dc3cd64-355a-4b4b-a5e1-5d165978d8a3\") " pod="openshift-image-registry/node-ca-g8wmb" Apr 22 15:07:42.501300 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499615 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9dc3cd64-355a-4b4b-a5e1-5d165978d8a3-serviceca\") pod \"node-ca-g8wmb\" (UID: \"9dc3cd64-355a-4b4b-a5e1-5d165978d8a3\") " pod="openshift-image-registry/node-ca-g8wmb" Apr 22 15:07:42.501300 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499630 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-etc-systemd\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.501300 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499632 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6b9559b7-6b5b-4ecf-9655-4379447b15c7-cni-binary-copy\") pod \"multus-additional-cni-plugins-2jf6l\" (UID: \"6b9559b7-6b5b-4ecf-9655-4379447b15c7\") " pod="openshift-multus/multus-additional-cni-plugins-2jf6l" Apr 22 15:07:42.501300 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499647 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0510159c-4726-4872-8eb0-5d435faa0b32-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vmfs2\" (UID: \"0510159c-4726-4872-8eb0-5d435faa0b32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" Apr 22 15:07:42.501300 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499594 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-host-slash\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.501300 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499670 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-var-lib-openvswitch\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.502138 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499684 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9dc3cd64-355a-4b4b-a5e1-5d165978d8a3-host\") pod \"node-ca-g8wmb\" (UID: \"9dc3cd64-355a-4b4b-a5e1-5d165978d8a3\") " pod="openshift-image-registry/node-ca-g8wmb" Apr 22 15:07:42.502138 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499689 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0510159c-4726-4872-8eb0-5d435faa0b32-registration-dir\") pod \"aws-ebs-csi-driver-node-vmfs2\" (UID: \"0510159c-4726-4872-8eb0-5d435faa0b32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" Apr 22 15:07:42.502138 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499694 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-etc-openvswitch\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.502138 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499702 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-etc-sysconfig\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.502138 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499722 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-node-log\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.502138 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499738 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-etc-openvswitch\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.502138 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499741 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-var-lib-openvswitch\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.502138 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499750 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6b9559b7-6b5b-4ecf-9655-4379447b15c7-cnibin\") pod \"multus-additional-cni-plugins-2jf6l\" (UID: \"6b9559b7-6b5b-4ecf-9655-4379447b15c7\") " pod="openshift-multus/multus-additional-cni-plugins-2jf6l" Apr 22 15:07:42.502138 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499775 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-node-log\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.502138 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499784 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-multus-cni-dir\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.502138 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499809 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-hostroot\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.502138 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499815 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6b9559b7-6b5b-4ecf-9655-4379447b15c7-cnibin\") pod \"multus-additional-cni-plugins-2jf6l\" (UID: \"6b9559b7-6b5b-4ecf-9655-4379447b15c7\") " pod="openshift-multus/multus-additional-cni-plugins-2jf6l" Apr 22 15:07:42.502138 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499833 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-74hl5\" (UniqueName: \"kubernetes.io/projected/727c0127-0608-49d1-a2a3-aa2d700b9898-kube-api-access-74hl5\") pod \"iptables-alerter-rrrjp\" (UID: \"727c0127-0608-49d1-a2a3-aa2d700b9898\") " pod="openshift-network-operator/iptables-alerter-rrrjp" Apr 22 15:07:42.502138 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499835 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-multus-cni-dir\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.502138 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499865 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-sys\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.502138 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499869 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-hostroot\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.502138 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499893 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w9n55\" (UniqueName: \"kubernetes.io/projected/68a481e1-e027-433e-84ea-cb8d42406d8f-kube-api-access-w9n55\") pod \"network-check-target-8lfsw\" (UID: \"68a481e1-e027-433e-84ea-cb8d42406d8f\") " pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:07:42.502138 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499919 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-host-run-netns\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.502999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499920 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-sys\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.502999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.499951 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-host-run-netns\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.502999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500006 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-host-cni-netd\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.502999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500033 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xj8f4\" (UniqueName: \"kubernetes.io/projected/0510159c-4726-4872-8eb0-5d435faa0b32-kube-api-access-xj8f4\") pod \"aws-ebs-csi-driver-node-vmfs2\" (UID: \"0510159c-4726-4872-8eb0-5d435faa0b32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" Apr 22 15:07:42.502999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500072 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-system-cni-dir\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.502999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500075 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-host-cni-netd\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.502999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500094 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-cnibin\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.502999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500139 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-os-release\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.502999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500141 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ab8e8378-2063-4f8e-bb2a-cc3cd4ca56d1-konnectivity-ca\") pod \"konnectivity-agent-mzm96\" (UID: \"ab8e8378-2063-4f8e-bb2a-cc3cd4ca56d1\") " pod="kube-system/konnectivity-agent-mzm96" Apr 22 15:07:42.502999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500164 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-host-run-k8s-cni-cncf-io\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.502999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500199 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-env-overrides\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.502999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500204 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-system-cni-dir\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.502999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500224 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ab8e8378-2063-4f8e-bb2a-cc3cd4ca56d1-agent-certs\") pod \"konnectivity-agent-mzm96\" (UID: \"ab8e8378-2063-4f8e-bb2a-cc3cd4ca56d1\") " pod="kube-system/konnectivity-agent-mzm96" Apr 22 15:07:42.502999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500253 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6b9559b7-6b5b-4ecf-9655-4379447b15c7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2jf6l\" (UID: \"6b9559b7-6b5b-4ecf-9655-4379447b15c7\") " pod="openshift-multus/multus-additional-cni-plugins-2jf6l" Apr 22 15:07:42.502999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500283 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-multus-conf-dir\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.502999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500286 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-os-release\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.502999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500266 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6b9559b7-6b5b-4ecf-9655-4379447b15c7-cni-binary-copy\") pod \"multus-additional-cni-plugins-2jf6l\" (UID: \"6b9559b7-6b5b-4ecf-9655-4379447b15c7\") " pod="openshift-multus/multus-additional-cni-plugins-2jf6l" Apr 22 15:07:42.502999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500255 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-host-run-k8s-cni-cncf-io\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.503648 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500296 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-cnibin\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.503648 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500349 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/727c0127-0608-49d1-a2a3-aa2d700b9898-iptables-alerter-script\") pod \"iptables-alerter-rrrjp\" (UID: \"727c0127-0608-49d1-a2a3-aa2d700b9898\") " pod="openshift-network-operator/iptables-alerter-rrrjp" Apr 22 15:07:42.503648 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500364 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-multus-conf-dir\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.503648 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500380 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-systemd-units\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.503648 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500407 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0510159c-4726-4872-8eb0-5d435faa0b32-device-dir\") pod \"aws-ebs-csi-driver-node-vmfs2\" (UID: \"0510159c-4726-4872-8eb0-5d435faa0b32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" Apr 22 15:07:42.503648 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500429 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0510159c-4726-4872-8eb0-5d435faa0b32-etc-selinux\") pod \"aws-ebs-csi-driver-node-vmfs2\" (UID: \"0510159c-4726-4872-8eb0-5d435faa0b32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" Apr 22 15:07:42.503648 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500451 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-lib-modules\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.503648 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500473 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-run-openvswitch\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.503648 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500496 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-host-run-ovn-kubernetes\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.503648 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500524 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-host-run-netns\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.503648 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500550 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-host-run-multus-certs\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.503648 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500589 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nvrrn\" (UniqueName: \"kubernetes.io/projected/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-kube-api-access-nvrrn\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.503648 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500613 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/727c0127-0608-49d1-a2a3-aa2d700b9898-host-slash\") pod \"iptables-alerter-rrrjp\" (UID: \"727c0127-0608-49d1-a2a3-aa2d700b9898\") " pod="openshift-network-operator/iptables-alerter-rrrjp" Apr 22 15:07:42.503648 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500633 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-etc-sysctl-conf\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.503648 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500676 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-lib-modules\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.503648 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500679 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5cfcb\" (UniqueName: \"kubernetes.io/projected/be504427-e741-4041-98dc-92cc1db1293a-kube-api-access-5cfcb\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.503648 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500717 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-ovnkube-script-lib\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.504286 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500743 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-cni-binary-copy\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.504286 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500768 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-host-var-lib-cni-bin\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.504286 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500791 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-etc-kubernetes\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.504286 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500814 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-env-overrides\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.504286 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500814 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6b9559b7-6b5b-4ecf-9655-4379447b15c7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2jf6l\" (UID: \"6b9559b7-6b5b-4ecf-9655-4379447b15c7\") " pod="openshift-multus/multus-additional-cni-plugins-2jf6l" Apr 22 15:07:42.504286 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500816 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-var-lib-kubelet\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.504286 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500856 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-var-lib-kubelet\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.504286 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500871 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-systemd-units\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.504286 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500894 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/727c0127-0608-49d1-a2a3-aa2d700b9898-iptables-alerter-script\") pod \"iptables-alerter-rrrjp\" (UID: \"727c0127-0608-49d1-a2a3-aa2d700b9898\") " pod="openshift-network-operator/iptables-alerter-rrrjp" Apr 22 15:07:42.504286 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500894 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-host-run-ovn-kubernetes\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.504286 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500926 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/727c0127-0608-49d1-a2a3-aa2d700b9898-host-slash\") pod \"iptables-alerter-rrrjp\" (UID: \"727c0127-0608-49d1-a2a3-aa2d700b9898\") " pod="openshift-network-operator/iptables-alerter-rrrjp" Apr 22 15:07:42.504286 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500871 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-ovnkube-config\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.504286 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500927 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-host-run-netns\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.504286 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500983 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-host-var-lib-kubelet\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.504286 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.501012 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-multus-daemon-config\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.504286 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.501022 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0510159c-4726-4872-8eb0-5d435faa0b32-etc-selinux\") pod \"aws-ebs-csi-driver-node-vmfs2\" (UID: \"0510159c-4726-4872-8eb0-5d435faa0b32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" Apr 22 15:07:42.504286 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.501067 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-etc-sysctl-conf\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.504930 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.501071 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-run-ovn\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.504930 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.501086 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-host-var-lib-kubelet\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.504930 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.501112 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-host-cni-bin\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.504930 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.501108 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-run-ovn\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.504930 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.501140 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-host\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.504930 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.501169 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/be504427-e741-4041-98dc-92cc1db1293a-tmp\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.504930 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.501195 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-host-kubelet\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.504930 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.501261 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-host-kubelet\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.504930 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.501300 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-host-var-lib-cni-bin\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.504930 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.500916 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-host-run-multus-certs\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.504930 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.501390 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-run-openvswitch\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.504930 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.501396 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-ovnkube-config\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.504930 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.501439 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0510159c-4726-4872-8eb0-5d435faa0b32-device-dir\") pod \"aws-ebs-csi-driver-node-vmfs2\" (UID: \"0510159c-4726-4872-8eb0-5d435faa0b32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" Apr 22 15:07:42.504930 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.501452 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-cni-binary-copy\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.504930 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.501466 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/be504427-e741-4041-98dc-92cc1db1293a-host\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.504930 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.501473 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-etc-kubernetes\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.504930 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.501481 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-host-cni-bin\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.504930 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.502234 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-ovnkube-script-lib\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.505547 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.502521 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/be504427-e741-4041-98dc-92cc1db1293a-etc-tuned\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.505547 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.502585 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-multus-daemon-config\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.505547 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.502941 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ab8e8378-2063-4f8e-bb2a-cc3cd4ca56d1-agent-certs\") pod \"konnectivity-agent-mzm96\" (UID: \"ab8e8378-2063-4f8e-bb2a-cc3cd4ca56d1\") " pod="kube-system/konnectivity-agent-mzm96" Apr 22 15:07:42.505547 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.503227 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-ovn-node-metrics-cert\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.505547 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.503578 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/be504427-e741-4041-98dc-92cc1db1293a-tmp\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.505547 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.504951 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-168.ec2.internal" event={"ID":"ccd5fe942c1851cfa510f628904b2dc3","Type":"ContainerStarted","Data":"374e5265f940fe628c0d058d21de9cc82bfed5e42d4ea62a6e1fcdd95d63dcac"} Apr 22 15:07:42.506145 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.506123 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-168.ec2.internal" event={"ID":"5e64b9b4046044f766b93ef1fd3f3a00","Type":"ContainerStarted","Data":"d7373b1249a6cbec69666beff4a7ea7abc6fdbf2d0c4b1ada511941377c07037"} Apr 22 15:07:42.508213 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.508193 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvk42\" (UniqueName: \"kubernetes.io/projected/6b9559b7-6b5b-4ecf-9655-4379447b15c7-kube-api-access-kvk42\") pod \"multus-additional-cni-plugins-2jf6l\" (UID: \"6b9559b7-6b5b-4ecf-9655-4379447b15c7\") " pod="openshift-multus/multus-additional-cni-plugins-2jf6l" Apr 22 15:07:42.508369 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.508313 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjdwk\" (UniqueName: \"kubernetes.io/projected/bdcb6dd7-b076-455a-8529-a6c7a1eeae89-kube-api-access-tjdwk\") pod \"ovnkube-node-s6qsb\" (UID: \"bdcb6dd7-b076-455a-8529-a6c7a1eeae89\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.508641 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.508627 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rdnm\" (UniqueName: \"kubernetes.io/projected/487f23b6-9470-41e9-abbd-295b6d317b10-kube-api-access-5rdnm\") pod \"network-metrics-daemon-z9ckr\" (UID: \"487f23b6-9470-41e9-abbd-295b6d317b10\") " pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:07:42.509624 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:42.509571 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 22 15:07:42.509624 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:42.509596 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 22 15:07:42.509624 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:42.509611 2577 projected.go:194] Error preparing data for projected volume kube-api-access-w9n55 for pod openshift-network-diagnostics/network-check-target-8lfsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 22 15:07:42.509795 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:42.509691 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68a481e1-e027-433e-84ea-cb8d42406d8f-kube-api-access-w9n55 podName:68a481e1-e027-433e-84ea-cb8d42406d8f nodeName:}" failed. No retries permitted until 2026-04-22 15:07:43.009670729 +0000 UTC m=+2.066408805 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-w9n55" (UniqueName: "kubernetes.io/projected/68a481e1-e027-433e-84ea-cb8d42406d8f-kube-api-access-w9n55") pod "network-check-target-8lfsw" (UID: "68a481e1-e027-433e-84ea-cb8d42406d8f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 22 15:07:42.511293 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.511274 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cfcb\" (UniqueName: \"kubernetes.io/projected/be504427-e741-4041-98dc-92cc1db1293a-kube-api-access-5cfcb\") pod \"tuned-hz7vw\" (UID: \"be504427-e741-4041-98dc-92cc1db1293a\") " pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.511626 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.511601 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8skwh\" (UniqueName: \"kubernetes.io/projected/9dc3cd64-355a-4b4b-a5e1-5d165978d8a3-kube-api-access-8skwh\") pod \"node-ca-g8wmb\" (UID: \"9dc3cd64-355a-4b4b-a5e1-5d165978d8a3\") " pod="openshift-image-registry/node-ca-g8wmb" Apr 22 15:07:42.511699 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.511668 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj8f4\" (UniqueName: \"kubernetes.io/projected/0510159c-4726-4872-8eb0-5d435faa0b32-kube-api-access-xj8f4\") pod \"aws-ebs-csi-driver-node-vmfs2\" (UID: \"0510159c-4726-4872-8eb0-5d435faa0b32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" Apr 22 15:07:42.511787 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.511768 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74hl5\" (UniqueName: \"kubernetes.io/projected/727c0127-0608-49d1-a2a3-aa2d700b9898-kube-api-access-74hl5\") pod \"iptables-alerter-rrrjp\" (UID: \"727c0127-0608-49d1-a2a3-aa2d700b9898\") " pod="openshift-network-operator/iptables-alerter-rrrjp" Apr 22 15:07:42.511939 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.511926 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvrrn\" (UniqueName: \"kubernetes.io/projected/0c332b2a-7ac7-4b26-b9d4-272932e01b8e-kube-api-access-nvrrn\") pod \"multus-h5xww\" (UID: \"0c332b2a-7ac7-4b26-b9d4-272932e01b8e\") " pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.680114 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.680079 2577 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Apr 22 15:07:42.724903 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.724825 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mzm96" Apr 22 15:07:42.730630 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.730611 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" Apr 22 15:07:42.732063 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:42.732037 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab8e8378_2063_4f8e_bb2a_cc3cd4ca56d1.slice/crio-636b60fb5ea7a7e34500d145a561469a142da1873b231a265ebc7eefeed4670e WatchSource:0}: Error finding container 636b60fb5ea7a7e34500d145a561469a142da1873b231a265ebc7eefeed4670e: Status 404 returned error can't find the container with id 636b60fb5ea7a7e34500d145a561469a142da1873b231a265ebc7eefeed4670e Apr 22 15:07:42.737189 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:42.737166 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0510159c_4726_4872_8eb0_5d435faa0b32.slice/crio-8e4811109892954d7d71f1432aec8db0d88565025d06b86f0c04b7935cf3943a WatchSource:0}: Error finding container 8e4811109892954d7d71f1432aec8db0d88565025d06b86f0c04b7935cf3943a: Status 404 returned error can't find the container with id 8e4811109892954d7d71f1432aec8db0d88565025d06b86f0c04b7935cf3943a Apr 22 15:07:42.737281 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.737203 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g8wmb" Apr 22 15:07:42.743422 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.743397 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rrrjp" Apr 22 15:07:42.743685 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:42.743661 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dc3cd64_355a_4b4b_a5e1_5d165978d8a3.slice/crio-552b722409d5759ba34bfb41d9e5b8eeb0d2f740d2e24df37899f84576cb4e78 WatchSource:0}: Error finding container 552b722409d5759ba34bfb41d9e5b8eeb0d2f740d2e24df37899f84576cb4e78: Status 404 returned error can't find the container with id 552b722409d5759ba34bfb41d9e5b8eeb0d2f740d2e24df37899f84576cb4e78 Apr 22 15:07:42.748681 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.748659 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2jf6l" Apr 22 15:07:42.751001 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:42.750980 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod727c0127_0608_49d1_a2a3_aa2d700b9898.slice/crio-34aab2373d16659183ec72f516aed2bb11e4dd295a9669f3bdb663c9d87116e6 WatchSource:0}: Error finding container 34aab2373d16659183ec72f516aed2bb11e4dd295a9669f3bdb663c9d87116e6: Status 404 returned error can't find the container with id 34aab2373d16659183ec72f516aed2bb11e4dd295a9669f3bdb663c9d87116e6 Apr 22 15:07:42.754546 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.754528 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" Apr 22 15:07:42.756309 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:42.756283 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b9559b7_6b5b_4ecf_9655_4379447b15c7.slice/crio-139f35dbe1f6e6c5cb1d083c5cbc10739868ed515eaf0d3c7f15591bb0d0f97f WatchSource:0}: Error finding container 139f35dbe1f6e6c5cb1d083c5cbc10739868ed515eaf0d3c7f15591bb0d0f97f: Status 404 returned error can't find the container with id 139f35dbe1f6e6c5cb1d083c5cbc10739868ed515eaf0d3c7f15591bb0d0f97f Apr 22 15:07:42.760229 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.760211 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h5xww" Apr 22 15:07:42.760995 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:42.760975 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe504427_e741_4041_98dc_92cc1db1293a.slice/crio-83b1e520c47d269c7483ff9dba4979a027060654c9ffa1923f427e566763b5fa WatchSource:0}: Error finding container 83b1e520c47d269c7483ff9dba4979a027060654c9ffa1923f427e566763b5fa: Status 404 returned error can't find the container with id 83b1e520c47d269c7483ff9dba4979a027060654c9ffa1923f427e566763b5fa Apr 22 15:07:42.764951 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.764882 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:07:42.766899 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:42.766876 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c332b2a_7ac7_4b26_b9d4_272932e01b8e.slice/crio-5811852c0f3884b9e0234624be68e1050a6df36b56245b76cc9533ad0225ad7b WatchSource:0}: Error finding container 5811852c0f3884b9e0234624be68e1050a6df36b56245b76cc9533ad0225ad7b: Status 404 returned error can't find the container with id 5811852c0f3884b9e0234624be68e1050a6df36b56245b76cc9533ad0225ad7b Apr 22 15:07:42.771918 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:07:42.771895 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdcb6dd7_b076_455a_8529_a6c7a1eeae89.slice/crio-cb5c39b15d0f2d56c2b93aaf6724075943acf5ea9f35799f987b31ed6148a38d WatchSource:0}: Error finding container cb5c39b15d0f2d56c2b93aaf6724075943acf5ea9f35799f987b31ed6148a38d: Status 404 returned error can't find the container with id cb5c39b15d0f2d56c2b93aaf6724075943acf5ea9f35799f987b31ed6148a38d Apr 22 15:07:42.899051 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:42.899017 2577 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Apr 22 15:07:43.004818 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:43.004784 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/487f23b6-9470-41e9-abbd-295b6d317b10-metrics-certs\") pod \"network-metrics-daemon-z9ckr\" (UID: \"487f23b6-9470-41e9-abbd-295b6d317b10\") " pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:07:43.005017 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:43.004952 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 15:07:43.005088 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:43.005032 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/487f23b6-9470-41e9-abbd-295b6d317b10-metrics-certs podName:487f23b6-9470-41e9-abbd-295b6d317b10 nodeName:}" failed. No retries permitted until 2026-04-22 15:07:44.005012593 +0000 UTC m=+3.061750661 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/487f23b6-9470-41e9-abbd-295b6d317b10-metrics-certs") pod "network-metrics-daemon-z9ckr" (UID: "487f23b6-9470-41e9-abbd-295b6d317b10") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 15:07:43.105888 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:43.105849 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w9n55\" (UniqueName: \"kubernetes.io/projected/68a481e1-e027-433e-84ea-cb8d42406d8f-kube-api-access-w9n55\") pod \"network-check-target-8lfsw\" (UID: \"68a481e1-e027-433e-84ea-cb8d42406d8f\") " pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:07:43.106073 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:43.105995 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 22 15:07:43.106073 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:43.106014 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 22 15:07:43.106073 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:43.106024 2577 projected.go:194] Error preparing data for projected volume kube-api-access-w9n55 for pod openshift-network-diagnostics/network-check-target-8lfsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 22 15:07:43.106073 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:43.106074 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68a481e1-e027-433e-84ea-cb8d42406d8f-kube-api-access-w9n55 podName:68a481e1-e027-433e-84ea-cb8d42406d8f nodeName:}" failed. No retries permitted until 2026-04-22 15:07:44.106058843 +0000 UTC m=+3.162796909 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-w9n55" (UniqueName: "kubernetes.io/projected/68a481e1-e027-433e-84ea-cb8d42406d8f-kube-api-access-w9n55") pod "network-check-target-8lfsw" (UID: "68a481e1-e027-433e-84ea-cb8d42406d8f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 22 15:07:43.471067 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:43.470970 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-04-21 15:02:42 +0000 UTC" deadline="2027-10-02 05:21:32.339575999 +0000 UTC" Apr 22 15:07:43.471067 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:43.471015 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12662h13m48.868565119s" Apr 22 15:07:43.505015 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:43.504937 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:07:43.505187 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:43.505073 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z9ckr" podUID="487f23b6-9470-41e9-abbd-295b6d317b10" Apr 22 15:07:43.505525 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:43.505503 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:07:43.505626 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:43.505597 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lfsw" podUID="68a481e1-e027-433e-84ea-cb8d42406d8f" Apr 22 15:07:43.515544 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:43.515497 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g8wmb" event={"ID":"9dc3cd64-355a-4b4b-a5e1-5d165978d8a3","Type":"ContainerStarted","Data":"552b722409d5759ba34bfb41d9e5b8eeb0d2f740d2e24df37899f84576cb4e78"} Apr 22 15:07:43.523629 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:43.523594 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" event={"ID":"0510159c-4726-4872-8eb0-5d435faa0b32","Type":"ContainerStarted","Data":"8e4811109892954d7d71f1432aec8db0d88565025d06b86f0c04b7935cf3943a"} Apr 22 15:07:43.527975 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:43.527886 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h5xww" event={"ID":"0c332b2a-7ac7-4b26-b9d4-272932e01b8e","Type":"ContainerStarted","Data":"5811852c0f3884b9e0234624be68e1050a6df36b56245b76cc9533ad0225ad7b"} Apr 22 15:07:43.534146 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:43.534109 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" event={"ID":"be504427-e741-4041-98dc-92cc1db1293a","Type":"ContainerStarted","Data":"83b1e520c47d269c7483ff9dba4979a027060654c9ffa1923f427e566763b5fa"} Apr 22 15:07:43.539912 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:43.539864 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2jf6l" event={"ID":"6b9559b7-6b5b-4ecf-9655-4379447b15c7","Type":"ContainerStarted","Data":"139f35dbe1f6e6c5cb1d083c5cbc10739868ed515eaf0d3c7f15591bb0d0f97f"} Apr 22 15:07:43.542918 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:43.542864 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rrrjp" event={"ID":"727c0127-0608-49d1-a2a3-aa2d700b9898","Type":"ContainerStarted","Data":"34aab2373d16659183ec72f516aed2bb11e4dd295a9669f3bdb663c9d87116e6"} Apr 22 15:07:43.544594 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:43.544539 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mzm96" event={"ID":"ab8e8378-2063-4f8e-bb2a-cc3cd4ca56d1","Type":"ContainerStarted","Data":"636b60fb5ea7a7e34500d145a561469a142da1873b231a265ebc7eefeed4670e"} Apr 22 15:07:43.545958 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:43.545913 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" event={"ID":"bdcb6dd7-b076-455a-8529-a6c7a1eeae89","Type":"ContainerStarted","Data":"cb5c39b15d0f2d56c2b93aaf6724075943acf5ea9f35799f987b31ed6148a38d"} Apr 22 15:07:43.825001 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:43.824958 2577 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Apr 22 15:07:44.014274 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:44.014233 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/487f23b6-9470-41e9-abbd-295b6d317b10-metrics-certs\") pod \"network-metrics-daemon-z9ckr\" (UID: \"487f23b6-9470-41e9-abbd-295b6d317b10\") " pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:07:44.014500 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:44.014456 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 15:07:44.014573 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:44.014551 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/487f23b6-9470-41e9-abbd-295b6d317b10-metrics-certs podName:487f23b6-9470-41e9-abbd-295b6d317b10 nodeName:}" failed. No retries permitted until 2026-04-22 15:07:46.014529932 +0000 UTC m=+5.071268015 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/487f23b6-9470-41e9-abbd-295b6d317b10-metrics-certs") pod "network-metrics-daemon-z9ckr" (UID: "487f23b6-9470-41e9-abbd-295b6d317b10") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 15:07:44.115692 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:44.115607 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w9n55\" (UniqueName: \"kubernetes.io/projected/68a481e1-e027-433e-84ea-cb8d42406d8f-kube-api-access-w9n55\") pod \"network-check-target-8lfsw\" (UID: \"68a481e1-e027-433e-84ea-cb8d42406d8f\") " pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:07:44.115857 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:44.115781 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 22 15:07:44.115857 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:44.115798 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 22 15:07:44.115857 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:44.115811 2577 projected.go:194] Error preparing data for projected volume kube-api-access-w9n55 for pod openshift-network-diagnostics/network-check-target-8lfsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 22 15:07:44.116006 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:44.115908 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68a481e1-e027-433e-84ea-cb8d42406d8f-kube-api-access-w9n55 podName:68a481e1-e027-433e-84ea-cb8d42406d8f nodeName:}" failed. No retries permitted until 2026-04-22 15:07:46.115852561 +0000 UTC m=+5.172590638 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-w9n55" (UniqueName: "kubernetes.io/projected/68a481e1-e027-433e-84ea-cb8d42406d8f-kube-api-access-w9n55") pod "network-check-target-8lfsw" (UID: "68a481e1-e027-433e-84ea-cb8d42406d8f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 22 15:07:44.193901 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:44.193871 2577 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Apr 22 15:07:44.472023 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:44.471899 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-04-21 15:02:42 +0000 UTC" deadline="2027-11-11 06:43:49.925533792 +0000 UTC" Apr 22 15:07:44.472023 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:44.471938 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13623h36m5.453600005s" Apr 22 15:07:45.502978 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:45.502947 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:07:45.503446 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:45.502954 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:07:45.503446 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:45.503083 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lfsw" podUID="68a481e1-e027-433e-84ea-cb8d42406d8f" Apr 22 15:07:45.503446 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:45.503205 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z9ckr" podUID="487f23b6-9470-41e9-abbd-295b6d317b10" Apr 22 15:07:46.032504 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:46.032391 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/487f23b6-9470-41e9-abbd-295b6d317b10-metrics-certs\") pod \"network-metrics-daemon-z9ckr\" (UID: \"487f23b6-9470-41e9-abbd-295b6d317b10\") " pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:07:46.032677 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:46.032560 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 15:07:46.032677 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:46.032628 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/487f23b6-9470-41e9-abbd-295b6d317b10-metrics-certs podName:487f23b6-9470-41e9-abbd-295b6d317b10 nodeName:}" failed. No retries permitted until 2026-04-22 15:07:50.032609281 +0000 UTC m=+9.089347351 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/487f23b6-9470-41e9-abbd-295b6d317b10-metrics-certs") pod "network-metrics-daemon-z9ckr" (UID: "487f23b6-9470-41e9-abbd-295b6d317b10") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 15:07:46.133393 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:46.133356 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w9n55\" (UniqueName: \"kubernetes.io/projected/68a481e1-e027-433e-84ea-cb8d42406d8f-kube-api-access-w9n55\") pod \"network-check-target-8lfsw\" (UID: \"68a481e1-e027-433e-84ea-cb8d42406d8f\") " pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:07:46.133580 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:46.133513 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 22 15:07:46.133580 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:46.133535 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 22 15:07:46.133580 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:46.133550 2577 projected.go:194] Error preparing data for projected volume kube-api-access-w9n55 for pod openshift-network-diagnostics/network-check-target-8lfsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 22 15:07:46.133740 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:46.133619 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68a481e1-e027-433e-84ea-cb8d42406d8f-kube-api-access-w9n55 podName:68a481e1-e027-433e-84ea-cb8d42406d8f nodeName:}" failed. No retries permitted until 2026-04-22 15:07:50.133598639 +0000 UTC m=+9.190336702 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-w9n55" (UniqueName: "kubernetes.io/projected/68a481e1-e027-433e-84ea-cb8d42406d8f-kube-api-access-w9n55") pod "network-check-target-8lfsw" (UID: "68a481e1-e027-433e-84ea-cb8d42406d8f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 22 15:07:47.502813 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:47.502781 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:07:47.503265 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:47.502966 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:07:47.503265 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:47.502996 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lfsw" podUID="68a481e1-e027-433e-84ea-cb8d42406d8f" Apr 22 15:07:47.503265 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:47.503144 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z9ckr" podUID="487f23b6-9470-41e9-abbd-295b6d317b10" Apr 22 15:07:49.505550 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:49.505516 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:07:49.506015 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:49.505631 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lfsw" podUID="68a481e1-e027-433e-84ea-cb8d42406d8f" Apr 22 15:07:49.506015 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:49.505516 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:07:49.506230 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:49.506189 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z9ckr" podUID="487f23b6-9470-41e9-abbd-295b6d317b10" Apr 22 15:07:50.067373 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:50.067311 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/487f23b6-9470-41e9-abbd-295b6d317b10-metrics-certs\") pod \"network-metrics-daemon-z9ckr\" (UID: \"487f23b6-9470-41e9-abbd-295b6d317b10\") " pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:07:50.067580 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:50.067510 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 15:07:50.067580 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:50.067576 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/487f23b6-9470-41e9-abbd-295b6d317b10-metrics-certs podName:487f23b6-9470-41e9-abbd-295b6d317b10 nodeName:}" failed. No retries permitted until 2026-04-22 15:07:58.067556579 +0000 UTC m=+17.124294642 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/487f23b6-9470-41e9-abbd-295b6d317b10-metrics-certs") pod "network-metrics-daemon-z9ckr" (UID: "487f23b6-9470-41e9-abbd-295b6d317b10") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 15:07:50.168372 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:50.168270 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w9n55\" (UniqueName: \"kubernetes.io/projected/68a481e1-e027-433e-84ea-cb8d42406d8f-kube-api-access-w9n55\") pod \"network-check-target-8lfsw\" (UID: \"68a481e1-e027-433e-84ea-cb8d42406d8f\") " pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:07:50.168547 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:50.168460 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 22 15:07:50.168547 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:50.168481 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 22 15:07:50.168547 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:50.168495 2577 projected.go:194] Error preparing data for projected volume kube-api-access-w9n55 for pod openshift-network-diagnostics/network-check-target-8lfsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 22 15:07:50.168720 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:50.168555 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68a481e1-e027-433e-84ea-cb8d42406d8f-kube-api-access-w9n55 podName:68a481e1-e027-433e-84ea-cb8d42406d8f nodeName:}" failed. No retries permitted until 2026-04-22 15:07:58.168536662 +0000 UTC m=+17.225274727 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-w9n55" (UniqueName: "kubernetes.io/projected/68a481e1-e027-433e-84ea-cb8d42406d8f-kube-api-access-w9n55") pod "network-check-target-8lfsw" (UID: "68a481e1-e027-433e-84ea-cb8d42406d8f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 22 15:07:51.505706 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:51.505672 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:07:51.506148 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:51.505783 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z9ckr" podUID="487f23b6-9470-41e9-abbd-295b6d317b10" Apr 22 15:07:51.506148 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:51.506078 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:07:51.506148 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:51.506135 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lfsw" podUID="68a481e1-e027-433e-84ea-cb8d42406d8f" Apr 22 15:07:53.502715 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:53.502674 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:07:53.503139 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:53.502802 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lfsw" podUID="68a481e1-e027-433e-84ea-cb8d42406d8f" Apr 22 15:07:53.503139 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:53.502865 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:07:53.503139 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:53.502987 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z9ckr" podUID="487f23b6-9470-41e9-abbd-295b6d317b10" Apr 22 15:07:55.502988 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:55.502949 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:07:55.502988 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:55.502960 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:07:55.503508 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:55.503086 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z9ckr" podUID="487f23b6-9470-41e9-abbd-295b6d317b10" Apr 22 15:07:55.503508 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:55.503231 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lfsw" podUID="68a481e1-e027-433e-84ea-cb8d42406d8f" Apr 22 15:07:57.503157 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:57.503119 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:07:57.503583 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:57.503173 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:07:57.503583 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:57.503266 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z9ckr" podUID="487f23b6-9470-41e9-abbd-295b6d317b10" Apr 22 15:07:57.503583 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:57.503405 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lfsw" podUID="68a481e1-e027-433e-84ea-cb8d42406d8f" Apr 22 15:07:58.124027 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:58.123996 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/487f23b6-9470-41e9-abbd-295b6d317b10-metrics-certs\") pod \"network-metrics-daemon-z9ckr\" (UID: \"487f23b6-9470-41e9-abbd-295b6d317b10\") " pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:07:58.124223 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:58.124133 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 15:07:58.124223 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:58.124196 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/487f23b6-9470-41e9-abbd-295b6d317b10-metrics-certs podName:487f23b6-9470-41e9-abbd-295b6d317b10 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:14.124174548 +0000 UTC m=+33.180912613 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/487f23b6-9470-41e9-abbd-295b6d317b10-metrics-certs") pod "network-metrics-daemon-z9ckr" (UID: "487f23b6-9470-41e9-abbd-295b6d317b10") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 15:07:58.224461 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:58.224430 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w9n55\" (UniqueName: \"kubernetes.io/projected/68a481e1-e027-433e-84ea-cb8d42406d8f-kube-api-access-w9n55\") pod \"network-check-target-8lfsw\" (UID: \"68a481e1-e027-433e-84ea-cb8d42406d8f\") " pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:07:58.224628 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:58.224575 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 22 15:07:58.224628 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:58.224596 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 22 15:07:58.224628 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:58.224605 2577 projected.go:194] Error preparing data for projected volume kube-api-access-w9n55 for pod openshift-network-diagnostics/network-check-target-8lfsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 22 15:07:58.224760 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:58.224668 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68a481e1-e027-433e-84ea-cb8d42406d8f-kube-api-access-w9n55 podName:68a481e1-e027-433e-84ea-cb8d42406d8f nodeName:}" failed. No retries permitted until 2026-04-22 15:08:14.22464984 +0000 UTC m=+33.281387906 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-w9n55" (UniqueName: "kubernetes.io/projected/68a481e1-e027-433e-84ea-cb8d42406d8f-kube-api-access-w9n55") pod "network-check-target-8lfsw" (UID: "68a481e1-e027-433e-84ea-cb8d42406d8f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 22 15:07:59.502541 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:59.502500 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:07:59.502986 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:07:59.502510 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:07:59.502986 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:59.502622 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lfsw" podUID="68a481e1-e027-433e-84ea-cb8d42406d8f" Apr 22 15:07:59.502986 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:07:59.502711 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z9ckr" podUID="487f23b6-9470-41e9-abbd-295b6d317b10" Apr 22 15:08:01.504300 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:01.503874 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:08:01.505068 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:01.503918 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:08:01.505068 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:01.504446 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z9ckr" podUID="487f23b6-9470-41e9-abbd-295b6d317b10" Apr 22 15:08:01.505068 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:01.504541 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lfsw" podUID="68a481e1-e027-433e-84ea-cb8d42406d8f" Apr 22 15:08:01.579844 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:01.579785 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-168.ec2.internal" event={"ID":"ccd5fe942c1851cfa510f628904b2dc3","Type":"ContainerStarted","Data":"9f645a83da48806c9293ad47685cea3ba37f537392dfcc5906530d40cd88df3c"} Apr 22 15:08:01.581303 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:01.581277 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h5xww" event={"ID":"0c332b2a-7ac7-4b26-b9d4-272932e01b8e","Type":"ContainerStarted","Data":"9ecb560fa5acd92ee6ddb5319cb07459099384716590d64ab88612a4b8fb78df"} Apr 22 15:08:01.582592 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:01.582570 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" event={"ID":"be504427-e741-4041-98dc-92cc1db1293a","Type":"ContainerStarted","Data":"bd7ce4eb734706136351404b872b655757f17256fc2baf5218838041229a1d9f"} Apr 22 15:08:01.584356 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:01.584317 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2jf6l" event={"ID":"6b9559b7-6b5b-4ecf-9655-4379447b15c7","Type":"ContainerStarted","Data":"9817c0b93545292611f5c63acd272f54fbe66f28110275985af685d8d0942a9a"} Apr 22 15:08:01.586017 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:01.585990 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mzm96" event={"ID":"ab8e8378-2063-4f8e-bb2a-cc3cd4ca56d1","Type":"ContainerStarted","Data":"e69cc3bbfac9a06b3a4881be34633d158ec946e0c67ee63944c505cf361e7d7c"} Apr 22 15:08:01.587616 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:01.587577 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-168.ec2.internal" event={"ID":"5e64b9b4046044f766b93ef1fd3f3a00","Type":"ContainerStarted","Data":"4dd7963dc053e3d0462158295665f79b93afac2fdebbc26d5a8ae2c203a7600a"} Apr 22 15:08:01.589962 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:01.589938 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" event={"ID":"bdcb6dd7-b076-455a-8529-a6c7a1eeae89","Type":"ContainerStarted","Data":"c858898060b2ce735979bd08dfd85da62725ee66cb39a637e2e8636095bb1018"} Apr 22 15:08:01.590067 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:01.589964 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" event={"ID":"bdcb6dd7-b076-455a-8529-a6c7a1eeae89","Type":"ContainerStarted","Data":"551fea39e59d98d6863316e477f83ca08e09867b507e2de6178d6a082bf67055"} Apr 22 15:08:01.590067 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:01.589984 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" event={"ID":"bdcb6dd7-b076-455a-8529-a6c7a1eeae89","Type":"ContainerStarted","Data":"1008f611c7f514a96443d6ca8d1d3c464c7760e666bd80aba86eac9946b95d22"} Apr 22 15:08:01.590067 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:01.589995 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" event={"ID":"bdcb6dd7-b076-455a-8529-a6c7a1eeae89","Type":"ContainerStarted","Data":"84ce54114011aa9143d5567ef7a91cac0d1ff1d0b9b78fabe5eb364b0f6165cd"} Apr 22 15:08:01.590067 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:01.590006 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" event={"ID":"bdcb6dd7-b076-455a-8529-a6c7a1eeae89","Type":"ContainerStarted","Data":"641df807be8c80b3731ba77a1d9fd10c9f0edcf24fe412df174592e36e212d96"} Apr 22 15:08:01.590067 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:01.590016 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" event={"ID":"bdcb6dd7-b076-455a-8529-a6c7a1eeae89","Type":"ContainerStarted","Data":"ba289d7daccde0c73c7319d0987982604dc06708b5434dde3db5fb49bf440edb"} Apr 22 15:08:01.591294 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:01.591270 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g8wmb" event={"ID":"9dc3cd64-355a-4b4b-a5e1-5d165978d8a3","Type":"ContainerStarted","Data":"8c49751ee8b0b9676596b814cc18ca5896fd6d58f783e888f05881d4f19631a7"} Apr 22 15:08:01.592828 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:01.592807 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" event={"ID":"0510159c-4726-4872-8eb0-5d435faa0b32","Type":"ContainerStarted","Data":"1e2348a7866226de4f81a8b02f47843a98f417fce4e099d810aeae2dd2b1f9bc"} Apr 22 15:08:01.613262 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:01.613225 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-168.ec2.internal" podStartSLOduration=19.613214393 podStartE2EDuration="19.613214393s" podCreationTimestamp="2026-04-22 15:07:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-22 15:08:01.613004747 +0000 UTC m=+20.669742831" watchObservedRunningTime="2026-04-22 15:08:01.613214393 +0000 UTC m=+20.669952480" Apr 22 15:08:01.641503 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:01.641465 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mzm96" podStartSLOduration=2.898900979 podStartE2EDuration="20.641449101s" podCreationTimestamp="2026-04-22 15:07:41 +0000 UTC" firstStartedPulling="2026-04-22 15:07:42.733998166 +0000 UTC m=+1.790736232" lastFinishedPulling="2026-04-22 15:08:00.476546288 +0000 UTC m=+19.533284354" observedRunningTime="2026-04-22 15:08:01.627242716 +0000 UTC m=+20.683980812" watchObservedRunningTime="2026-04-22 15:08:01.641449101 +0000 UTC m=+20.698187179" Apr 22 15:08:01.641637 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:01.641618 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-g8wmb" podStartSLOduration=2.913876754 podStartE2EDuration="20.641614055s" podCreationTimestamp="2026-04-22 15:07:41 +0000 UTC" firstStartedPulling="2026-04-22 15:07:42.748570457 +0000 UTC m=+1.805308520" lastFinishedPulling="2026-04-22 15:08:00.476307749 +0000 UTC m=+19.533045821" observedRunningTime="2026-04-22 15:08:01.641357933 +0000 UTC m=+20.698096015" watchObservedRunningTime="2026-04-22 15:08:01.641614055 +0000 UTC m=+20.698352140" Apr 22 15:08:01.676300 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:01.676256 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hz7vw" podStartSLOduration=2.930628916 podStartE2EDuration="20.676242268s" podCreationTimestamp="2026-04-22 15:07:41 +0000 UTC" firstStartedPulling="2026-04-22 15:07:42.763482824 +0000 UTC m=+1.820220887" lastFinishedPulling="2026-04-22 15:08:00.509096172 +0000 UTC m=+19.565834239" observedRunningTime="2026-04-22 15:08:01.675737835 +0000 UTC m=+20.732475932" watchObservedRunningTime="2026-04-22 15:08:01.676242268 +0000 UTC m=+20.732980353" Apr 22 15:08:01.692308 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:01.692271 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-h5xww" podStartSLOduration=2.953223243 podStartE2EDuration="20.692257604s" podCreationTimestamp="2026-04-22 15:07:41 +0000 UTC" firstStartedPulling="2026-04-22 15:07:42.768702738 +0000 UTC m=+1.825440801" lastFinishedPulling="2026-04-22 15:08:00.507737092 +0000 UTC m=+19.564475162" observedRunningTime="2026-04-22 15:08:01.692055956 +0000 UTC m=+20.748794040" watchObservedRunningTime="2026-04-22 15:08:01.692257604 +0000 UTC m=+20.748995688" Apr 22 15:08:02.595825 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:02.595793 2577 generic.go:358] "Generic (PLEG): container finished" podID="ccd5fe942c1851cfa510f628904b2dc3" containerID="9f645a83da48806c9293ad47685cea3ba37f537392dfcc5906530d40cd88df3c" exitCode=0 Apr 22 15:08:02.596264 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:02.596120 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-168.ec2.internal" event={"ID":"ccd5fe942c1851cfa510f628904b2dc3","Type":"ContainerDied","Data":"9f645a83da48806c9293ad47685cea3ba37f537392dfcc5906530d40cd88df3c"} Apr 22 15:08:02.598766 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:02.598191 2577 generic.go:358] "Generic (PLEG): container finished" podID="6b9559b7-6b5b-4ecf-9655-4379447b15c7" containerID="9817c0b93545292611f5c63acd272f54fbe66f28110275985af685d8d0942a9a" exitCode=0 Apr 22 15:08:02.598766 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:02.598259 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2jf6l" event={"ID":"6b9559b7-6b5b-4ecf-9655-4379447b15c7","Type":"ContainerDied","Data":"9817c0b93545292611f5c63acd272f54fbe66f28110275985af685d8d0942a9a"} Apr 22 15:08:02.601235 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:02.601209 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rrrjp" event={"ID":"727c0127-0608-49d1-a2a3-aa2d700b9898","Type":"ContainerStarted","Data":"9f0ec54747b7618a823f22695a4d200cb0fe29acaf6836389346936a2ec6477e"} Apr 22 15:08:02.624983 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:02.624921 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rrrjp" podStartSLOduration=3.901503219 podStartE2EDuration="21.624902063s" podCreationTimestamp="2026-04-22 15:07:41 +0000 UTC" firstStartedPulling="2026-04-22 15:07:42.753181806 +0000 UTC m=+1.809919868" lastFinishedPulling="2026-04-22 15:08:00.476580638 +0000 UTC m=+19.533318712" observedRunningTime="2026-04-22 15:08:02.624475325 +0000 UTC m=+21.681213410" watchObservedRunningTime="2026-04-22 15:08:02.624902063 +0000 UTC m=+21.681640149" Apr 22 15:08:02.681638 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:02.681612 2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Apr 22 15:08:03.471531 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:03.471399 2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-04-22T15:08:02.681635559Z","UUID":"b4507416-d93b-4b60-bec5-1f00bd9be8e4","Handler":null,"Name":"","Endpoint":""} Apr 22 15:08:03.473397 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:03.473368 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Apr 22 15:08:03.473397 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:03.473401 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Apr 22 15:08:03.503081 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:03.503053 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:08:03.503173 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:03.503155 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z9ckr" podUID="487f23b6-9470-41e9-abbd-295b6d317b10" Apr 22 15:08:03.503234 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:03.503216 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:08:03.503351 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:03.503318 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lfsw" podUID="68a481e1-e027-433e-84ea-cb8d42406d8f" Apr 22 15:08:03.607275 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:03.607032 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" event={"ID":"bdcb6dd7-b076-455a-8529-a6c7a1eeae89","Type":"ContainerStarted","Data":"ffd11b18d9267be335490cfd9acfcab385b2523fa0870f2aecd862d965e683d1"} Apr 22 15:08:03.609846 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:03.609812 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" event={"ID":"0510159c-4726-4872-8eb0-5d435faa0b32","Type":"ContainerStarted","Data":"d0d6b0f61497042671ee7f9ac1fee0aaab4881475461dd481ae4feea7fafa487"} Apr 22 15:08:03.611835 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:03.611789 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-168.ec2.internal" event={"ID":"ccd5fe942c1851cfa510f628904b2dc3","Type":"ContainerStarted","Data":"48c5023fd995db5df39fc1f89b6c36a1134331d0f42093d107935b23c9e50c07"} Apr 22 15:08:04.616342 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:04.616287 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" event={"ID":"0510159c-4726-4872-8eb0-5d435faa0b32","Type":"ContainerStarted","Data":"71984fd6627009c297dbe69e9ca808e559adba0752c83d45e9ef7441d89e0e63"} Apr 22 15:08:04.634953 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:04.634898 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-168.ec2.internal" podStartSLOduration=22.63487964 podStartE2EDuration="22.63487964s" podCreationTimestamp="2026-04-22 15:07:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-22 15:08:03.67114877 +0000 UTC m=+22.727886855" watchObservedRunningTime="2026-04-22 15:08:04.63487964 +0000 UTC m=+23.691617717" Apr 22 15:08:04.635433 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:04.635398 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmfs2" podStartSLOduration=2.837142092 podStartE2EDuration="23.635388624s" podCreationTimestamp="2026-04-22 15:07:41 +0000 UTC" firstStartedPulling="2026-04-22 15:07:42.739249308 +0000 UTC m=+1.795987374" lastFinishedPulling="2026-04-22 15:08:03.537495828 +0000 UTC m=+22.594233906" observedRunningTime="2026-04-22 15:08:04.634999344 +0000 UTC m=+23.691737429" watchObservedRunningTime="2026-04-22 15:08:04.635388624 +0000 UTC m=+23.692126711" Apr 22 15:08:04.783658 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:04.783624 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mzm96" Apr 22 15:08:04.784676 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:04.784654 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mzm96" Apr 22 15:08:04.866069 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:04.866039 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-xlxs5"] Apr 22 15:08:04.868815 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:04.868758 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xlxs5" Apr 22 15:08:04.872154 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:04.872131 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Apr 22 15:08:04.872274 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:04.872151 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Apr 22 15:08:04.872274 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:04.872256 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vrq7v\"" Apr 22 15:08:04.969225 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:04.969186 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n95c7\" (UniqueName: \"kubernetes.io/projected/74c26902-8853-4def-9827-b429513b3de8-kube-api-access-n95c7\") pod \"node-resolver-xlxs5\" (UID: \"74c26902-8853-4def-9827-b429513b3de8\") " pod="openshift-dns/node-resolver-xlxs5" Apr 22 15:08:04.969436 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:04.969243 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74c26902-8853-4def-9827-b429513b3de8-tmp-dir\") pod \"node-resolver-xlxs5\" (UID: \"74c26902-8853-4def-9827-b429513b3de8\") " pod="openshift-dns/node-resolver-xlxs5" Apr 22 15:08:04.969436 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:04.969375 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/74c26902-8853-4def-9827-b429513b3de8-hosts-file\") pod \"node-resolver-xlxs5\" (UID: \"74c26902-8853-4def-9827-b429513b3de8\") " pod="openshift-dns/node-resolver-xlxs5" Apr 22 15:08:05.070397 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:05.070355 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n95c7\" (UniqueName: \"kubernetes.io/projected/74c26902-8853-4def-9827-b429513b3de8-kube-api-access-n95c7\") pod \"node-resolver-xlxs5\" (UID: \"74c26902-8853-4def-9827-b429513b3de8\") " pod="openshift-dns/node-resolver-xlxs5" Apr 22 15:08:05.070580 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:05.070424 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74c26902-8853-4def-9827-b429513b3de8-tmp-dir\") pod \"node-resolver-xlxs5\" (UID: \"74c26902-8853-4def-9827-b429513b3de8\") " pod="openshift-dns/node-resolver-xlxs5" Apr 22 15:08:05.070580 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:05.070470 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/74c26902-8853-4def-9827-b429513b3de8-hosts-file\") pod \"node-resolver-xlxs5\" (UID: \"74c26902-8853-4def-9827-b429513b3de8\") " pod="openshift-dns/node-resolver-xlxs5" Apr 22 15:08:05.070580 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:05.070547 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/74c26902-8853-4def-9827-b429513b3de8-hosts-file\") pod \"node-resolver-xlxs5\" (UID: \"74c26902-8853-4def-9827-b429513b3de8\") " pod="openshift-dns/node-resolver-xlxs5" Apr 22 15:08:05.070824 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:05.070804 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74c26902-8853-4def-9827-b429513b3de8-tmp-dir\") pod \"node-resolver-xlxs5\" (UID: \"74c26902-8853-4def-9827-b429513b3de8\") " pod="openshift-dns/node-resolver-xlxs5" Apr 22 15:08:05.081225 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:05.081201 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n95c7\" (UniqueName: \"kubernetes.io/projected/74c26902-8853-4def-9827-b429513b3de8-kube-api-access-n95c7\") pod \"node-resolver-xlxs5\" (UID: \"74c26902-8853-4def-9827-b429513b3de8\") " pod="openshift-dns/node-resolver-xlxs5" Apr 22 15:08:05.179500 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:05.179423 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xlxs5" Apr 22 15:08:05.189035 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:08:05.188998 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74c26902_8853_4def_9827_b429513b3de8.slice/crio-c00c3ffd4049015b58482bcbecbc1920cca496b4e91095169b0b88263afb700d WatchSource:0}: Error finding container c00c3ffd4049015b58482bcbecbc1920cca496b4e91095169b0b88263afb700d: Status 404 returned error can't find the container with id c00c3ffd4049015b58482bcbecbc1920cca496b4e91095169b0b88263afb700d Apr 22 15:08:05.502590 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:05.502549 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:08:05.502758 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:05.502681 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z9ckr" podUID="487f23b6-9470-41e9-abbd-295b6d317b10" Apr 22 15:08:05.502758 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:05.502747 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:08:05.502878 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:05.502857 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lfsw" podUID="68a481e1-e027-433e-84ea-cb8d42406d8f" Apr 22 15:08:05.620182 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:05.619957 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xlxs5" event={"ID":"74c26902-8853-4def-9827-b429513b3de8","Type":"ContainerStarted","Data":"c4cf2d95ef1ce952e1237978326e7674b6c6c3e93a1080345e5e43df5f4c820c"} Apr 22 15:08:05.620911 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:05.620207 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xlxs5" event={"ID":"74c26902-8853-4def-9827-b429513b3de8","Type":"ContainerStarted","Data":"c00c3ffd4049015b58482bcbecbc1920cca496b4e91095169b0b88263afb700d"} Apr 22 15:08:05.620911 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:05.620444 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mzm96" Apr 22 15:08:05.621044 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:05.620933 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mzm96" Apr 22 15:08:05.640560 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:05.640507 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-xlxs5" podStartSLOduration=1.6404916630000002 podStartE2EDuration="1.640491663s" podCreationTimestamp="2026-04-22 15:08:04 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-22 15:08:05.64000022 +0000 UTC m=+24.696738308" watchObservedRunningTime="2026-04-22 15:08:05.640491663 +0000 UTC m=+24.697229747" Apr 22 15:08:06.625713 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:06.625672 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" event={"ID":"bdcb6dd7-b076-455a-8529-a6c7a1eeae89","Type":"ContainerStarted","Data":"343f0f1027fe1a41dc18480c7487a7946d966fea53e32a17ed6569050ad67bc0"} Apr 22 15:08:06.626285 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:06.626174 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:08:06.626285 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:06.626197 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:08:06.642502 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:06.642478 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:08:06.653211 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:06.653158 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" podStartSLOduration=7.483175125 podStartE2EDuration="25.653140927s" podCreationTimestamp="2026-04-22 15:07:41 +0000 UTC" firstStartedPulling="2026-04-22 15:07:42.773605006 +0000 UTC m=+1.830343072" lastFinishedPulling="2026-04-22 15:08:00.943570791 +0000 UTC m=+20.000308874" observedRunningTime="2026-04-22 15:08:06.652587016 +0000 UTC m=+25.709325102" watchObservedRunningTime="2026-04-22 15:08:06.653140927 +0000 UTC m=+25.709879012" Apr 22 15:08:07.502991 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:07.502947 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:08:07.503180 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:07.502947 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:08:07.503180 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:07.503086 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lfsw" podUID="68a481e1-e027-433e-84ea-cb8d42406d8f" Apr 22 15:08:07.503180 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:07.503137 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z9ckr" podUID="487f23b6-9470-41e9-abbd-295b6d317b10" Apr 22 15:08:07.627810 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:07.627777 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:08:07.641596 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:07.641573 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:08:07.847340 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:07.847235 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-z9ckr"] Apr 22 15:08:07.847489 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:07.847354 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:08:07.847489 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:07.847452 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z9ckr" podUID="487f23b6-9470-41e9-abbd-295b6d317b10" Apr 22 15:08:07.850311 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:07.850288 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8lfsw"] Apr 22 15:08:07.850423 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:07.850382 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:08:07.850472 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:07.850458 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lfsw" podUID="68a481e1-e027-433e-84ea-cb8d42406d8f" Apr 22 15:08:08.630608 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:08.630576 2577 generic.go:358] "Generic (PLEG): container finished" podID="6b9559b7-6b5b-4ecf-9655-4379447b15c7" containerID="7adfe27265e6ae9a4b0fe0aa296ddc7cd64e49ee77405640ac61419e27abc11c" exitCode=0 Apr 22 15:08:08.630990 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:08.630656 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2jf6l" event={"ID":"6b9559b7-6b5b-4ecf-9655-4379447b15c7","Type":"ContainerDied","Data":"7adfe27265e6ae9a4b0fe0aa296ddc7cd64e49ee77405640ac61419e27abc11c"} Apr 22 15:08:09.502729 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:09.502696 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:08:09.502893 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:09.502826 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lfsw" podUID="68a481e1-e027-433e-84ea-cb8d42406d8f" Apr 22 15:08:09.502971 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:09.502886 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:08:09.503037 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:09.503007 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z9ckr" podUID="487f23b6-9470-41e9-abbd-295b6d317b10" Apr 22 15:08:10.637719 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:10.637486 2577 generic.go:358] "Generic (PLEG): container finished" podID="6b9559b7-6b5b-4ecf-9655-4379447b15c7" containerID="3b9937cbe40d15ac0d0ce9e51455b1e40705ab46f052b99ef0d771b0dd7a15f0" exitCode=0 Apr 22 15:08:10.637719 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:10.637567 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2jf6l" event={"ID":"6b9559b7-6b5b-4ecf-9655-4379447b15c7","Type":"ContainerDied","Data":"3b9937cbe40d15ac0d0ce9e51455b1e40705ab46f052b99ef0d771b0dd7a15f0"} Apr 22 15:08:11.503240 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:11.503213 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:08:11.503460 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:11.503322 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lfsw" podUID="68a481e1-e027-433e-84ea-cb8d42406d8f" Apr 22 15:08:11.503460 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:11.503376 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:08:11.503568 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:11.503477 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z9ckr" podUID="487f23b6-9470-41e9-abbd-295b6d317b10" Apr 22 15:08:12.645539 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:12.645507 2577 generic.go:358] "Generic (PLEG): container finished" podID="6b9559b7-6b5b-4ecf-9655-4379447b15c7" containerID="c3775c2dd5287613277b3638121ae206e5eedc570b8451f03d6c99edb48bb324" exitCode=0 Apr 22 15:08:12.645893 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:12.645563 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2jf6l" event={"ID":"6b9559b7-6b5b-4ecf-9655-4379447b15c7","Type":"ContainerDied","Data":"c3775c2dd5287613277b3638121ae206e5eedc570b8451f03d6c99edb48bb324"} Apr 22 15:08:13.502950 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.502919 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:08:13.503146 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.502970 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:08:13.503146 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:13.503055 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z9ckr" podUID="487f23b6-9470-41e9-abbd-295b6d317b10" Apr 22 15:08:13.503265 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:13.503189 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lfsw" podUID="68a481e1-e027-433e-84ea-cb8d42406d8f" Apr 22 15:08:13.762855 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.762780 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-168.ec2.internal" event="NodeReady" Apr 22 15:08:13.763446 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.762927 2577 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Apr 22 15:08:13.801839 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.801811 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8bc6dfb5-5vmjx"] Apr 22 15:08:13.803777 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.803755 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:13.808301 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.808256 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Apr 22 15:08:13.808462 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.808306 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Apr 22 15:08:13.808604 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.808583 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Apr 22 15:08:13.809607 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.809173 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jdzns\"" Apr 22 15:08:13.809607 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.809429 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-585dfdc468-rxj2p"] Apr 22 15:08:13.811418 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.811216 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-k8gnz"] Apr 22 15:08:13.811418 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.811364 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-585dfdc468-rxj2p" Apr 22 15:08:13.813466 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.813415 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-lrs9z"] Apr 22 15:08:13.814184 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.813642 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-k8gnz" Apr 22 15:08:13.815051 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.815028 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-8894fc9bd-nmq7h"] Apr 22 15:08:13.815271 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.815257 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-lrs9z" Apr 22 15:08:13.816959 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.816920 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Apr 22 15:08:13.817912 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.817857 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Apr 22 15:08:13.818487 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.818467 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Apr 22 15:08:13.819103 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.819084 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Apr 22 15:08:13.819701 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.819682 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Apr 22 15:08:13.821895 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.821871 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Apr 22 15:08:13.822065 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.822042 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Apr 22 15:08:13.822170 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.822090 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Apr 22 15:08:13.822170 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.822157 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-k56g5\"" Apr 22 15:08:13.822473 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.822455 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Apr 22 15:08:13.822552 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.822524 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-8nk57\"" Apr 22 15:08:13.822552 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.822546 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-6mk5k\"" Apr 22 15:08:13.822937 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.822916 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Apr 22 15:08:13.823015 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.822984 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-9d4b6777b-s79j7"] Apr 22 15:08:13.823210 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.823190 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-8894fc9bd-nmq7h" Apr 22 15:08:13.824912 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.824892 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-d6fc45fc5-bd6gc"] Apr 22 15:08:13.825080 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.825062 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-9d4b6777b-s79j7" Apr 22 15:08:13.826817 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.826797 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Apr 22 15:08:13.826916 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.826902 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-bd6gc" Apr 22 15:08:13.826975 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.826818 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-cb95c66f6-6ljs8"] Apr 22 15:08:13.827085 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.827068 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Apr 22 15:08:13.827140 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.827087 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-xn4s7\"" Apr 22 15:08:13.827582 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.827564 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Apr 22 15:08:13.827974 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.827952 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Apr 22 15:08:13.827974 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.827971 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Apr 22 15:08:13.828109 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.828002 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-5lpdb\"" Apr 22 15:08:13.828109 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.828077 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Apr 22 15:08:13.828876 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.828859 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-64fd688846-mkdtj"] Apr 22 15:08:13.829053 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.829033 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-cb95c66f6-6ljs8" Apr 22 15:08:13.829627 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.829607 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Apr 22 15:08:13.830583 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.830565 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-fxjmk"] Apr 22 15:08:13.830731 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.830714 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:13.831239 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.831221 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Apr 22 15:08:13.831323 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.831273 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Apr 22 15:08:13.831531 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.831516 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-fxp8g\"" Apr 22 15:08:13.831621 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.831540 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Apr 22 15:08:13.831672 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.831656 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Apr 22 15:08:13.831740 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.831724 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Apr 22 15:08:13.831740 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.831591 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-mcrp8\"" Apr 22 15:08:13.832411 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.832386 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-594fffb56b-msk5s"] Apr 22 15:08:13.832544 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.832528 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-fxjmk" Apr 22 15:08:13.832643 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.832625 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Apr 22 15:08:13.834426 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.834409 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-75587bd455-29bzc"] Apr 22 15:08:13.834565 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.834541 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:13.835714 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.835691 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Apr 22 15:08:13.836192 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.836125 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-29bzc" Apr 22 15:08:13.836510 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.836489 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Apr 22 15:08:13.838607 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.838589 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Apr 22 15:08:13.838952 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.838618 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8bc6dfb5-5vmjx"] Apr 22 15:08:13.839014 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.838781 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Apr 22 15:08:13.840999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.840983 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Apr 22 15:08:13.841214 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.841200 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-qsm6d\"" Apr 22 15:08:13.841578 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.841562 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-k8gnz"] Apr 22 15:08:13.842212 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.842186 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Apr 22 15:08:13.842212 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.842210 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Apr 22 15:08:13.842641 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.842621 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-585dfdc468-rxj2p"] Apr 22 15:08:13.843472 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.843452 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4zrz9"] Apr 22 15:08:13.843967 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.843949 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Apr 22 15:08:13.847860 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.845520 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Apr 22 15:08:13.847860 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.845765 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Apr 22 15:08:13.847860 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.846907 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-r6fws\"" Apr 22 15:08:13.847860 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.847158 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Apr 22 15:08:13.847860 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.847853 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Apr 22 15:08:13.848357 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.848319 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Apr 22 15:08:13.850023 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.850002 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-j8ksr\"" Apr 22 15:08:13.850939 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.850900 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Apr 22 15:08:13.851523 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.851488 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Apr 22 15:08:13.852885 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.852864 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4qmd5"] Apr 22 15:08:13.853612 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.853593 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4zrz9" Apr 22 15:08:13.856847 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.856826 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-d6fc45fc5-bd6gc"] Apr 22 15:08:13.856942 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.856859 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-9d4b6777b-s79j7"] Apr 22 15:08:13.856942 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.856886 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-8894fc9bd-nmq7h"] Apr 22 15:08:13.856942 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.856903 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-lrs9z"] Apr 22 15:08:13.857233 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.857146 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4qmd5" Apr 22 15:08:13.857510 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.857492 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Apr 22 15:08:13.858256 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.858204 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-594fffb56b-msk5s"] Apr 22 15:08:13.858410 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.858397 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-75587bd455-29bzc"] Apr 22 15:08:13.858861 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.858846 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Apr 22 15:08:13.860708 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.860232 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-62nkc\"" Apr 22 15:08:13.860708 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.860398 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Apr 22 15:08:13.860708 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.860495 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-cb95c66f6-6ljs8"] Apr 22 15:08:13.861237 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.861221 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xqxh9\"" Apr 22 15:08:13.861601 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.861587 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4qmd5"] Apr 22 15:08:13.861820 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.861754 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Apr 22 15:08:13.862777 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.862757 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4zrz9"] Apr 22 15:08:13.864180 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.864162 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Apr 22 15:08:13.864340 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.864263 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-fxjmk"] Apr 22 15:08:13.865853 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.865836 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64fd688846-mkdtj"] Apr 22 15:08:13.938398 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.938368 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/df8a3082-b089-41db-bbd6-1e6b09fd544b-installation-pull-secrets\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:13.938526 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.938410 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5rdv\" (UniqueName: \"kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-kube-api-access-c5rdv\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:13.938526 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.938435 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/68da1e47-8873-416c-b2d9-d515d0985631-serving-cert\") pod \"service-ca-operator-d6fc45fc5-bd6gc\" (UID: \"68da1e47-8873-416c-b2d9-d515d0985631\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-bd6gc" Apr 22 15:08:13.938526 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.938473 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/19d55ae0-1958-4a44-a653-e45e72f6e6e2-default-certificate\") pod \"router-default-594fffb56b-msk5s\" (UID: \"19d55ae0-1958-4a44-a653-e45e72f6e6e2\") " pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:13.938526 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.938491 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7b8b7c3-8b82-46c5-bb74-0c90f6448aff-tmp\") pod \"insights-operator-585dfdc468-rxj2p\" (UID: \"c7b8b7c3-8b82-46c5-bb74-0c90f6448aff\") " pod="openshift-insights/insights-operator-585dfdc468-rxj2p" Apr 22 15:08:13.938526 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.938512 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-certificates\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:13.938769 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.938571 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6cd30837-92d3-44c3-ba6a-9c84a22a7c8b-config\") pod \"kube-storage-version-migrator-operator-6769c5d45-fxjmk\" (UID: \"6cd30837-92d3-44c3-ba6a-9c84a22a7c8b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-fxjmk" Apr 22 15:08:13.938769 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.938624 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7b8b7c3-8b82-46c5-bb74-0c90f6448aff-trusted-ca-bundle\") pod \"insights-operator-585dfdc468-rxj2p\" (UID: \"c7b8b7c3-8b82-46c5-bb74-0c90f6448aff\") " pod="openshift-insights/insights-operator-585dfdc468-rxj2p" Apr 22 15:08:13.938769 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.938655 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs9d9\" (UniqueName: \"kubernetes.io/projected/c7b8b7c3-8b82-46c5-bb74-0c90f6448aff-kube-api-access-bs9d9\") pod \"insights-operator-585dfdc468-rxj2p\" (UID: \"c7b8b7c3-8b82-46c5-bb74-0c90f6448aff\") " pod="openshift-insights/insights-operator-585dfdc468-rxj2p" Apr 22 15:08:13.938769 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.938684 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19d55ae0-1958-4a44-a653-e45e72f6e6e2-metrics-certs\") pod \"router-default-594fffb56b-msk5s\" (UID: \"19d55ae0-1958-4a44-a653-e45e72f6e6e2\") " pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:13.938769 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.938709 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbd7337b-38ca-4006-bd9f-d822e5410b7c-trusted-ca\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:13.938769 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.938737 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dc334588-fb2e-4ac6-bb34-460dad264804-config-volume\") pod \"dns-default-4zrz9\" (UID: \"dc334588-fb2e-4ac6-bb34-460dad264804\") " pod="openshift-dns/dns-default-4zrz9" Apr 22 15:08:13.938769 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.938762 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv76d\" (UniqueName: \"kubernetes.io/projected/3d4eaf4b-2b28-4ad4-866f-ac541d15b00b-kube-api-access-hv76d\") pod \"network-check-source-8894fc9bd-nmq7h\" (UID: \"3d4eaf4b-2b28-4ad4-866f-ac541d15b00b\") " pod="openshift-network-diagnostics/network-check-source-8894fc9bd-nmq7h" Apr 22 15:08:13.939046 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.938786 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bbd7337b-38ca-4006-bd9f-d822e5410b7c-installation-pull-secrets\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:13.939046 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.938811 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a84180fa-5ffa-4cc6-80cc-0cd55558a345-telemetry-config\") pod \"cluster-monitoring-operator-75587bd455-29bzc\" (UID: \"a84180fa-5ffa-4cc6-80cc-0cd55558a345\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-29bzc" Apr 22 15:08:13.939046 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.938828 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwbfv\" (UniqueName: \"kubernetes.io/projected/19d55ae0-1958-4a44-a653-e45e72f6e6e2-kube-api-access-rwbfv\") pod \"router-default-594fffb56b-msk5s\" (UID: \"19d55ae0-1958-4a44-a653-e45e72f6e6e2\") " pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:13.939046 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.938905 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94b8k\" (UniqueName: \"kubernetes.io/projected/6cd30837-92d3-44c3-ba6a-9c84a22a7c8b-kube-api-access-94b8k\") pod \"kube-storage-version-migrator-operator-6769c5d45-fxjmk\" (UID: \"6cd30837-92d3-44c3-ba6a-9c84a22a7c8b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-fxjmk" Apr 22 15:08:13.939046 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.938942 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-certificates\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:13.939046 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.938970 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9zt7\" (UniqueName: \"kubernetes.io/projected/dc334588-fb2e-4ac6-bb34-460dad264804-kube-api-access-w9zt7\") pod \"dns-default-4zrz9\" (UID: \"dc334588-fb2e-4ac6-bb34-460dad264804\") " pod="openshift-dns/dns-default-4zrz9" Apr 22 15:08:13.939046 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.938997 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dcqn\" (UniqueName: \"kubernetes.io/projected/a84180fa-5ffa-4cc6-80cc-0cd55558a345-kube-api-access-2dcqn\") pod \"cluster-monitoring-operator-75587bd455-29bzc\" (UID: \"a84180fa-5ffa-4cc6-80cc-0cd55558a345\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-29bzc" Apr 22 15:08:13.939046 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939019 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/df8a3082-b089-41db-bbd6-1e6b09fd544b-trusted-ca\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:13.939046 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939034 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-bound-sa-token\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:13.939483 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939056 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hmbv\" (UniqueName: \"kubernetes.io/projected/b3b905d0-a65d-4615-8789-ccfb785189a3-kube-api-access-7hmbv\") pod \"cluster-samples-operator-6dc5bdb6b4-k8gnz\" (UID: \"b3b905d0-a65d-4615-8789-ccfb785189a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-k8gnz" Apr 22 15:08:13.939483 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939079 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c7b8b7c3-8b82-46c5-bb74-0c90f6448aff-snapshots\") pod \"insights-operator-585dfdc468-rxj2p\" (UID: \"c7b8b7c3-8b82-46c5-bb74-0c90f6448aff\") " pod="openshift-insights/insights-operator-585dfdc468-rxj2p" Apr 22 15:08:13.939483 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939125 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c7b8b7c3-8b82-46c5-bb74-0c90f6448aff-serving-cert\") pod \"insights-operator-585dfdc468-rxj2p\" (UID: \"c7b8b7c3-8b82-46c5-bb74-0c90f6448aff\") " pod="openshift-insights/insights-operator-585dfdc468-rxj2p" Apr 22 15:08:13.939483 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939141 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bbd7337b-38ca-4006-bd9f-d822e5410b7c-ca-trust-extracted\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:13.939483 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939158 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eb3d139a-27c4-4fa7-8809-a33c13551872-cert\") pod \"ingress-canary-4qmd5\" (UID: \"eb3d139a-27c4-4fa7-8809-a33c13551872\") " pod="openshift-ingress-canary/ingress-canary-4qmd5" Apr 22 15:08:13.939483 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939181 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19d55ae0-1958-4a44-a653-e45e72f6e6e2-service-ca-bundle\") pod \"router-default-594fffb56b-msk5s\" (UID: \"19d55ae0-1958-4a44-a653-e45e72f6e6e2\") " pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:13.939483 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939202 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chrhs\" (UniqueName: \"kubernetes.io/projected/4ccb076c-b0c3-40bd-b647-ef47ba1d8681-kube-api-access-chrhs\") pod \"volume-data-source-validator-7c6cbb6c87-lrs9z\" (UID: \"4ccb076c-b0c3-40bd-b647-ef47ba1d8681\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-lrs9z" Apr 22 15:08:13.939483 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939220 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvf69\" (UniqueName: \"kubernetes.io/projected/89e3aa95-bc39-4144-bc81-144160cf50eb-kube-api-access-jvf69\") pod \"console-operator-9d4b6777b-s79j7\" (UID: \"89e3aa95-bc39-4144-bc81-144160cf50eb\") " pod="openshift-console-operator/console-operator-9d4b6777b-s79j7" Apr 22 15:08:13.939483 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939237 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrzpl\" (UniqueName: \"kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-kube-api-access-jrzpl\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:13.939483 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939262 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a84180fa-5ffa-4cc6-80cc-0cd55558a345-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-75587bd455-29bzc\" (UID: \"a84180fa-5ffa-4cc6-80cc-0cd55558a345\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-29bzc" Apr 22 15:08:13.939483 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939309 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/df8a3082-b089-41db-bbd6-1e6b09fd544b-image-registry-private-configuration\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:13.939483 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939352 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/df8a3082-b089-41db-bbd6-1e6b09fd544b-ca-trust-extracted\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:13.939483 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939370 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/68da1e47-8873-416c-b2d9-d515d0985631-config\") pod \"service-ca-operator-d6fc45fc5-bd6gc\" (UID: \"68da1e47-8873-416c-b2d9-d515d0985631\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-bd6gc" Apr 22 15:08:13.939483 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939384 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/acbfa8e1-30e3-467a-b70e-c9e23ebddcbe-nginx-conf\") pod \"networking-console-plugin-cb95c66f6-6ljs8\" (UID: \"acbfa8e1-30e3-467a-b70e-c9e23ebddcbe\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-6ljs8" Apr 22 15:08:13.939483 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939403 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7b8b7c3-8b82-46c5-bb74-0c90f6448aff-service-ca-bundle\") pod \"insights-operator-585dfdc468-rxj2p\" (UID: \"c7b8b7c3-8b82-46c5-bb74-0c90f6448aff\") " pod="openshift-insights/insights-operator-585dfdc468-rxj2p" Apr 22 15:08:13.940159 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939457 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smwnd\" (UniqueName: \"kubernetes.io/projected/68da1e47-8873-416c-b2d9-d515d0985631-kube-api-access-smwnd\") pod \"service-ca-operator-d6fc45fc5-bd6gc\" (UID: \"68da1e47-8873-416c-b2d9-d515d0985631\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-bd6gc" Apr 22 15:08:13.940159 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939482 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/acbfa8e1-30e3-467a-b70e-c9e23ebddcbe-networking-console-plugin-cert\") pod \"networking-console-plugin-cb95c66f6-6ljs8\" (UID: \"acbfa8e1-30e3-467a-b70e-c9e23ebddcbe\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-6ljs8" Apr 22 15:08:13.940159 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939535 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-tls\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:13.940159 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939561 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/89e3aa95-bc39-4144-bc81-144160cf50eb-config\") pod \"console-operator-9d4b6777b-s79j7\" (UID: \"89e3aa95-bc39-4144-bc81-144160cf50eb\") " pod="openshift-console-operator/console-operator-9d4b6777b-s79j7" Apr 22 15:08:13.940159 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939588 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mx29\" (UniqueName: \"kubernetes.io/projected/eb3d139a-27c4-4fa7-8809-a33c13551872-kube-api-access-7mx29\") pod \"ingress-canary-4qmd5\" (UID: \"eb3d139a-27c4-4fa7-8809-a33c13551872\") " pod="openshift-ingress-canary/ingress-canary-4qmd5" Apr 22 15:08:13.940159 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939614 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3b905d0-a65d-4615-8789-ccfb785189a3-samples-operator-tls\") pod \"cluster-samples-operator-6dc5bdb6b4-k8gnz\" (UID: \"b3b905d0-a65d-4615-8789-ccfb785189a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-k8gnz" Apr 22 15:08:13.940159 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939639 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bbd7337b-38ca-4006-bd9f-d822e5410b7c-image-registry-private-configuration\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:13.940159 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939659 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/89e3aa95-bc39-4144-bc81-144160cf50eb-serving-cert\") pod \"console-operator-9d4b6777b-s79j7\" (UID: \"89e3aa95-bc39-4144-bc81-144160cf50eb\") " pod="openshift-console-operator/console-operator-9d4b6777b-s79j7" Apr 22 15:08:13.940159 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939696 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-bound-sa-token\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:13.940159 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939734 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc334588-fb2e-4ac6-bb34-460dad264804-metrics-tls\") pod \"dns-default-4zrz9\" (UID: \"dc334588-fb2e-4ac6-bb34-460dad264804\") " pod="openshift-dns/dns-default-4zrz9" Apr 22 15:08:13.940159 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939751 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6cd30837-92d3-44c3-ba6a-9c84a22a7c8b-serving-cert\") pod \"kube-storage-version-migrator-operator-6769c5d45-fxjmk\" (UID: \"6cd30837-92d3-44c3-ba6a-9c84a22a7c8b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-fxjmk" Apr 22 15:08:13.940159 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939767 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc334588-fb2e-4ac6-bb34-460dad264804-tmp-dir\") pod \"dns-default-4zrz9\" (UID: \"dc334588-fb2e-4ac6-bb34-460dad264804\") " pod="openshift-dns/dns-default-4zrz9" Apr 22 15:08:13.940159 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939793 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/89e3aa95-bc39-4144-bc81-144160cf50eb-trusted-ca\") pod \"console-operator-9d4b6777b-s79j7\" (UID: \"89e3aa95-bc39-4144-bc81-144160cf50eb\") " pod="openshift-console-operator/console-operator-9d4b6777b-s79j7" Apr 22 15:08:13.940159 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939817 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/19d55ae0-1958-4a44-a653-e45e72f6e6e2-stats-auth\") pod \"router-default-594fffb56b-msk5s\" (UID: \"19d55ae0-1958-4a44-a653-e45e72f6e6e2\") " pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:13.940159 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:13.939840 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-tls\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:14.040566 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.040485 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/df8a3082-b089-41db-bbd6-1e6b09fd544b-image-registry-private-configuration\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:14.040566 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.040536 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/df8a3082-b089-41db-bbd6-1e6b09fd544b-ca-trust-extracted\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:14.040566 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.040555 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/68da1e47-8873-416c-b2d9-d515d0985631-config\") pod \"service-ca-operator-d6fc45fc5-bd6gc\" (UID: \"68da1e47-8873-416c-b2d9-d515d0985631\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-bd6gc" Apr 22 15:08:14.040826 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.040574 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/acbfa8e1-30e3-467a-b70e-c9e23ebddcbe-nginx-conf\") pod \"networking-console-plugin-cb95c66f6-6ljs8\" (UID: \"acbfa8e1-30e3-467a-b70e-c9e23ebddcbe\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-6ljs8" Apr 22 15:08:14.040826 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.040791 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7b8b7c3-8b82-46c5-bb74-0c90f6448aff-service-ca-bundle\") pod \"insights-operator-585dfdc468-rxj2p\" (UID: \"c7b8b7c3-8b82-46c5-bb74-0c90f6448aff\") " pod="openshift-insights/insights-operator-585dfdc468-rxj2p" Apr 22 15:08:14.040931 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.040834 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-smwnd\" (UniqueName: \"kubernetes.io/projected/68da1e47-8873-416c-b2d9-d515d0985631-kube-api-access-smwnd\") pod \"service-ca-operator-d6fc45fc5-bd6gc\" (UID: \"68da1e47-8873-416c-b2d9-d515d0985631\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-bd6gc" Apr 22 15:08:14.040931 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.040853 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/acbfa8e1-30e3-467a-b70e-c9e23ebddcbe-networking-console-plugin-cert\") pod \"networking-console-plugin-cb95c66f6-6ljs8\" (UID: \"acbfa8e1-30e3-467a-b70e-c9e23ebddcbe\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-6ljs8" Apr 22 15:08:14.041055 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.041029 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/df8a3082-b089-41db-bbd6-1e6b09fd544b-ca-trust-extracted\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:14.041111 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.041083 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-tls\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:14.041163 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.041118 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/89e3aa95-bc39-4144-bc81-144160cf50eb-config\") pod \"console-operator-9d4b6777b-s79j7\" (UID: \"89e3aa95-bc39-4144-bc81-144160cf50eb\") " pod="openshift-console-operator/console-operator-9d4b6777b-s79j7" Apr 22 15:08:14.041163 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.041153 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7mx29\" (UniqueName: \"kubernetes.io/projected/eb3d139a-27c4-4fa7-8809-a33c13551872-kube-api-access-7mx29\") pod \"ingress-canary-4qmd5\" (UID: \"eb3d139a-27c4-4fa7-8809-a33c13551872\") " pod="openshift-ingress-canary/ingress-canary-4qmd5" Apr 22 15:08:14.041257 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.041176 2577 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Apr 22 15:08:14.041257 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.041188 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3b905d0-a65d-4615-8789-ccfb785189a3-samples-operator-tls\") pod \"cluster-samples-operator-6dc5bdb6b4-k8gnz\" (UID: \"b3b905d0-a65d-4615-8789-ccfb785189a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-k8gnz" Apr 22 15:08:14.041257 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.041194 2577 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8bc6dfb5-5vmjx: secret "image-registry-tls" not found Apr 22 15:08:14.041257 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.041222 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/68da1e47-8873-416c-b2d9-d515d0985631-config\") pod \"service-ca-operator-d6fc45fc5-bd6gc\" (UID: \"68da1e47-8873-416c-b2d9-d515d0985631\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-bd6gc" Apr 22 15:08:14.041464 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.041268 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-tls podName:df8a3082-b089-41db-bbd6-1e6b09fd544b nodeName:}" failed. No retries permitted until 2026-04-22 15:08:14.541248272 +0000 UTC m=+33.597986346 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-tls") pod "image-registry-8bc6dfb5-5vmjx" (UID: "df8a3082-b089-41db-bbd6-1e6b09fd544b") : secret "image-registry-tls" not found Apr 22 15:08:14.041464 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.041374 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/acbfa8e1-30e3-467a-b70e-c9e23ebddcbe-nginx-conf\") pod \"networking-console-plugin-cb95c66f6-6ljs8\" (UID: \"acbfa8e1-30e3-467a-b70e-c9e23ebddcbe\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-6ljs8" Apr 22 15:08:14.041464 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.041386 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Apr 22 15:08:14.041464 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.041425 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bbd7337b-38ca-4006-bd9f-d822e5410b7c-image-registry-private-configuration\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:14.041464 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.041462 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7b8b7c3-8b82-46c5-bb74-0c90f6448aff-service-ca-bundle\") pod \"insights-operator-585dfdc468-rxj2p\" (UID: \"c7b8b7c3-8b82-46c5-bb74-0c90f6448aff\") " pod="openshift-insights/insights-operator-585dfdc468-rxj2p" Apr 22 15:08:14.041694 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.041472 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b905d0-a65d-4615-8789-ccfb785189a3-samples-operator-tls podName:b3b905d0-a65d-4615-8789-ccfb785189a3 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:14.541437756 +0000 UTC m=+33.598175822 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3b905d0-a65d-4615-8789-ccfb785189a3-samples-operator-tls") pod "cluster-samples-operator-6dc5bdb6b4-k8gnz" (UID: "b3b905d0-a65d-4615-8789-ccfb785189a3") : secret "samples-operator-tls" not found Apr 22 15:08:14.041694 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.041519 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/89e3aa95-bc39-4144-bc81-144160cf50eb-serving-cert\") pod \"console-operator-9d4b6777b-s79j7\" (UID: \"89e3aa95-bc39-4144-bc81-144160cf50eb\") " pod="openshift-console-operator/console-operator-9d4b6777b-s79j7" Apr 22 15:08:14.041694 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.041536 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Apr 22 15:08:14.041694 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.041546 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-bound-sa-token\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:14.041694 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.041577 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc334588-fb2e-4ac6-bb34-460dad264804-metrics-tls\") pod \"dns-default-4zrz9\" (UID: \"dc334588-fb2e-4ac6-bb34-460dad264804\") " pod="openshift-dns/dns-default-4zrz9" Apr 22 15:08:14.041694 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.041599 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acbfa8e1-30e3-467a-b70e-c9e23ebddcbe-networking-console-plugin-cert podName:acbfa8e1-30e3-467a-b70e-c9e23ebddcbe nodeName:}" failed. No retries permitted until 2026-04-22 15:08:14.541582295 +0000 UTC m=+33.598320380 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/acbfa8e1-30e3-467a-b70e-c9e23ebddcbe-networking-console-plugin-cert") pod "networking-console-plugin-cb95c66f6-6ljs8" (UID: "acbfa8e1-30e3-467a-b70e-c9e23ebddcbe") : secret "networking-console-plugin-cert" not found Apr 22 15:08:14.041694 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.041634 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6cd30837-92d3-44c3-ba6a-9c84a22a7c8b-serving-cert\") pod \"kube-storage-version-migrator-operator-6769c5d45-fxjmk\" (UID: \"6cd30837-92d3-44c3-ba6a-9c84a22a7c8b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-fxjmk" Apr 22 15:08:14.041694 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.041649 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 22 15:08:14.041694 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.041665 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc334588-fb2e-4ac6-bb34-460dad264804-tmp-dir\") pod \"dns-default-4zrz9\" (UID: \"dc334588-fb2e-4ac6-bb34-460dad264804\") " pod="openshift-dns/dns-default-4zrz9" Apr 22 15:08:14.041694 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.041693 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc334588-fb2e-4ac6-bb34-460dad264804-metrics-tls podName:dc334588-fb2e-4ac6-bb34-460dad264804 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:14.541678059 +0000 UTC m=+33.598416135 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dc334588-fb2e-4ac6-bb34-460dad264804-metrics-tls") pod "dns-default-4zrz9" (UID: "dc334588-fb2e-4ac6-bb34-460dad264804") : secret "dns-default-metrics-tls" not found Apr 22 15:08:14.042193 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.041724 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/89e3aa95-bc39-4144-bc81-144160cf50eb-trusted-ca\") pod \"console-operator-9d4b6777b-s79j7\" (UID: \"89e3aa95-bc39-4144-bc81-144160cf50eb\") " pod="openshift-console-operator/console-operator-9d4b6777b-s79j7" Apr 22 15:08:14.042193 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.041756 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/19d55ae0-1958-4a44-a653-e45e72f6e6e2-stats-auth\") pod \"router-default-594fffb56b-msk5s\" (UID: \"19d55ae0-1958-4a44-a653-e45e72f6e6e2\") " pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:14.042193 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.041784 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-tls\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:14.042193 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.041803 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/89e3aa95-bc39-4144-bc81-144160cf50eb-config\") pod \"console-operator-9d4b6777b-s79j7\" (UID: \"89e3aa95-bc39-4144-bc81-144160cf50eb\") " pod="openshift-console-operator/console-operator-9d4b6777b-s79j7" Apr 22 15:08:14.042193 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.041819 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/df8a3082-b089-41db-bbd6-1e6b09fd544b-installation-pull-secrets\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:14.042193 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.041849 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c5rdv\" (UniqueName: \"kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-kube-api-access-c5rdv\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:14.042193 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.041876 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/68da1e47-8873-416c-b2d9-d515d0985631-serving-cert\") pod \"service-ca-operator-d6fc45fc5-bd6gc\" (UID: \"68da1e47-8873-416c-b2d9-d515d0985631\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-bd6gc" Apr 22 15:08:14.042193 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.041912 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/19d55ae0-1958-4a44-a653-e45e72f6e6e2-default-certificate\") pod \"router-default-594fffb56b-msk5s\" (UID: \"19d55ae0-1958-4a44-a653-e45e72f6e6e2\") " pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:14.042193 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.041934 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc334588-fb2e-4ac6-bb34-460dad264804-tmp-dir\") pod \"dns-default-4zrz9\" (UID: \"dc334588-fb2e-4ac6-bb34-460dad264804\") " pod="openshift-dns/dns-default-4zrz9" Apr 22 15:08:14.042193 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.042000 2577 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Apr 22 15:08:14.042193 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.042014 2577 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64fd688846-mkdtj: secret "image-registry-tls" not found Apr 22 15:08:14.042193 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.042052 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-tls podName:bbd7337b-38ca-4006-bd9f-d822e5410b7c nodeName:}" failed. No retries permitted until 2026-04-22 15:08:14.542038831 +0000 UTC m=+33.598776904 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-tls") pod "image-registry-64fd688846-mkdtj" (UID: "bbd7337b-38ca-4006-bd9f-d822e5410b7c") : secret "image-registry-tls" not found Apr 22 15:08:14.042784 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.042202 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7b8b7c3-8b82-46c5-bb74-0c90f6448aff-tmp\") pod \"insights-operator-585dfdc468-rxj2p\" (UID: \"c7b8b7c3-8b82-46c5-bb74-0c90f6448aff\") " pod="openshift-insights/insights-operator-585dfdc468-rxj2p" Apr 22 15:08:14.042784 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.041935 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7b8b7c3-8b82-46c5-bb74-0c90f6448aff-tmp\") pod \"insights-operator-585dfdc468-rxj2p\" (UID: \"c7b8b7c3-8b82-46c5-bb74-0c90f6448aff\") " pod="openshift-insights/insights-operator-585dfdc468-rxj2p" Apr 22 15:08:14.042784 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.042476 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-certificates\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:14.042784 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.042512 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6cd30837-92d3-44c3-ba6a-9c84a22a7c8b-config\") pod \"kube-storage-version-migrator-operator-6769c5d45-fxjmk\" (UID: \"6cd30837-92d3-44c3-ba6a-9c84a22a7c8b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-fxjmk" Apr 22 15:08:14.042784 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.042547 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7b8b7c3-8b82-46c5-bb74-0c90f6448aff-trusted-ca-bundle\") pod \"insights-operator-585dfdc468-rxj2p\" (UID: \"c7b8b7c3-8b82-46c5-bb74-0c90f6448aff\") " pod="openshift-insights/insights-operator-585dfdc468-rxj2p" Apr 22 15:08:14.042784 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.042572 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bs9d9\" (UniqueName: \"kubernetes.io/projected/c7b8b7c3-8b82-46c5-bb74-0c90f6448aff-kube-api-access-bs9d9\") pod \"insights-operator-585dfdc468-rxj2p\" (UID: \"c7b8b7c3-8b82-46c5-bb74-0c90f6448aff\") " pod="openshift-insights/insights-operator-585dfdc468-rxj2p" Apr 22 15:08:14.042784 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.042633 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19d55ae0-1958-4a44-a653-e45e72f6e6e2-metrics-certs\") pod \"router-default-594fffb56b-msk5s\" (UID: \"19d55ae0-1958-4a44-a653-e45e72f6e6e2\") " pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:14.042784 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.042655 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbd7337b-38ca-4006-bd9f-d822e5410b7c-trusted-ca\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:14.042784 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.042686 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dc334588-fb2e-4ac6-bb34-460dad264804-config-volume\") pod \"dns-default-4zrz9\" (UID: \"dc334588-fb2e-4ac6-bb34-460dad264804\") " pod="openshift-dns/dns-default-4zrz9" Apr 22 15:08:14.042784 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.042713 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hv76d\" (UniqueName: \"kubernetes.io/projected/3d4eaf4b-2b28-4ad4-866f-ac541d15b00b-kube-api-access-hv76d\") pod \"network-check-source-8894fc9bd-nmq7h\" (UID: \"3d4eaf4b-2b28-4ad4-866f-ac541d15b00b\") " pod="openshift-network-diagnostics/network-check-source-8894fc9bd-nmq7h" Apr 22 15:08:14.042784 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.042738 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bbd7337b-38ca-4006-bd9f-d822e5410b7c-installation-pull-secrets\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:14.042784 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.042764 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a84180fa-5ffa-4cc6-80cc-0cd55558a345-telemetry-config\") pod \"cluster-monitoring-operator-75587bd455-29bzc\" (UID: \"a84180fa-5ffa-4cc6-80cc-0cd55558a345\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-29bzc" Apr 22 15:08:14.042784 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.042790 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rwbfv\" (UniqueName: \"kubernetes.io/projected/19d55ae0-1958-4a44-a653-e45e72f6e6e2-kube-api-access-rwbfv\") pod \"router-default-594fffb56b-msk5s\" (UID: \"19d55ae0-1958-4a44-a653-e45e72f6e6e2\") " pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:14.043415 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.042848 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-94b8k\" (UniqueName: \"kubernetes.io/projected/6cd30837-92d3-44c3-ba6a-9c84a22a7c8b-kube-api-access-94b8k\") pod \"kube-storage-version-migrator-operator-6769c5d45-fxjmk\" (UID: \"6cd30837-92d3-44c3-ba6a-9c84a22a7c8b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-fxjmk" Apr 22 15:08:14.043415 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.042877 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-certificates\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:14.043415 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.042902 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w9zt7\" (UniqueName: \"kubernetes.io/projected/dc334588-fb2e-4ac6-bb34-460dad264804-kube-api-access-w9zt7\") pod \"dns-default-4zrz9\" (UID: \"dc334588-fb2e-4ac6-bb34-460dad264804\") " pod="openshift-dns/dns-default-4zrz9" Apr 22 15:08:14.043415 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.042933 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2dcqn\" (UniqueName: \"kubernetes.io/projected/a84180fa-5ffa-4cc6-80cc-0cd55558a345-kube-api-access-2dcqn\") pod \"cluster-monitoring-operator-75587bd455-29bzc\" (UID: \"a84180fa-5ffa-4cc6-80cc-0cd55558a345\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-29bzc" Apr 22 15:08:14.043415 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.042960 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/df8a3082-b089-41db-bbd6-1e6b09fd544b-trusted-ca\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:14.043415 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.042984 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-bound-sa-token\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:14.043415 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.043017 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7hmbv\" (UniqueName: \"kubernetes.io/projected/b3b905d0-a65d-4615-8789-ccfb785189a3-kube-api-access-7hmbv\") pod \"cluster-samples-operator-6dc5bdb6b4-k8gnz\" (UID: \"b3b905d0-a65d-4615-8789-ccfb785189a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-k8gnz" Apr 22 15:08:14.043415 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.043041 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c7b8b7c3-8b82-46c5-bb74-0c90f6448aff-snapshots\") pod \"insights-operator-585dfdc468-rxj2p\" (UID: \"c7b8b7c3-8b82-46c5-bb74-0c90f6448aff\") " pod="openshift-insights/insights-operator-585dfdc468-rxj2p" Apr 22 15:08:14.043415 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.043085 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c7b8b7c3-8b82-46c5-bb74-0c90f6448aff-serving-cert\") pod \"insights-operator-585dfdc468-rxj2p\" (UID: \"c7b8b7c3-8b82-46c5-bb74-0c90f6448aff\") " pod="openshift-insights/insights-operator-585dfdc468-rxj2p" Apr 22 15:08:14.043415 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.043120 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bbd7337b-38ca-4006-bd9f-d822e5410b7c-ca-trust-extracted\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:14.043415 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.043164 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eb3d139a-27c4-4fa7-8809-a33c13551872-cert\") pod \"ingress-canary-4qmd5\" (UID: \"eb3d139a-27c4-4fa7-8809-a33c13551872\") " pod="openshift-ingress-canary/ingress-canary-4qmd5" Apr 22 15:08:14.043415 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.043192 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19d55ae0-1958-4a44-a653-e45e72f6e6e2-service-ca-bundle\") pod \"router-default-594fffb56b-msk5s\" (UID: \"19d55ae0-1958-4a44-a653-e45e72f6e6e2\") " pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:14.043415 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.043221 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-chrhs\" (UniqueName: \"kubernetes.io/projected/4ccb076c-b0c3-40bd-b647-ef47ba1d8681-kube-api-access-chrhs\") pod \"volume-data-source-validator-7c6cbb6c87-lrs9z\" (UID: \"4ccb076c-b0c3-40bd-b647-ef47ba1d8681\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-lrs9z" Apr 22 15:08:14.043415 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.043252 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jvf69\" (UniqueName: \"kubernetes.io/projected/89e3aa95-bc39-4144-bc81-144160cf50eb-kube-api-access-jvf69\") pod \"console-operator-9d4b6777b-s79j7\" (UID: \"89e3aa95-bc39-4144-bc81-144160cf50eb\") " pod="openshift-console-operator/console-operator-9d4b6777b-s79j7" Apr 22 15:08:14.043415 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.043283 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jrzpl\" (UniqueName: \"kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-kube-api-access-jrzpl\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:14.043415 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.043312 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a84180fa-5ffa-4cc6-80cc-0cd55558a345-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-75587bd455-29bzc\" (UID: \"a84180fa-5ffa-4cc6-80cc-0cd55558a345\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-29bzc" Apr 22 15:08:14.044180 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.043461 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Apr 22 15:08:14.044180 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.043551 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a84180fa-5ffa-4cc6-80cc-0cd55558a345-cluster-monitoring-operator-tls podName:a84180fa-5ffa-4cc6-80cc-0cd55558a345 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:14.543491815 +0000 UTC m=+33.600229878 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a84180fa-5ffa-4cc6-80cc-0cd55558a345-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-75587bd455-29bzc" (UID: "a84180fa-5ffa-4cc6-80cc-0cd55558a345") : secret "cluster-monitoring-operator-tls" not found Apr 22 15:08:14.044180 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.043642 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-certificates\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:14.044180 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.044171 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6cd30837-92d3-44c3-ba6a-9c84a22a7c8b-config\") pod \"kube-storage-version-migrator-operator-6769c5d45-fxjmk\" (UID: \"6cd30837-92d3-44c3-ba6a-9c84a22a7c8b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-fxjmk" Apr 22 15:08:14.044409 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.044357 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/89e3aa95-bc39-4144-bc81-144160cf50eb-trusted-ca\") pod \"console-operator-9d4b6777b-s79j7\" (UID: \"89e3aa95-bc39-4144-bc81-144160cf50eb\") " pod="openshift-console-operator/console-operator-9d4b6777b-s79j7" Apr 22 15:08:14.046871 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.044623 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbd7337b-38ca-4006-bd9f-d822e5410b7c-trusted-ca\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:14.046871 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.044937 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7b8b7c3-8b82-46c5-bb74-0c90f6448aff-trusted-ca-bundle\") pod \"insights-operator-585dfdc468-rxj2p\" (UID: \"c7b8b7c3-8b82-46c5-bb74-0c90f6448aff\") " pod="openshift-insights/insights-operator-585dfdc468-rxj2p" Apr 22 15:08:14.046871 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.045415 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/df8a3082-b089-41db-bbd6-1e6b09fd544b-trusted-ca\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:14.046871 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.045848 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6cd30837-92d3-44c3-ba6a-9c84a22a7c8b-serving-cert\") pod \"kube-storage-version-migrator-operator-6769c5d45-fxjmk\" (UID: \"6cd30837-92d3-44c3-ba6a-9c84a22a7c8b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-fxjmk" Apr 22 15:08:14.046871 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.046190 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-certificates\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:14.046871 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.046256 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Apr 22 15:08:14.046871 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.046321 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19d55ae0-1958-4a44-a653-e45e72f6e6e2-metrics-certs podName:19d55ae0-1958-4a44-a653-e45e72f6e6e2 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:14.546305081 +0000 UTC m=+33.603043147 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19d55ae0-1958-4a44-a653-e45e72f6e6e2-metrics-certs") pod "router-default-594fffb56b-msk5s" (UID: "19d55ae0-1958-4a44-a653-e45e72f6e6e2") : secret "router-metrics-certs-default" not found Apr 22 15:08:14.046871 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.046375 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bbd7337b-38ca-4006-bd9f-d822e5410b7c-ca-trust-extracted\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:14.046871 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.046397 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/68da1e47-8873-416c-b2d9-d515d0985631-serving-cert\") pod \"service-ca-operator-d6fc45fc5-bd6gc\" (UID: \"68da1e47-8873-416c-b2d9-d515d0985631\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-bd6gc" Apr 22 15:08:14.046871 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.046622 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bbd7337b-38ca-4006-bd9f-d822e5410b7c-image-registry-private-configuration\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:14.046871 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.046251 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/89e3aa95-bc39-4144-bc81-144160cf50eb-serving-cert\") pod \"console-operator-9d4b6777b-s79j7\" (UID: \"89e3aa95-bc39-4144-bc81-144160cf50eb\") " pod="openshift-console-operator/console-operator-9d4b6777b-s79j7" Apr 22 15:08:14.046871 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.046681 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 22 15:08:14.046871 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.046726 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb3d139a-27c4-4fa7-8809-a33c13551872-cert podName:eb3d139a-27c4-4fa7-8809-a33c13551872 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:14.546711383 +0000 UTC m=+33.603449447 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/eb3d139a-27c4-4fa7-8809-a33c13551872-cert") pod "ingress-canary-4qmd5" (UID: "eb3d139a-27c4-4fa7-8809-a33c13551872") : secret "canary-serving-cert" not found Apr 22 15:08:14.046871 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.046777 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/19d55ae0-1958-4a44-a653-e45e72f6e6e2-service-ca-bundle podName:19d55ae0-1958-4a44-a653-e45e72f6e6e2 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:14.546762209 +0000 UTC m=+33.603500274 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/19d55ae0-1958-4a44-a653-e45e72f6e6e2-service-ca-bundle") pod "router-default-594fffb56b-msk5s" (UID: "19d55ae0-1958-4a44-a653-e45e72f6e6e2") : configmap references non-existent config key: service-ca.crt Apr 22 15:08:14.048390 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.048367 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a84180fa-5ffa-4cc6-80cc-0cd55558a345-telemetry-config\") pod \"cluster-monitoring-operator-75587bd455-29bzc\" (UID: \"a84180fa-5ffa-4cc6-80cc-0cd55558a345\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-29bzc" Apr 22 15:08:14.048466 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.048427 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dc334588-fb2e-4ac6-bb34-460dad264804-config-volume\") pod \"dns-default-4zrz9\" (UID: \"dc334588-fb2e-4ac6-bb34-460dad264804\") " pod="openshift-dns/dns-default-4zrz9" Apr 22 15:08:14.048466 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.048446 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c7b8b7c3-8b82-46c5-bb74-0c90f6448aff-snapshots\") pod \"insights-operator-585dfdc468-rxj2p\" (UID: \"c7b8b7c3-8b82-46c5-bb74-0c90f6448aff\") " pod="openshift-insights/insights-operator-585dfdc468-rxj2p" Apr 22 15:08:14.048919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.048856 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/df8a3082-b089-41db-bbd6-1e6b09fd544b-installation-pull-secrets\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:14.049391 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.049118 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bbd7337b-38ca-4006-bd9f-d822e5410b7c-installation-pull-secrets\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:14.049391 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.049184 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/df8a3082-b089-41db-bbd6-1e6b09fd544b-image-registry-private-configuration\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:14.049391 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.049384 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/19d55ae0-1958-4a44-a653-e45e72f6e6e2-default-certificate\") pod \"router-default-594fffb56b-msk5s\" (UID: \"19d55ae0-1958-4a44-a653-e45e72f6e6e2\") " pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:14.049921 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.049904 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/19d55ae0-1958-4a44-a653-e45e72f6e6e2-stats-auth\") pod \"router-default-594fffb56b-msk5s\" (UID: \"19d55ae0-1958-4a44-a653-e45e72f6e6e2\") " pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:14.054879 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.054845 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c7b8b7c3-8b82-46c5-bb74-0c90f6448aff-serving-cert\") pod \"insights-operator-585dfdc468-rxj2p\" (UID: \"c7b8b7c3-8b82-46c5-bb74-0c90f6448aff\") " pod="openshift-insights/insights-operator-585dfdc468-rxj2p" Apr 22 15:08:14.055514 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.055478 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-smwnd\" (UniqueName: \"kubernetes.io/projected/68da1e47-8873-416c-b2d9-d515d0985631-kube-api-access-smwnd\") pod \"service-ca-operator-d6fc45fc5-bd6gc\" (UID: \"68da1e47-8873-416c-b2d9-d515d0985631\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-bd6gc" Apr 22 15:08:14.056240 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.056219 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-bound-sa-token\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:14.056587 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.056563 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5rdv\" (UniqueName: \"kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-kube-api-access-c5rdv\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:14.058346 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.057961 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mx29\" (UniqueName: \"kubernetes.io/projected/eb3d139a-27c4-4fa7-8809-a33c13551872-kube-api-access-7mx29\") pod \"ingress-canary-4qmd5\" (UID: \"eb3d139a-27c4-4fa7-8809-a33c13551872\") " pod="openshift-ingress-canary/ingress-canary-4qmd5" Apr 22 15:08:14.059260 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.059234 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chrhs\" (UniqueName: \"kubernetes.io/projected/4ccb076c-b0c3-40bd-b647-ef47ba1d8681-kube-api-access-chrhs\") pod \"volume-data-source-validator-7c6cbb6c87-lrs9z\" (UID: \"4ccb076c-b0c3-40bd-b647-ef47ba1d8681\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-lrs9z" Apr 22 15:08:14.059875 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.059834 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hmbv\" (UniqueName: \"kubernetes.io/projected/b3b905d0-a65d-4615-8789-ccfb785189a3-kube-api-access-7hmbv\") pod \"cluster-samples-operator-6dc5bdb6b4-k8gnz\" (UID: \"b3b905d0-a65d-4615-8789-ccfb785189a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-k8gnz" Apr 22 15:08:14.061153 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.061126 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs9d9\" (UniqueName: \"kubernetes.io/projected/c7b8b7c3-8b82-46c5-bb74-0c90f6448aff-kube-api-access-bs9d9\") pod \"insights-operator-585dfdc468-rxj2p\" (UID: \"c7b8b7c3-8b82-46c5-bb74-0c90f6448aff\") " pod="openshift-insights/insights-operator-585dfdc468-rxj2p" Apr 22 15:08:14.061241 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.061194 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrzpl\" (UniqueName: \"kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-kube-api-access-jrzpl\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:14.061415 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.061319 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9zt7\" (UniqueName: \"kubernetes.io/projected/dc334588-fb2e-4ac6-bb34-460dad264804-kube-api-access-w9zt7\") pod \"dns-default-4zrz9\" (UID: \"dc334588-fb2e-4ac6-bb34-460dad264804\") " pod="openshift-dns/dns-default-4zrz9" Apr 22 15:08:14.061636 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.061615 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvf69\" (UniqueName: \"kubernetes.io/projected/89e3aa95-bc39-4144-bc81-144160cf50eb-kube-api-access-jvf69\") pod \"console-operator-9d4b6777b-s79j7\" (UID: \"89e3aa95-bc39-4144-bc81-144160cf50eb\") " pod="openshift-console-operator/console-operator-9d4b6777b-s79j7" Apr 22 15:08:14.062235 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.061645 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94b8k\" (UniqueName: \"kubernetes.io/projected/6cd30837-92d3-44c3-ba6a-9c84a22a7c8b-kube-api-access-94b8k\") pod \"kube-storage-version-migrator-operator-6769c5d45-fxjmk\" (UID: \"6cd30837-92d3-44c3-ba6a-9c84a22a7c8b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-fxjmk" Apr 22 15:08:14.062235 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.061780 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwbfv\" (UniqueName: \"kubernetes.io/projected/19d55ae0-1958-4a44-a653-e45e72f6e6e2-kube-api-access-rwbfv\") pod \"router-default-594fffb56b-msk5s\" (UID: \"19d55ae0-1958-4a44-a653-e45e72f6e6e2\") " pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:14.062371 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.062350 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv76d\" (UniqueName: \"kubernetes.io/projected/3d4eaf4b-2b28-4ad4-866f-ac541d15b00b-kube-api-access-hv76d\") pod \"network-check-source-8894fc9bd-nmq7h\" (UID: \"3d4eaf4b-2b28-4ad4-866f-ac541d15b00b\") " pod="openshift-network-diagnostics/network-check-source-8894fc9bd-nmq7h" Apr 22 15:08:14.062951 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.062929 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dcqn\" (UniqueName: \"kubernetes.io/projected/a84180fa-5ffa-4cc6-80cc-0cd55558a345-kube-api-access-2dcqn\") pod \"cluster-monitoring-operator-75587bd455-29bzc\" (UID: \"a84180fa-5ffa-4cc6-80cc-0cd55558a345\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-29bzc" Apr 22 15:08:14.063506 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.063473 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-bound-sa-token\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:14.130073 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.130044 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-585dfdc468-rxj2p" Apr 22 15:08:14.144443 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.144412 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/487f23b6-9470-41e9-abbd-295b6d317b10-metrics-certs\") pod \"network-metrics-daemon-z9ckr\" (UID: \"487f23b6-9470-41e9-abbd-295b6d317b10\") " pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:08:14.144567 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.144548 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 15:08:14.144640 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.144625 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/487f23b6-9470-41e9-abbd-295b6d317b10-metrics-certs podName:487f23b6-9470-41e9-abbd-295b6d317b10 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:46.144609356 +0000 UTC m=+65.201347419 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/487f23b6-9470-41e9-abbd-295b6d317b10-metrics-certs") pod "network-metrics-daemon-z9ckr" (UID: "487f23b6-9470-41e9-abbd-295b6d317b10") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 15:08:14.152779 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.152750 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-lrs9z" Apr 22 15:08:14.164868 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.164639 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-8894fc9bd-nmq7h" Apr 22 15:08:14.174154 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.174127 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-9d4b6777b-s79j7" Apr 22 15:08:14.183006 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.182971 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-bd6gc" Apr 22 15:08:14.207154 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.207129 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-fxjmk" Apr 22 15:08:14.250088 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.249747 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w9n55\" (UniqueName: \"kubernetes.io/projected/68a481e1-e027-433e-84ea-cb8d42406d8f-kube-api-access-w9n55\") pod \"network-check-target-8lfsw\" (UID: \"68a481e1-e027-433e-84ea-cb8d42406d8f\") " pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:08:14.254908 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.254850 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9n55\" (UniqueName: \"kubernetes.io/projected/68a481e1-e027-433e-84ea-cb8d42406d8f-kube-api-access-w9n55\") pod \"network-check-target-8lfsw\" (UID: \"68a481e1-e027-433e-84ea-cb8d42406d8f\") " pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:08:14.356779 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.356721 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-lrs9z"] Apr 22 15:08:14.359095 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.358755 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-585dfdc468-rxj2p"] Apr 22 15:08:14.361071 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:08:14.360998 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ccb076c_b0c3_40bd_b647_ef47ba1d8681.slice/crio-9db3256ada4946a678fd7569bd84fba67ec7f709a6e782b0025e10a17a8c11ef WatchSource:0}: Error finding container 9db3256ada4946a678fd7569bd84fba67ec7f709a6e782b0025e10a17a8c11ef: Status 404 returned error can't find the container with id 9db3256ada4946a678fd7569bd84fba67ec7f709a6e782b0025e10a17a8c11ef Apr 22 15:08:14.371280 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:08:14.371235 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7b8b7c3_8b82_46c5_bb74_0c90f6448aff.slice/crio-a2bd5510e541f6252f75e4d0b7dc26be2663a937a5035c5bfd63b6f98f6f2325 WatchSource:0}: Error finding container a2bd5510e541f6252f75e4d0b7dc26be2663a937a5035c5bfd63b6f98f6f2325: Status 404 returned error can't find the container with id a2bd5510e541f6252f75e4d0b7dc26be2663a937a5035c5bfd63b6f98f6f2325 Apr 22 15:08:14.406999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.406972 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-8894fc9bd-nmq7h"] Apr 22 15:08:14.411528 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:08:14.411498 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d4eaf4b_2b28_4ad4_866f_ac541d15b00b.slice/crio-721fb3636e47db01c8f3e5e88df62c62fbf0601c2b7831c9ad7d76d6190250a1 WatchSource:0}: Error finding container 721fb3636e47db01c8f3e5e88df62c62fbf0601c2b7831c9ad7d76d6190250a1: Status 404 returned error can't find the container with id 721fb3636e47db01c8f3e5e88df62c62fbf0601c2b7831c9ad7d76d6190250a1 Apr 22 15:08:14.413747 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:08:14.413713 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68da1e47_8873_416c_b2d9_d515d0985631.slice/crio-ec5a5e0a57b713b502032c70ae6819450dc441605a45b75614020085578f45a8 WatchSource:0}: Error finding container ec5a5e0a57b713b502032c70ae6819450dc441605a45b75614020085578f45a8: Status 404 returned error can't find the container with id ec5a5e0a57b713b502032c70ae6819450dc441605a45b75614020085578f45a8 Apr 22 15:08:14.414562 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.414518 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-d6fc45fc5-bd6gc"] Apr 22 15:08:14.425396 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.425374 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-9d4b6777b-s79j7"] Apr 22 15:08:14.429222 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:08:14.429191 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89e3aa95_bc39_4144_bc81_144160cf50eb.slice/crio-0d8a820c4126f7523103a4007493b3cfaa1bc6ef80145710895587672635185f WatchSource:0}: Error finding container 0d8a820c4126f7523103a4007493b3cfaa1bc6ef80145710895587672635185f: Status 404 returned error can't find the container with id 0d8a820c4126f7523103a4007493b3cfaa1bc6ef80145710895587672635185f Apr 22 15:08:14.440133 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.440112 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-fxjmk"] Apr 22 15:08:14.442861 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:08:14.442840 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cd30837_92d3_44c3_ba6a_9c84a22a7c8b.slice/crio-ca48732e0f27b6b7623ce4bf494df513a7b54e7abdc2372af683964a6cc0772d WatchSource:0}: Error finding container ca48732e0f27b6b7623ce4bf494df513a7b54e7abdc2372af683964a6cc0772d: Status 404 returned error can't find the container with id ca48732e0f27b6b7623ce4bf494df513a7b54e7abdc2372af683964a6cc0772d Apr 22 15:08:14.554702 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.554663 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-tls\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:14.554877 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.554738 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19d55ae0-1958-4a44-a653-e45e72f6e6e2-metrics-certs\") pod \"router-default-594fffb56b-msk5s\" (UID: \"19d55ae0-1958-4a44-a653-e45e72f6e6e2\") " pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:14.554877 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.554814 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eb3d139a-27c4-4fa7-8809-a33c13551872-cert\") pod \"ingress-canary-4qmd5\" (UID: \"eb3d139a-27c4-4fa7-8809-a33c13551872\") " pod="openshift-ingress-canary/ingress-canary-4qmd5" Apr 22 15:08:14.554877 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.554841 2577 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Apr 22 15:08:14.554877 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.554866 2577 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64fd688846-mkdtj: secret "image-registry-tls" not found Apr 22 15:08:14.555014 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.554887 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Apr 22 15:08:14.555014 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.554844 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19d55ae0-1958-4a44-a653-e45e72f6e6e2-service-ca-bundle\") pod \"router-default-594fffb56b-msk5s\" (UID: \"19d55ae0-1958-4a44-a653-e45e72f6e6e2\") " pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:14.555014 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.554930 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-tls podName:bbd7337b-38ca-4006-bd9f-d822e5410b7c nodeName:}" failed. No retries permitted until 2026-04-22 15:08:15.554908193 +0000 UTC m=+34.611646259 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-tls") pod "image-registry-64fd688846-mkdtj" (UID: "bbd7337b-38ca-4006-bd9f-d822e5410b7c") : secret "image-registry-tls" not found Apr 22 15:08:14.555014 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.554965 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/19d55ae0-1958-4a44-a653-e45e72f6e6e2-service-ca-bundle podName:19d55ae0-1958-4a44-a653-e45e72f6e6e2 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:15.554947773 +0000 UTC m=+34.611685844 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/19d55ae0-1958-4a44-a653-e45e72f6e6e2-service-ca-bundle") pod "router-default-594fffb56b-msk5s" (UID: "19d55ae0-1958-4a44-a653-e45e72f6e6e2") : configmap references non-existent config key: service-ca.crt Apr 22 15:08:14.555014 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.554965 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 22 15:08:14.555014 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.554982 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19d55ae0-1958-4a44-a653-e45e72f6e6e2-metrics-certs podName:19d55ae0-1958-4a44-a653-e45e72f6e6e2 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:15.554973566 +0000 UTC m=+34.611711632 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19d55ae0-1958-4a44-a653-e45e72f6e6e2-metrics-certs") pod "router-default-594fffb56b-msk5s" (UID: "19d55ae0-1958-4a44-a653-e45e72f6e6e2") : secret "router-metrics-certs-default" not found Apr 22 15:08:14.555014 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.555002 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb3d139a-27c4-4fa7-8809-a33c13551872-cert podName:eb3d139a-27c4-4fa7-8809-a33c13551872 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:15.554993096 +0000 UTC m=+34.611731160 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/eb3d139a-27c4-4fa7-8809-a33c13551872-cert") pod "ingress-canary-4qmd5" (UID: "eb3d139a-27c4-4fa7-8809-a33c13551872") : secret "canary-serving-cert" not found Apr 22 15:08:14.555262 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.555042 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a84180fa-5ffa-4cc6-80cc-0cd55558a345-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-75587bd455-29bzc\" (UID: \"a84180fa-5ffa-4cc6-80cc-0cd55558a345\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-29bzc" Apr 22 15:08:14.555262 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.555082 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/acbfa8e1-30e3-467a-b70e-c9e23ebddcbe-networking-console-plugin-cert\") pod \"networking-console-plugin-cb95c66f6-6ljs8\" (UID: \"acbfa8e1-30e3-467a-b70e-c9e23ebddcbe\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-6ljs8" Apr 22 15:08:14.555262 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.555105 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-tls\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:14.555262 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.555128 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3b905d0-a65d-4615-8789-ccfb785189a3-samples-operator-tls\") pod \"cluster-samples-operator-6dc5bdb6b4-k8gnz\" (UID: \"b3b905d0-a65d-4615-8789-ccfb785189a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-k8gnz" Apr 22 15:08:14.555262 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.555152 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc334588-fb2e-4ac6-bb34-460dad264804-metrics-tls\") pod \"dns-default-4zrz9\" (UID: \"dc334588-fb2e-4ac6-bb34-460dad264804\") " pod="openshift-dns/dns-default-4zrz9" Apr 22 15:08:14.555262 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.555238 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 22 15:08:14.555262 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.555261 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc334588-fb2e-4ac6-bb34-460dad264804-metrics-tls podName:dc334588-fb2e-4ac6-bb34-460dad264804 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:15.555254518 +0000 UTC m=+34.611992581 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dc334588-fb2e-4ac6-bb34-460dad264804-metrics-tls") pod "dns-default-4zrz9" (UID: "dc334588-fb2e-4ac6-bb34-460dad264804") : secret "dns-default-metrics-tls" not found Apr 22 15:08:14.555518 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.555297 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Apr 22 15:08:14.555518 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.555319 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a84180fa-5ffa-4cc6-80cc-0cd55558a345-cluster-monitoring-operator-tls podName:a84180fa-5ffa-4cc6-80cc-0cd55558a345 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:15.555310009 +0000 UTC m=+34.612048073 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a84180fa-5ffa-4cc6-80cc-0cd55558a345-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-75587bd455-29bzc" (UID: "a84180fa-5ffa-4cc6-80cc-0cd55558a345") : secret "cluster-monitoring-operator-tls" not found Apr 22 15:08:14.555518 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.555387 2577 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Apr 22 15:08:14.555518 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.555403 2577 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8bc6dfb5-5vmjx: secret "image-registry-tls" not found Apr 22 15:08:14.555518 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.555420 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Apr 22 15:08:14.555518 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.555390 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Apr 22 15:08:14.555518 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.555441 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-tls podName:df8a3082-b089-41db-bbd6-1e6b09fd544b nodeName:}" failed. No retries permitted until 2026-04-22 15:08:15.555428095 +0000 UTC m=+34.612166161 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-tls") pod "image-registry-8bc6dfb5-5vmjx" (UID: "df8a3082-b089-41db-bbd6-1e6b09fd544b") : secret "image-registry-tls" not found Apr 22 15:08:14.555518 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.555482 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b905d0-a65d-4615-8789-ccfb785189a3-samples-operator-tls podName:b3b905d0-a65d-4615-8789-ccfb785189a3 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:15.555447866 +0000 UTC m=+34.612185929 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3b905d0-a65d-4615-8789-ccfb785189a3-samples-operator-tls") pod "cluster-samples-operator-6dc5bdb6b4-k8gnz" (UID: "b3b905d0-a65d-4615-8789-ccfb785189a3") : secret "samples-operator-tls" not found Apr 22 15:08:14.555518 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:14.555494 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acbfa8e1-30e3-467a-b70e-c9e23ebddcbe-networking-console-plugin-cert podName:acbfa8e1-30e3-467a-b70e-c9e23ebddcbe nodeName:}" failed. No retries permitted until 2026-04-22 15:08:15.555487578 +0000 UTC m=+34.612225641 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/acbfa8e1-30e3-467a-b70e-c9e23ebddcbe-networking-console-plugin-cert") pod "networking-console-plugin-cb95c66f6-6ljs8" (UID: "acbfa8e1-30e3-467a-b70e-c9e23ebddcbe") : secret "networking-console-plugin-cert" not found Apr 22 15:08:14.651027 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.650947 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-fxjmk" event={"ID":"6cd30837-92d3-44c3-ba6a-9c84a22a7c8b","Type":"ContainerStarted","Data":"ca48732e0f27b6b7623ce4bf494df513a7b54e7abdc2372af683964a6cc0772d"} Apr 22 15:08:14.652082 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.652038 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-8894fc9bd-nmq7h" event={"ID":"3d4eaf4b-2b28-4ad4-866f-ac541d15b00b","Type":"ContainerStarted","Data":"721fb3636e47db01c8f3e5e88df62c62fbf0601c2b7831c9ad7d76d6190250a1"} Apr 22 15:08:14.653061 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.653040 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-lrs9z" event={"ID":"4ccb076c-b0c3-40bd-b647-ef47ba1d8681","Type":"ContainerStarted","Data":"9db3256ada4946a678fd7569bd84fba67ec7f709a6e782b0025e10a17a8c11ef"} Apr 22 15:08:14.654123 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.654100 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-585dfdc468-rxj2p" event={"ID":"c7b8b7c3-8b82-46c5-bb74-0c90f6448aff","Type":"ContainerStarted","Data":"a2bd5510e541f6252f75e4d0b7dc26be2663a937a5035c5bfd63b6f98f6f2325"} Apr 22 15:08:14.655114 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.655094 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-9d4b6777b-s79j7" event={"ID":"89e3aa95-bc39-4144-bc81-144160cf50eb","Type":"ContainerStarted","Data":"0d8a820c4126f7523103a4007493b3cfaa1bc6ef80145710895587672635185f"} Apr 22 15:08:14.656265 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:14.656231 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-bd6gc" event={"ID":"68da1e47-8873-416c-b2d9-d515d0985631","Type":"ContainerStarted","Data":"ec5a5e0a57b713b502032c70ae6819450dc441605a45b75614020085578f45a8"} Apr 22 15:08:15.504505 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:15.504471 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:08:15.505083 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:15.504489 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:08:15.510143 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:15.509840 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pdgcv\"" Apr 22 15:08:15.510143 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:15.509907 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lqzj2\"" Apr 22 15:08:15.510143 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:15.509850 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Apr 22 15:08:15.523860 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:15.523476 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:08:15.568537 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:15.566806 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a84180fa-5ffa-4cc6-80cc-0cd55558a345-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-75587bd455-29bzc\" (UID: \"a84180fa-5ffa-4cc6-80cc-0cd55558a345\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-29bzc" Apr 22 15:08:15.568537 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:15.566914 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/acbfa8e1-30e3-467a-b70e-c9e23ebddcbe-networking-console-plugin-cert\") pod \"networking-console-plugin-cb95c66f6-6ljs8\" (UID: \"acbfa8e1-30e3-467a-b70e-c9e23ebddcbe\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-6ljs8" Apr 22 15:08:15.568537 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:15.566986 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-tls\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:15.568537 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:15.567021 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3b905d0-a65d-4615-8789-ccfb785189a3-samples-operator-tls\") pod \"cluster-samples-operator-6dc5bdb6b4-k8gnz\" (UID: \"b3b905d0-a65d-4615-8789-ccfb785189a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-k8gnz" Apr 22 15:08:15.568537 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:15.567061 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc334588-fb2e-4ac6-bb34-460dad264804-metrics-tls\") pod \"dns-default-4zrz9\" (UID: \"dc334588-fb2e-4ac6-bb34-460dad264804\") " pod="openshift-dns/dns-default-4zrz9" Apr 22 15:08:15.568537 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:15.567095 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-tls\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:15.568537 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:15.567140 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19d55ae0-1958-4a44-a653-e45e72f6e6e2-metrics-certs\") pod \"router-default-594fffb56b-msk5s\" (UID: \"19d55ae0-1958-4a44-a653-e45e72f6e6e2\") " pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:15.568537 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:15.567210 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eb3d139a-27c4-4fa7-8809-a33c13551872-cert\") pod \"ingress-canary-4qmd5\" (UID: \"eb3d139a-27c4-4fa7-8809-a33c13551872\") " pod="openshift-ingress-canary/ingress-canary-4qmd5" Apr 22 15:08:15.568537 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:15.567238 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19d55ae0-1958-4a44-a653-e45e72f6e6e2-service-ca-bundle\") pod \"router-default-594fffb56b-msk5s\" (UID: \"19d55ae0-1958-4a44-a653-e45e72f6e6e2\") " pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:15.568537 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:15.567439 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Apr 22 15:08:15.568537 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:15.567513 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b905d0-a65d-4615-8789-ccfb785189a3-samples-operator-tls podName:b3b905d0-a65d-4615-8789-ccfb785189a3 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:17.567484881 +0000 UTC m=+36.624222965 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3b905d0-a65d-4615-8789-ccfb785189a3-samples-operator-tls") pod "cluster-samples-operator-6dc5bdb6b4-k8gnz" (UID: "b3b905d0-a65d-4615-8789-ccfb785189a3") : secret "samples-operator-tls" not found Apr 22 15:08:15.568537 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:15.567899 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Apr 22 15:08:15.568537 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:15.567948 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a84180fa-5ffa-4cc6-80cc-0cd55558a345-cluster-monitoring-operator-tls podName:a84180fa-5ffa-4cc6-80cc-0cd55558a345 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:17.567932891 +0000 UTC m=+36.624670971 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a84180fa-5ffa-4cc6-80cc-0cd55558a345-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-75587bd455-29bzc" (UID: "a84180fa-5ffa-4cc6-80cc-0cd55558a345") : secret "cluster-monitoring-operator-tls" not found Apr 22 15:08:15.568537 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:15.568001 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Apr 22 15:08:15.568537 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:15.568032 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acbfa8e1-30e3-467a-b70e-c9e23ebddcbe-networking-console-plugin-cert podName:acbfa8e1-30e3-467a-b70e-c9e23ebddcbe nodeName:}" failed. No retries permitted until 2026-04-22 15:08:17.568022017 +0000 UTC m=+36.624760083 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/acbfa8e1-30e3-467a-b70e-c9e23ebddcbe-networking-console-plugin-cert") pod "networking-console-plugin-cb95c66f6-6ljs8" (UID: "acbfa8e1-30e3-467a-b70e-c9e23ebddcbe") : secret "networking-console-plugin-cert" not found Apr 22 15:08:15.568537 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:15.568103 2577 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Apr 22 15:08:15.569436 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:15.568113 2577 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8bc6dfb5-5vmjx: secret "image-registry-tls" not found Apr 22 15:08:15.569436 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:15.568143 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-tls podName:df8a3082-b089-41db-bbd6-1e6b09fd544b nodeName:}" failed. No retries permitted until 2026-04-22 15:08:17.568133761 +0000 UTC m=+36.624871825 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-tls") pod "image-registry-8bc6dfb5-5vmjx" (UID: "df8a3082-b089-41db-bbd6-1e6b09fd544b") : secret "image-registry-tls" not found Apr 22 15:08:15.569436 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:15.568203 2577 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Apr 22 15:08:15.569436 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:15.568212 2577 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64fd688846-mkdtj: secret "image-registry-tls" not found Apr 22 15:08:15.569436 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:15.568239 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-tls podName:bbd7337b-38ca-4006-bd9f-d822e5410b7c nodeName:}" failed. No retries permitted until 2026-04-22 15:08:17.568229307 +0000 UTC m=+36.624967373 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-tls") pod "image-registry-64fd688846-mkdtj" (UID: "bbd7337b-38ca-4006-bd9f-d822e5410b7c") : secret "image-registry-tls" not found Apr 22 15:08:15.569436 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:15.568290 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 22 15:08:15.569436 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:15.568317 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc334588-fb2e-4ac6-bb34-460dad264804-metrics-tls podName:dc334588-fb2e-4ac6-bb34-460dad264804 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:17.568307701 +0000 UTC m=+36.625045798 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dc334588-fb2e-4ac6-bb34-460dad264804-metrics-tls") pod "dns-default-4zrz9" (UID: "dc334588-fb2e-4ac6-bb34-460dad264804") : secret "dns-default-metrics-tls" not found Apr 22 15:08:15.569436 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:15.568393 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Apr 22 15:08:15.569436 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:15.568427 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19d55ae0-1958-4a44-a653-e45e72f6e6e2-metrics-certs podName:19d55ae0-1958-4a44-a653-e45e72f6e6e2 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:17.56841517 +0000 UTC m=+36.625153234 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19d55ae0-1958-4a44-a653-e45e72f6e6e2-metrics-certs") pod "router-default-594fffb56b-msk5s" (UID: "19d55ae0-1958-4a44-a653-e45e72f6e6e2") : secret "router-metrics-certs-default" not found Apr 22 15:08:15.569436 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:15.568477 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 22 15:08:15.569436 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:15.568504 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb3d139a-27c4-4fa7-8809-a33c13551872-cert podName:eb3d139a-27c4-4fa7-8809-a33c13551872 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:17.568494137 +0000 UTC m=+36.625232218 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/eb3d139a-27c4-4fa7-8809-a33c13551872-cert") pod "ingress-canary-4qmd5" (UID: "eb3d139a-27c4-4fa7-8809-a33c13551872") : secret "canary-serving-cert" not found Apr 22 15:08:15.570096 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:15.570027 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/19d55ae0-1958-4a44-a653-e45e72f6e6e2-service-ca-bundle podName:19d55ae0-1958-4a44-a653-e45e72f6e6e2 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:17.570007951 +0000 UTC m=+36.626746025 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/19d55ae0-1958-4a44-a653-e45e72f6e6e2-service-ca-bundle") pod "router-default-594fffb56b-msk5s" (UID: "19d55ae0-1958-4a44-a653-e45e72f6e6e2") : configmap references non-existent config key: service-ca.crt Apr 22 15:08:15.702487 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:15.701354 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8lfsw"] Apr 22 15:08:15.710512 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:08:15.710463 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68a481e1_e027_433e_84ea_cb8d42406d8f.slice/crio-006654d1875dbd4bf152d4d29c363fab4c1bec5391647d2399d717f6f9177e8d WatchSource:0}: Error finding container 006654d1875dbd4bf152d4d29c363fab4c1bec5391647d2399d717f6f9177e8d: Status 404 returned error can't find the container with id 006654d1875dbd4bf152d4d29c363fab4c1bec5391647d2399d717f6f9177e8d Apr 22 15:08:16.671736 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:16.671483 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8lfsw" event={"ID":"68a481e1-e027-433e-84ea-cb8d42406d8f","Type":"ContainerStarted","Data":"006654d1875dbd4bf152d4d29c363fab4c1bec5391647d2399d717f6f9177e8d"} Apr 22 15:08:16.819122 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:16.818397 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-8l6g2"] Apr 22 15:08:16.823131 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:16.823105 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8l6g2" Apr 22 15:08:16.830746 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:16.830535 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Apr 22 15:08:16.842114 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:16.837604 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8l6g2"] Apr 22 15:08:16.988274 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:16.988237 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fb9ab245-62d7-4d19-93cd-34d5f9595d3d-dbus\") pod \"global-pull-secret-syncer-8l6g2\" (UID: \"fb9ab245-62d7-4d19-93cd-34d5f9595d3d\") " pod="kube-system/global-pull-secret-syncer-8l6g2" Apr 22 15:08:16.988454 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:16.988320 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fb9ab245-62d7-4d19-93cd-34d5f9595d3d-kubelet-config\") pod \"global-pull-secret-syncer-8l6g2\" (UID: \"fb9ab245-62d7-4d19-93cd-34d5f9595d3d\") " pod="kube-system/global-pull-secret-syncer-8l6g2" Apr 22 15:08:16.988518 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:16.988502 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb9ab245-62d7-4d19-93cd-34d5f9595d3d-original-pull-secret\") pod \"global-pull-secret-syncer-8l6g2\" (UID: \"fb9ab245-62d7-4d19-93cd-34d5f9595d3d\") " pod="kube-system/global-pull-secret-syncer-8l6g2" Apr 22 15:08:17.089895 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:17.089861 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb9ab245-62d7-4d19-93cd-34d5f9595d3d-original-pull-secret\") pod \"global-pull-secret-syncer-8l6g2\" (UID: \"fb9ab245-62d7-4d19-93cd-34d5f9595d3d\") " pod="kube-system/global-pull-secret-syncer-8l6g2" Apr 22 15:08:17.090066 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:17.090036 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fb9ab245-62d7-4d19-93cd-34d5f9595d3d-dbus\") pod \"global-pull-secret-syncer-8l6g2\" (UID: \"fb9ab245-62d7-4d19-93cd-34d5f9595d3d\") " pod="kube-system/global-pull-secret-syncer-8l6g2" Apr 22 15:08:17.090129 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:17.090104 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fb9ab245-62d7-4d19-93cd-34d5f9595d3d-kubelet-config\") pod \"global-pull-secret-syncer-8l6g2\" (UID: \"fb9ab245-62d7-4d19-93cd-34d5f9595d3d\") " pod="kube-system/global-pull-secret-syncer-8l6g2" Apr 22 15:08:17.090217 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:17.090200 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fb9ab245-62d7-4d19-93cd-34d5f9595d3d-kubelet-config\") pod \"global-pull-secret-syncer-8l6g2\" (UID: \"fb9ab245-62d7-4d19-93cd-34d5f9595d3d\") " pod="kube-system/global-pull-secret-syncer-8l6g2" Apr 22 15:08:17.090409 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:17.090387 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fb9ab245-62d7-4d19-93cd-34d5f9595d3d-dbus\") pod \"global-pull-secret-syncer-8l6g2\" (UID: \"fb9ab245-62d7-4d19-93cd-34d5f9595d3d\") " pod="kube-system/global-pull-secret-syncer-8l6g2" Apr 22 15:08:17.098790 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:17.098760 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb9ab245-62d7-4d19-93cd-34d5f9595d3d-original-pull-secret\") pod \"global-pull-secret-syncer-8l6g2\" (UID: \"fb9ab245-62d7-4d19-93cd-34d5f9595d3d\") " pod="kube-system/global-pull-secret-syncer-8l6g2" Apr 22 15:08:17.140072 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:17.140035 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8l6g2" Apr 22 15:08:17.595080 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:17.595001 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19d55ae0-1958-4a44-a653-e45e72f6e6e2-metrics-certs\") pod \"router-default-594fffb56b-msk5s\" (UID: \"19d55ae0-1958-4a44-a653-e45e72f6e6e2\") " pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:17.595237 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:17.595091 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eb3d139a-27c4-4fa7-8809-a33c13551872-cert\") pod \"ingress-canary-4qmd5\" (UID: \"eb3d139a-27c4-4fa7-8809-a33c13551872\") " pod="openshift-ingress-canary/ingress-canary-4qmd5" Apr 22 15:08:17.595237 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:17.595125 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19d55ae0-1958-4a44-a653-e45e72f6e6e2-service-ca-bundle\") pod \"router-default-594fffb56b-msk5s\" (UID: \"19d55ae0-1958-4a44-a653-e45e72f6e6e2\") " pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:17.595237 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:17.595142 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Apr 22 15:08:17.595237 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:17.595160 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a84180fa-5ffa-4cc6-80cc-0cd55558a345-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-75587bd455-29bzc\" (UID: \"a84180fa-5ffa-4cc6-80cc-0cd55558a345\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-29bzc" Apr 22 15:08:17.595237 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:17.595202 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19d55ae0-1958-4a44-a653-e45e72f6e6e2-metrics-certs podName:19d55ae0-1958-4a44-a653-e45e72f6e6e2 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:21.595186738 +0000 UTC m=+40.651924801 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19d55ae0-1958-4a44-a653-e45e72f6e6e2-metrics-certs") pod "router-default-594fffb56b-msk5s" (UID: "19d55ae0-1958-4a44-a653-e45e72f6e6e2") : secret "router-metrics-certs-default" not found Apr 22 15:08:17.595237 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:17.595231 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/acbfa8e1-30e3-467a-b70e-c9e23ebddcbe-networking-console-plugin-cert\") pod \"networking-console-plugin-cb95c66f6-6ljs8\" (UID: \"acbfa8e1-30e3-467a-b70e-c9e23ebddcbe\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-6ljs8" Apr 22 15:08:17.595598 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:17.595257 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-tls\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:17.595598 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:17.595261 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Apr 22 15:08:17.595598 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:17.595278 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3b905d0-a65d-4615-8789-ccfb785189a3-samples-operator-tls\") pod \"cluster-samples-operator-6dc5bdb6b4-k8gnz\" (UID: \"b3b905d0-a65d-4615-8789-ccfb785189a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-k8gnz" Apr 22 15:08:17.595598 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:17.595304 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc334588-fb2e-4ac6-bb34-460dad264804-metrics-tls\") pod \"dns-default-4zrz9\" (UID: \"dc334588-fb2e-4ac6-bb34-460dad264804\") " pod="openshift-dns/dns-default-4zrz9" Apr 22 15:08:17.595598 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:17.595318 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a84180fa-5ffa-4cc6-80cc-0cd55558a345-cluster-monitoring-operator-tls podName:a84180fa-5ffa-4cc6-80cc-0cd55558a345 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:21.595302744 +0000 UTC m=+40.652040810 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a84180fa-5ffa-4cc6-80cc-0cd55558a345-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-75587bd455-29bzc" (UID: "a84180fa-5ffa-4cc6-80cc-0cd55558a345") : secret "cluster-monitoring-operator-tls" not found Apr 22 15:08:17.595598 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:17.595368 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 22 15:08:17.595598 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:17.595376 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-tls\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:17.595598 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:17.595401 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc334588-fb2e-4ac6-bb34-460dad264804-metrics-tls podName:dc334588-fb2e-4ac6-bb34-460dad264804 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:21.595390589 +0000 UTC m=+40.652128654 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dc334588-fb2e-4ac6-bb34-460dad264804-metrics-tls") pod "dns-default-4zrz9" (UID: "dc334588-fb2e-4ac6-bb34-460dad264804") : secret "dns-default-metrics-tls" not found Apr 22 15:08:17.595598 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:17.595413 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Apr 22 15:08:17.595598 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:17.595448 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 22 15:08:17.595598 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:17.595466 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acbfa8e1-30e3-467a-b70e-c9e23ebddcbe-networking-console-plugin-cert podName:acbfa8e1-30e3-467a-b70e-c9e23ebddcbe nodeName:}" failed. No retries permitted until 2026-04-22 15:08:21.595450052 +0000 UTC m=+40.652188119 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/acbfa8e1-30e3-467a-b70e-c9e23ebddcbe-networking-console-plugin-cert") pod "networking-console-plugin-cb95c66f6-6ljs8" (UID: "acbfa8e1-30e3-467a-b70e-c9e23ebddcbe") : secret "networking-console-plugin-cert" not found Apr 22 15:08:17.595598 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:17.595467 2577 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Apr 22 15:08:17.595598 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:17.595483 2577 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64fd688846-mkdtj: secret "image-registry-tls" not found Apr 22 15:08:17.595598 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:17.595497 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb3d139a-27c4-4fa7-8809-a33c13551872-cert podName:eb3d139a-27c4-4fa7-8809-a33c13551872 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:21.595485191 +0000 UTC m=+40.652223258 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/eb3d139a-27c4-4fa7-8809-a33c13551872-cert") pod "ingress-canary-4qmd5" (UID: "eb3d139a-27c4-4fa7-8809-a33c13551872") : secret "canary-serving-cert" not found Apr 22 15:08:17.595598 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:17.595498 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Apr 22 15:08:17.595598 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:17.595452 2577 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Apr 22 15:08:17.595598 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:17.595514 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-tls podName:bbd7337b-38ca-4006-bd9f-d822e5410b7c nodeName:}" failed. No retries permitted until 2026-04-22 15:08:21.595504105 +0000 UTC m=+40.652242169 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-tls") pod "image-registry-64fd688846-mkdtj" (UID: "bbd7337b-38ca-4006-bd9f-d822e5410b7c") : secret "image-registry-tls" not found Apr 22 15:08:17.595598 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:17.595515 2577 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8bc6dfb5-5vmjx: secret "image-registry-tls" not found Apr 22 15:08:17.596528 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:17.595530 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/19d55ae0-1958-4a44-a653-e45e72f6e6e2-service-ca-bundle podName:19d55ae0-1958-4a44-a653-e45e72f6e6e2 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:21.595521768 +0000 UTC m=+40.652259831 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/19d55ae0-1958-4a44-a653-e45e72f6e6e2-service-ca-bundle") pod "router-default-594fffb56b-msk5s" (UID: "19d55ae0-1958-4a44-a653-e45e72f6e6e2") : configmap references non-existent config key: service-ca.crt Apr 22 15:08:17.596528 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:17.595546 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-tls podName:df8a3082-b089-41db-bbd6-1e6b09fd544b nodeName:}" failed. No retries permitted until 2026-04-22 15:08:21.595537307 +0000 UTC m=+40.652275377 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-tls") pod "image-registry-8bc6dfb5-5vmjx" (UID: "df8a3082-b089-41db-bbd6-1e6b09fd544b") : secret "image-registry-tls" not found Apr 22 15:08:17.596528 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:17.595562 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b905d0-a65d-4615-8789-ccfb785189a3-samples-operator-tls podName:b3b905d0-a65d-4615-8789-ccfb785189a3 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:21.595554763 +0000 UTC m=+40.652292825 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3b905d0-a65d-4615-8789-ccfb785189a3-samples-operator-tls") pod "cluster-samples-operator-6dc5bdb6b4-k8gnz" (UID: "b3b905d0-a65d-4615-8789-ccfb785189a3") : secret "samples-operator-tls" not found Apr 22 15:08:21.635791 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:21.635688 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/acbfa8e1-30e3-467a-b70e-c9e23ebddcbe-networking-console-plugin-cert\") pod \"networking-console-plugin-cb95c66f6-6ljs8\" (UID: \"acbfa8e1-30e3-467a-b70e-c9e23ebddcbe\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-6ljs8" Apr 22 15:08:21.635791 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:21.635749 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-tls\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:21.635791 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:21.635781 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3b905d0-a65d-4615-8789-ccfb785189a3-samples-operator-tls\") pod \"cluster-samples-operator-6dc5bdb6b4-k8gnz\" (UID: \"b3b905d0-a65d-4615-8789-ccfb785189a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-k8gnz" Apr 22 15:08:21.636614 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:21.635819 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc334588-fb2e-4ac6-bb34-460dad264804-metrics-tls\") pod \"dns-default-4zrz9\" (UID: \"dc334588-fb2e-4ac6-bb34-460dad264804\") " pod="openshift-dns/dns-default-4zrz9" Apr 22 15:08:21.636614 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:21.635856 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Apr 22 15:08:21.636614 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:21.635915 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 22 15:08:21.636614 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:21.635924 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acbfa8e1-30e3-467a-b70e-c9e23ebddcbe-networking-console-plugin-cert podName:acbfa8e1-30e3-467a-b70e-c9e23ebddcbe nodeName:}" failed. No retries permitted until 2026-04-22 15:08:29.635908726 +0000 UTC m=+48.692646789 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/acbfa8e1-30e3-467a-b70e-c9e23ebddcbe-networking-console-plugin-cert") pod "networking-console-plugin-cb95c66f6-6ljs8" (UID: "acbfa8e1-30e3-467a-b70e-c9e23ebddcbe") : secret "networking-console-plugin-cert" not found Apr 22 15:08:21.636614 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:21.635927 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Apr 22 15:08:21.636614 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:21.635948 2577 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Apr 22 15:08:21.636614 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:21.635968 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc334588-fb2e-4ac6-bb34-460dad264804-metrics-tls podName:dc334588-fb2e-4ac6-bb34-460dad264804 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:29.63595153 +0000 UTC m=+48.692689593 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dc334588-fb2e-4ac6-bb34-460dad264804-metrics-tls") pod "dns-default-4zrz9" (UID: "dc334588-fb2e-4ac6-bb34-460dad264804") : secret "dns-default-metrics-tls" not found Apr 22 15:08:21.636614 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:21.635967 2577 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8bc6dfb5-5vmjx: secret "image-registry-tls" not found Apr 22 15:08:21.636614 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:21.635995 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-tls\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:21.636614 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:21.636008 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-tls podName:df8a3082-b089-41db-bbd6-1e6b09fd544b nodeName:}" failed. No retries permitted until 2026-04-22 15:08:29.635998262 +0000 UTC m=+48.692736325 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-tls") pod "image-registry-8bc6dfb5-5vmjx" (UID: "df8a3082-b089-41db-bbd6-1e6b09fd544b") : secret "image-registry-tls" not found Apr 22 15:08:21.636614 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:21.636031 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b905d0-a65d-4615-8789-ccfb785189a3-samples-operator-tls podName:b3b905d0-a65d-4615-8789-ccfb785189a3 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:29.636023497 +0000 UTC m=+48.692761567 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3b905d0-a65d-4615-8789-ccfb785189a3-samples-operator-tls") pod "cluster-samples-operator-6dc5bdb6b4-k8gnz" (UID: "b3b905d0-a65d-4615-8789-ccfb785189a3") : secret "samples-operator-tls" not found Apr 22 15:08:21.636614 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:21.636048 2577 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Apr 22 15:08:21.636614 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:21.636057 2577 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64fd688846-mkdtj: secret "image-registry-tls" not found Apr 22 15:08:21.636614 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:21.636076 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19d55ae0-1958-4a44-a653-e45e72f6e6e2-metrics-certs\") pod \"router-default-594fffb56b-msk5s\" (UID: \"19d55ae0-1958-4a44-a653-e45e72f6e6e2\") " pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:21.636614 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:21.636109 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-tls podName:bbd7337b-38ca-4006-bd9f-d822e5410b7c nodeName:}" failed. No retries permitted until 2026-04-22 15:08:29.63609451 +0000 UTC m=+48.692832574 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-tls") pod "image-registry-64fd688846-mkdtj" (UID: "bbd7337b-38ca-4006-bd9f-d822e5410b7c") : secret "image-registry-tls" not found Apr 22 15:08:21.636614 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:21.636128 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Apr 22 15:08:21.636614 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:21.636157 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19d55ae0-1958-4a44-a653-e45e72f6e6e2-metrics-certs podName:19d55ae0-1958-4a44-a653-e45e72f6e6e2 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:29.636147433 +0000 UTC m=+48.692885497 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19d55ae0-1958-4a44-a653-e45e72f6e6e2-metrics-certs") pod "router-default-594fffb56b-msk5s" (UID: "19d55ae0-1958-4a44-a653-e45e72f6e6e2") : secret "router-metrics-certs-default" not found Apr 22 15:08:21.637291 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:21.636204 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eb3d139a-27c4-4fa7-8809-a33c13551872-cert\") pod \"ingress-canary-4qmd5\" (UID: \"eb3d139a-27c4-4fa7-8809-a33c13551872\") " pod="openshift-ingress-canary/ingress-canary-4qmd5" Apr 22 15:08:21.637291 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:21.636232 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19d55ae0-1958-4a44-a653-e45e72f6e6e2-service-ca-bundle\") pod \"router-default-594fffb56b-msk5s\" (UID: \"19d55ae0-1958-4a44-a653-e45e72f6e6e2\") " pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:21.637291 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:21.636263 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a84180fa-5ffa-4cc6-80cc-0cd55558a345-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-75587bd455-29bzc\" (UID: \"a84180fa-5ffa-4cc6-80cc-0cd55558a345\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-29bzc" Apr 22 15:08:21.637291 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:21.636346 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 22 15:08:21.637291 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:21.636370 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Apr 22 15:08:21.637291 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:21.636401 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb3d139a-27c4-4fa7-8809-a33c13551872-cert podName:eb3d139a-27c4-4fa7-8809-a33c13551872 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:29.636387491 +0000 UTC m=+48.693125555 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/eb3d139a-27c4-4fa7-8809-a33c13551872-cert") pod "ingress-canary-4qmd5" (UID: "eb3d139a-27c4-4fa7-8809-a33c13551872") : secret "canary-serving-cert" not found Apr 22 15:08:21.637291 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:21.636420 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a84180fa-5ffa-4cc6-80cc-0cd55558a345-cluster-monitoring-operator-tls podName:a84180fa-5ffa-4cc6-80cc-0cd55558a345 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:29.636410368 +0000 UTC m=+48.693148430 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a84180fa-5ffa-4cc6-80cc-0cd55558a345-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-75587bd455-29bzc" (UID: "a84180fa-5ffa-4cc6-80cc-0cd55558a345") : secret "cluster-monitoring-operator-tls" not found Apr 22 15:08:21.637291 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:21.636435 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/19d55ae0-1958-4a44-a653-e45e72f6e6e2-service-ca-bundle podName:19d55ae0-1958-4a44-a653-e45e72f6e6e2 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:29.63642762 +0000 UTC m=+48.693165682 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/19d55ae0-1958-4a44-a653-e45e72f6e6e2-service-ca-bundle") pod "router-default-594fffb56b-msk5s" (UID: "19d55ae0-1958-4a44-a653-e45e72f6e6e2") : configmap references non-existent config key: service-ca.crt Apr 22 15:08:23.907870 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:23.907820 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8l6g2"] Apr 22 15:08:24.020419 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:08:24.020386 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb9ab245_62d7_4d19_93cd_34d5f9595d3d.slice/crio-87ff2460a0a2346d4c2c53fbaab28b7d73af652fae2dc9920c3277c22522d691 WatchSource:0}: Error finding container 87ff2460a0a2346d4c2c53fbaab28b7d73af652fae2dc9920c3277c22522d691: Status 404 returned error can't find the container with id 87ff2460a0a2346d4c2c53fbaab28b7d73af652fae2dc9920c3277c22522d691 Apr 22 15:08:24.687447 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:24.687366 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8l6g2" event={"ID":"fb9ab245-62d7-4d19-93cd-34d5f9595d3d","Type":"ContainerStarted","Data":"87ff2460a0a2346d4c2c53fbaab28b7d73af652fae2dc9920c3277c22522d691"} Apr 22 15:08:24.690120 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:24.690091 2577 generic.go:358] "Generic (PLEG): container finished" podID="6b9559b7-6b5b-4ecf-9655-4379447b15c7" containerID="2dc4cedddf482fe1a3f77297ec7571f0eab2fd2edfe93aba146385769ff4820c" exitCode=0 Apr 22 15:08:24.690264 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:24.690190 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2jf6l" event={"ID":"6b9559b7-6b5b-4ecf-9655-4379447b15c7","Type":"ContainerDied","Data":"2dc4cedddf482fe1a3f77297ec7571f0eab2fd2edfe93aba146385769ff4820c"} Apr 22 15:08:24.691768 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:24.691745 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-fxjmk" event={"ID":"6cd30837-92d3-44c3-ba6a-9c84a22a7c8b","Type":"ContainerStarted","Data":"0cd4621403ba3c8feb895b673b6405e184eefd9483492c07faecec39deb09138"} Apr 22 15:08:24.693439 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:24.693411 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-8894fc9bd-nmq7h" event={"ID":"3d4eaf4b-2b28-4ad4-866f-ac541d15b00b","Type":"ContainerStarted","Data":"b6dace3f198b99d31cfe3cfcb5b3728b1b3be0586f6a712a2f5fbd2f0963ed01"} Apr 22 15:08:24.694893 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:24.694872 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-lrs9z" event={"ID":"4ccb076c-b0c3-40bd-b647-ef47ba1d8681","Type":"ContainerStarted","Data":"d30116ff9a339f85b1e0a2661decaf951ad61993bb29886d66b9df2d8d10596f"} Apr 22 15:08:24.697795 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:24.697703 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-585dfdc468-rxj2p" event={"ID":"c7b8b7c3-8b82-46c5-bb74-0c90f6448aff","Type":"ContainerStarted","Data":"49ec7cfdef76678998bfcceb426679b5d5ebb1f6cc510b35eb0a80986efde0f9"} Apr 22 15:08:24.700420 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:24.700268 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-s79j7_89e3aa95-bc39-4144-bc81-144160cf50eb/console-operator/0.log" Apr 22 15:08:24.700420 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:24.700299 2577 generic.go:358] "Generic (PLEG): container finished" podID="89e3aa95-bc39-4144-bc81-144160cf50eb" containerID="0b69510fe3d5f879aa76bc4724322f1045c41b10e8861c08cd1b24cb0981b5f2" exitCode=255 Apr 22 15:08:24.700420 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:24.700391 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-9d4b6777b-s79j7" event={"ID":"89e3aa95-bc39-4144-bc81-144160cf50eb","Type":"ContainerDied","Data":"0b69510fe3d5f879aa76bc4724322f1045c41b10e8861c08cd1b24cb0981b5f2"} Apr 22 15:08:24.700619 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:24.700595 2577 scope.go:117] "RemoveContainer" containerID="0b69510fe3d5f879aa76bc4724322f1045c41b10e8861c08cd1b24cb0981b5f2" Apr 22 15:08:24.702915 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:24.702874 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8lfsw" event={"ID":"68a481e1-e027-433e-84ea-cb8d42406d8f","Type":"ContainerStarted","Data":"39865eb28883168e0e25dafdcb7b2a834e53a001b6fc8aaa2d96375f12eb24fd"} Apr 22 15:08:24.704593 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:24.704570 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-bd6gc" event={"ID":"68da1e47-8873-416c-b2d9-d515d0985631","Type":"ContainerStarted","Data":"56b5e1edef4cccabab1a3457edd58092598b186b19dcc9a7f12d7b247689185c"} Apr 22 15:08:24.736728 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:24.735686 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-operator-585dfdc468-rxj2p" podStartSLOduration=16.327330034 podStartE2EDuration="25.735668893s" podCreationTimestamp="2026-04-22 15:07:59 +0000 UTC" firstStartedPulling="2026-04-22 15:08:14.373391092 +0000 UTC m=+33.430129157" lastFinishedPulling="2026-04-22 15:08:23.78172995 +0000 UTC m=+42.838468016" observedRunningTime="2026-04-22 15:08:24.734610893 +0000 UTC m=+43.791348980" watchObservedRunningTime="2026-04-22 15:08:24.735668893 +0000 UTC m=+43.792406979" Apr 22 15:08:24.752630 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:24.751229 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-8894fc9bd-nmq7h" podStartSLOduration=16.383584389 podStartE2EDuration="25.751213256s" podCreationTimestamp="2026-04-22 15:07:59 +0000 UTC" firstStartedPulling="2026-04-22 15:08:14.414077666 +0000 UTC m=+33.470815736" lastFinishedPulling="2026-04-22 15:08:23.781706526 +0000 UTC m=+42.838444603" observedRunningTime="2026-04-22 15:08:24.750610542 +0000 UTC m=+43.807348627" watchObservedRunningTime="2026-04-22 15:08:24.751213256 +0000 UTC m=+43.807951342" Apr 22 15:08:24.771359 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:24.769882 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-fxjmk" podStartSLOduration=16.433244616 podStartE2EDuration="25.769865361s" podCreationTimestamp="2026-04-22 15:07:59 +0000 UTC" firstStartedPulling="2026-04-22 15:08:14.445182239 +0000 UTC m=+33.501920319" lastFinishedPulling="2026-04-22 15:08:23.781802987 +0000 UTC m=+42.838541064" observedRunningTime="2026-04-22 15:08:24.769088465 +0000 UTC m=+43.825826551" watchObservedRunningTime="2026-04-22 15:08:24.769865361 +0000 UTC m=+43.826603446" Apr 22 15:08:24.809263 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:24.808145 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-lrs9z" podStartSLOduration=16.420623642 podStartE2EDuration="25.808125123s" podCreationTimestamp="2026-04-22 15:07:59 +0000 UTC" firstStartedPulling="2026-04-22 15:08:14.368248913 +0000 UTC m=+33.424986993" lastFinishedPulling="2026-04-22 15:08:23.755750405 +0000 UTC m=+42.812488474" observedRunningTime="2026-04-22 15:08:24.807954917 +0000 UTC m=+43.864693004" watchObservedRunningTime="2026-04-22 15:08:24.808125123 +0000 UTC m=+43.864863207" Apr 22 15:08:24.830801 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:24.829781 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-bd6gc" podStartSLOduration=16.458558752 podStartE2EDuration="25.829763512s" podCreationTimestamp="2026-04-22 15:07:59 +0000 UTC" firstStartedPulling="2026-04-22 15:08:14.416231045 +0000 UTC m=+33.472969114" lastFinishedPulling="2026-04-22 15:08:23.787435808 +0000 UTC m=+42.844173874" observedRunningTime="2026-04-22 15:08:24.828678165 +0000 UTC m=+43.885416250" watchObservedRunningTime="2026-04-22 15:08:24.829763512 +0000 UTC m=+43.886501597" Apr 22 15:08:24.850506 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:24.850006 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8lfsw" podStartSLOduration=35.407829484 podStartE2EDuration="43.849987414s" podCreationTimestamp="2026-04-22 15:07:41 +0000 UTC" firstStartedPulling="2026-04-22 15:08:15.718259268 +0000 UTC m=+34.774997333" lastFinishedPulling="2026-04-22 15:08:24.1604172 +0000 UTC m=+43.217155263" observedRunningTime="2026-04-22 15:08:24.849621733 +0000 UTC m=+43.906359819" watchObservedRunningTime="2026-04-22 15:08:24.849987414 +0000 UTC m=+43.906725501" Apr 22 15:08:25.711142 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:25.710981 2577 generic.go:358] "Generic (PLEG): container finished" podID="6b9559b7-6b5b-4ecf-9655-4379447b15c7" containerID="c41eb8f0cdbce6176ee085e4c3071b9e6f23ff495323afed8b73faefac9a9913" exitCode=0 Apr 22 15:08:25.711142 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:25.711068 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2jf6l" event={"ID":"6b9559b7-6b5b-4ecf-9655-4379447b15c7","Type":"ContainerDied","Data":"c41eb8f0cdbce6176ee085e4c3071b9e6f23ff495323afed8b73faefac9a9913"} Apr 22 15:08:25.713893 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:25.713865 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-s79j7_89e3aa95-bc39-4144-bc81-144160cf50eb/console-operator/1.log" Apr 22 15:08:25.714254 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:25.714236 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-s79j7_89e3aa95-bc39-4144-bc81-144160cf50eb/console-operator/0.log" Apr 22 15:08:25.714418 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:25.714269 2577 generic.go:358] "Generic (PLEG): container finished" podID="89e3aa95-bc39-4144-bc81-144160cf50eb" containerID="167f84c1796ba5279002b3c191547a04e3ff7b10d9f1e05c9c9be3467b5f9d98" exitCode=255 Apr 22 15:08:25.715050 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:25.715019 2577 scope.go:117] "RemoveContainer" containerID="167f84c1796ba5279002b3c191547a04e3ff7b10d9f1e05c9c9be3467b5f9d98" Apr 22 15:08:25.715202 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:25.715177 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-9d4b6777b-s79j7_openshift-console-operator(89e3aa95-bc39-4144-bc81-144160cf50eb)\"" pod="openshift-console-operator/console-operator-9d4b6777b-s79j7" podUID="89e3aa95-bc39-4144-bc81-144160cf50eb" Apr 22 15:08:25.715256 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:25.715212 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-9d4b6777b-s79j7" event={"ID":"89e3aa95-bc39-4144-bc81-144160cf50eb","Type":"ContainerDied","Data":"167f84c1796ba5279002b3c191547a04e3ff7b10d9f1e05c9c9be3467b5f9d98"} Apr 22 15:08:25.715256 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:25.715249 2577 scope.go:117] "RemoveContainer" containerID="0b69510fe3d5f879aa76bc4724322f1045c41b10e8861c08cd1b24cb0981b5f2" Apr 22 15:08:25.716160 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:25.716143 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:08:25.835957 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:25.835699 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-74bb7799d9-c866s"] Apr 22 15:08:25.858418 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:25.858319 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-74bb7799d9-c866s"] Apr 22 15:08:25.858563 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:25.858484 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-74bb7799d9-c866s" Apr 22 15:08:25.861251 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:25.861227 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-zwps6\"" Apr 22 15:08:25.861613 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:25.861592 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Apr 22 15:08:25.861837 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:25.861821 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Apr 22 15:08:25.880068 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:25.880042 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gv88\" (UniqueName: \"kubernetes.io/projected/5cd9c342-f4f7-49d3-be86-3617d485feb5-kube-api-access-4gv88\") pod \"migrator-74bb7799d9-c866s\" (UID: \"5cd9c342-f4f7-49d3-be86-3617d485feb5\") " pod="openshift-kube-storage-version-migrator/migrator-74bb7799d9-c866s" Apr 22 15:08:25.981617 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:25.981588 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4gv88\" (UniqueName: \"kubernetes.io/projected/5cd9c342-f4f7-49d3-be86-3617d485feb5-kube-api-access-4gv88\") pod \"migrator-74bb7799d9-c866s\" (UID: \"5cd9c342-f4f7-49d3-be86-3617d485feb5\") " pod="openshift-kube-storage-version-migrator/migrator-74bb7799d9-c866s" Apr 22 15:08:25.991096 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:25.991071 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gv88\" (UniqueName: \"kubernetes.io/projected/5cd9c342-f4f7-49d3-be86-3617d485feb5-kube-api-access-4gv88\") pod \"migrator-74bb7799d9-c866s\" (UID: \"5cd9c342-f4f7-49d3-be86-3617d485feb5\") " pod="openshift-kube-storage-version-migrator/migrator-74bb7799d9-c866s" Apr 22 15:08:26.172425 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:26.172390 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-74bb7799d9-c866s" Apr 22 15:08:26.295579 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:26.295539 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-74bb7799d9-c866s"] Apr 22 15:08:26.299964 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:08:26.299931 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cd9c342_f4f7_49d3_be86_3617d485feb5.slice/crio-2309e30a2215cb5cf0805b00a75752c835e5be8bb906ed1fc8b71caee74ad58b WatchSource:0}: Error finding container 2309e30a2215cb5cf0805b00a75752c835e5be8bb906ed1fc8b71caee74ad58b: Status 404 returned error can't find the container with id 2309e30a2215cb5cf0805b00a75752c835e5be8bb906ed1fc8b71caee74ad58b Apr 22 15:08:26.720841 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:26.720759 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2jf6l" event={"ID":"6b9559b7-6b5b-4ecf-9655-4379447b15c7","Type":"ContainerStarted","Data":"697cd8c22be0933bea7435211af355afe026864cffad969ec93aa41363a17090"} Apr 22 15:08:26.722087 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:26.722068 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-s79j7_89e3aa95-bc39-4144-bc81-144160cf50eb/console-operator/1.log" Apr 22 15:08:26.722404 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:26.722385 2577 scope.go:117] "RemoveContainer" containerID="167f84c1796ba5279002b3c191547a04e3ff7b10d9f1e05c9c9be3467b5f9d98" Apr 22 15:08:26.722571 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:26.722552 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-9d4b6777b-s79j7_openshift-console-operator(89e3aa95-bc39-4144-bc81-144160cf50eb)\"" pod="openshift-console-operator/console-operator-9d4b6777b-s79j7" podUID="89e3aa95-bc39-4144-bc81-144160cf50eb" Apr 22 15:08:26.723049 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:26.723025 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-74bb7799d9-c866s" event={"ID":"5cd9c342-f4f7-49d3-be86-3617d485feb5","Type":"ContainerStarted","Data":"2309e30a2215cb5cf0805b00a75752c835e5be8bb906ed1fc8b71caee74ad58b"} Apr 22 15:08:26.753682 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:26.753642 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2jf6l" podStartSLOduration=4.364791582 podStartE2EDuration="45.753631796s" podCreationTimestamp="2026-04-22 15:07:41 +0000 UTC" firstStartedPulling="2026-04-22 15:07:42.757763565 +0000 UTC m=+1.814501629" lastFinishedPulling="2026-04-22 15:08:24.146603777 +0000 UTC m=+43.203341843" observedRunningTime="2026-04-22 15:08:26.749678009 +0000 UTC m=+45.806416096" watchObservedRunningTime="2026-04-22 15:08:26.753631796 +0000 UTC m=+45.810369881" Apr 22 15:08:28.246369 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:28.246317 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-865cb79987-pgmlp"] Apr 22 15:08:28.271114 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:28.271085 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-865cb79987-pgmlp"] Apr 22 15:08:28.271265 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:28.271214 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-865cb79987-pgmlp" Apr 22 15:08:28.274856 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:28.274835 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-vgsqf\"" Apr 22 15:08:28.274978 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:28.274835 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Apr 22 15:08:28.275157 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:28.275135 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Apr 22 15:08:28.275431 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:28.275409 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Apr 22 15:08:28.275521 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:28.275481 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Apr 22 15:08:28.305592 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:28.305565 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7f71ee10-0a7c-4fd9-80ee-56c493e3034e-signing-cabundle\") pod \"service-ca-865cb79987-pgmlp\" (UID: \"7f71ee10-0a7c-4fd9-80ee-56c493e3034e\") " pod="openshift-service-ca/service-ca-865cb79987-pgmlp" Apr 22 15:08:28.305744 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:28.305717 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsjf7\" (UniqueName: \"kubernetes.io/projected/7f71ee10-0a7c-4fd9-80ee-56c493e3034e-kube-api-access-fsjf7\") pod \"service-ca-865cb79987-pgmlp\" (UID: \"7f71ee10-0a7c-4fd9-80ee-56c493e3034e\") " pod="openshift-service-ca/service-ca-865cb79987-pgmlp" Apr 22 15:08:28.305892 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:28.305874 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7f71ee10-0a7c-4fd9-80ee-56c493e3034e-signing-key\") pod \"service-ca-865cb79987-pgmlp\" (UID: \"7f71ee10-0a7c-4fd9-80ee-56c493e3034e\") " pod="openshift-service-ca/service-ca-865cb79987-pgmlp" Apr 22 15:08:28.407377 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:28.407322 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fsjf7\" (UniqueName: \"kubernetes.io/projected/7f71ee10-0a7c-4fd9-80ee-56c493e3034e-kube-api-access-fsjf7\") pod \"service-ca-865cb79987-pgmlp\" (UID: \"7f71ee10-0a7c-4fd9-80ee-56c493e3034e\") " pod="openshift-service-ca/service-ca-865cb79987-pgmlp" Apr 22 15:08:28.407544 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:28.407475 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7f71ee10-0a7c-4fd9-80ee-56c493e3034e-signing-key\") pod \"service-ca-865cb79987-pgmlp\" (UID: \"7f71ee10-0a7c-4fd9-80ee-56c493e3034e\") " pod="openshift-service-ca/service-ca-865cb79987-pgmlp" Apr 22 15:08:28.407544 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:28.407515 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7f71ee10-0a7c-4fd9-80ee-56c493e3034e-signing-cabundle\") pod \"service-ca-865cb79987-pgmlp\" (UID: \"7f71ee10-0a7c-4fd9-80ee-56c493e3034e\") " pod="openshift-service-ca/service-ca-865cb79987-pgmlp" Apr 22 15:08:28.408204 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:28.408179 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7f71ee10-0a7c-4fd9-80ee-56c493e3034e-signing-cabundle\") pod \"service-ca-865cb79987-pgmlp\" (UID: \"7f71ee10-0a7c-4fd9-80ee-56c493e3034e\") " pod="openshift-service-ca/service-ca-865cb79987-pgmlp" Apr 22 15:08:28.410156 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:28.410132 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7f71ee10-0a7c-4fd9-80ee-56c493e3034e-signing-key\") pod \"service-ca-865cb79987-pgmlp\" (UID: \"7f71ee10-0a7c-4fd9-80ee-56c493e3034e\") " pod="openshift-service-ca/service-ca-865cb79987-pgmlp" Apr 22 15:08:28.419933 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:28.419912 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsjf7\" (UniqueName: \"kubernetes.io/projected/7f71ee10-0a7c-4fd9-80ee-56c493e3034e-kube-api-access-fsjf7\") pod \"service-ca-865cb79987-pgmlp\" (UID: \"7f71ee10-0a7c-4fd9-80ee-56c493e3034e\") " pod="openshift-service-ca/service-ca-865cb79987-pgmlp" Apr 22 15:08:28.580299 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:28.580220 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-865cb79987-pgmlp" Apr 22 15:08:28.698697 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:28.698667 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xlxs5_74c26902-8853-4def-9827-b429513b3de8/dns-node-resolver/0.log" Apr 22 15:08:29.693179 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:29.693153 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g8wmb_9dc3cd64-355a-4b4b-a5e1-5d165978d8a3/node-ca/0.log" Apr 22 15:08:29.719569 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:29.719542 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3b905d0-a65d-4615-8789-ccfb785189a3-samples-operator-tls\") pod \"cluster-samples-operator-6dc5bdb6b4-k8gnz\" (UID: \"b3b905d0-a65d-4615-8789-ccfb785189a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-k8gnz" Apr 22 15:08:29.719711 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:29.719579 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc334588-fb2e-4ac6-bb34-460dad264804-metrics-tls\") pod \"dns-default-4zrz9\" (UID: \"dc334588-fb2e-4ac6-bb34-460dad264804\") " pod="openshift-dns/dns-default-4zrz9" Apr 22 15:08:29.719711 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:29.719601 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-tls\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:29.719711 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:29.719630 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19d55ae0-1958-4a44-a653-e45e72f6e6e2-metrics-certs\") pod \"router-default-594fffb56b-msk5s\" (UID: \"19d55ae0-1958-4a44-a653-e45e72f6e6e2\") " pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:29.719711 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:29.719668 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eb3d139a-27c4-4fa7-8809-a33c13551872-cert\") pod \"ingress-canary-4qmd5\" (UID: \"eb3d139a-27c4-4fa7-8809-a33c13551872\") " pod="openshift-ingress-canary/ingress-canary-4qmd5" Apr 22 15:08:29.719711 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:29.719681 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Apr 22 15:08:29.719711 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:29.719688 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19d55ae0-1958-4a44-a653-e45e72f6e6e2-service-ca-bundle\") pod \"router-default-594fffb56b-msk5s\" (UID: \"19d55ae0-1958-4a44-a653-e45e72f6e6e2\") " pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:29.719711 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:29.719694 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 22 15:08:29.720074 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:29.719720 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a84180fa-5ffa-4cc6-80cc-0cd55558a345-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-75587bd455-29bzc\" (UID: \"a84180fa-5ffa-4cc6-80cc-0cd55558a345\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-29bzc" Apr 22 15:08:29.720074 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:29.719744 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b905d0-a65d-4615-8789-ccfb785189a3-samples-operator-tls podName:b3b905d0-a65d-4615-8789-ccfb785189a3 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:45.719723844 +0000 UTC m=+64.776461907 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3b905d0-a65d-4615-8789-ccfb785189a3-samples-operator-tls") pod "cluster-samples-operator-6dc5bdb6b4-k8gnz" (UID: "b3b905d0-a65d-4615-8789-ccfb785189a3") : secret "samples-operator-tls" not found Apr 22 15:08:29.720074 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:29.719766 2577 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Apr 22 15:08:29.720074 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:29.719775 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Apr 22 15:08:29.720074 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:29.719783 2577 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64fd688846-mkdtj: secret "image-registry-tls" not found Apr 22 15:08:29.720074 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:29.719795 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Apr 22 15:08:29.720074 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:29.719793 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc334588-fb2e-4ac6-bb34-460dad264804-metrics-tls podName:dc334588-fb2e-4ac6-bb34-460dad264804 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:45.719762911 +0000 UTC m=+64.776500979 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dc334588-fb2e-4ac6-bb34-460dad264804-metrics-tls") pod "dns-default-4zrz9" (UID: "dc334588-fb2e-4ac6-bb34-460dad264804") : secret "dns-default-metrics-tls" not found Apr 22 15:08:29.720074 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:29.719812 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 22 15:08:29.720074 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:29.719829 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-tls podName:bbd7337b-38ca-4006-bd9f-d822e5410b7c nodeName:}" failed. No retries permitted until 2026-04-22 15:08:45.719817067 +0000 UTC m=+64.776555137 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-tls") pod "image-registry-64fd688846-mkdtj" (UID: "bbd7337b-38ca-4006-bd9f-d822e5410b7c") : secret "image-registry-tls" not found Apr 22 15:08:29.720074 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:29.719848 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/19d55ae0-1958-4a44-a653-e45e72f6e6e2-service-ca-bundle podName:19d55ae0-1958-4a44-a653-e45e72f6e6e2 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:45.719838723 +0000 UTC m=+64.776576793 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/19d55ae0-1958-4a44-a653-e45e72f6e6e2-service-ca-bundle") pod "router-default-594fffb56b-msk5s" (UID: "19d55ae0-1958-4a44-a653-e45e72f6e6e2") : configmap references non-existent config key: service-ca.crt Apr 22 15:08:29.720074 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:29.719865 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19d55ae0-1958-4a44-a653-e45e72f6e6e2-metrics-certs podName:19d55ae0-1958-4a44-a653-e45e72f6e6e2 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:45.719856802 +0000 UTC m=+64.776594871 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19d55ae0-1958-4a44-a653-e45e72f6e6e2-metrics-certs") pod "router-default-594fffb56b-msk5s" (UID: "19d55ae0-1958-4a44-a653-e45e72f6e6e2") : secret "router-metrics-certs-default" not found Apr 22 15:08:29.720074 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:29.719882 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a84180fa-5ffa-4cc6-80cc-0cd55558a345-cluster-monitoring-operator-tls podName:a84180fa-5ffa-4cc6-80cc-0cd55558a345 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:45.719874339 +0000 UTC m=+64.776612412 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a84180fa-5ffa-4cc6-80cc-0cd55558a345-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-75587bd455-29bzc" (UID: "a84180fa-5ffa-4cc6-80cc-0cd55558a345") : secret "cluster-monitoring-operator-tls" not found Apr 22 15:08:29.720074 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:29.719904 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb3d139a-27c4-4fa7-8809-a33c13551872-cert podName:eb3d139a-27c4-4fa7-8809-a33c13551872 nodeName:}" failed. No retries permitted until 2026-04-22 15:08:45.719891143 +0000 UTC m=+64.776629213 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/eb3d139a-27c4-4fa7-8809-a33c13551872-cert") pod "ingress-canary-4qmd5" (UID: "eb3d139a-27c4-4fa7-8809-a33c13551872") : secret "canary-serving-cert" not found Apr 22 15:08:29.720074 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:29.719961 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/acbfa8e1-30e3-467a-b70e-c9e23ebddcbe-networking-console-plugin-cert\") pod \"networking-console-plugin-cb95c66f6-6ljs8\" (UID: \"acbfa8e1-30e3-467a-b70e-c9e23ebddcbe\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-6ljs8" Apr 22 15:08:29.720591 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:29.720006 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-tls\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:29.720591 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:29.720094 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Apr 22 15:08:29.720591 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:29.720125 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acbfa8e1-30e3-467a-b70e-c9e23ebddcbe-networking-console-plugin-cert podName:acbfa8e1-30e3-467a-b70e-c9e23ebddcbe nodeName:}" failed. No retries permitted until 2026-04-22 15:08:45.7201164 +0000 UTC m=+64.776854462 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/acbfa8e1-30e3-467a-b70e-c9e23ebddcbe-networking-console-plugin-cert") pod "networking-console-plugin-cb95c66f6-6ljs8" (UID: "acbfa8e1-30e3-467a-b70e-c9e23ebddcbe") : secret "networking-console-plugin-cert" not found Apr 22 15:08:29.720591 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:29.720127 2577 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Apr 22 15:08:29.720591 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:29.720140 2577 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8bc6dfb5-5vmjx: secret "image-registry-tls" not found Apr 22 15:08:29.720591 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:29.720216 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-tls podName:df8a3082-b089-41db-bbd6-1e6b09fd544b nodeName:}" failed. No retries permitted until 2026-04-22 15:08:45.720203616 +0000 UTC m=+64.776941699 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-tls") pod "image-registry-8bc6dfb5-5vmjx" (UID: "df8a3082-b089-41db-bbd6-1e6b09fd544b") : secret "image-registry-tls" not found Apr 22 15:08:29.852635 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:29.852610 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-865cb79987-pgmlp"] Apr 22 15:08:29.856455 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:08:29.856428 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f71ee10_0a7c_4fd9_80ee_56c493e3034e.slice/crio-dcb627040cadc65b26f77fca3756fd56321cd722bad7530a9d72a14fed01e1d5 WatchSource:0}: Error finding container dcb627040cadc65b26f77fca3756fd56321cd722bad7530a9d72a14fed01e1d5: Status 404 returned error can't find the container with id dcb627040cadc65b26f77fca3756fd56321cd722bad7530a9d72a14fed01e1d5 Apr 22 15:08:30.696520 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:30.696487 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6769c5d45-fxjmk_6cd30837-92d3-44c3-ba6a-9c84a22a7c8b/kube-storage-version-migrator-operator/0.log" Apr 22 15:08:30.738399 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:30.738361 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-74bb7799d9-c866s" event={"ID":"5cd9c342-f4f7-49d3-be86-3617d485feb5","Type":"ContainerStarted","Data":"62d5b24e6915a8beb1905adb4d579ee7d8ce88f1fb01c34642a687a87483cff6"} Apr 22 15:08:30.738399 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:30.738403 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-74bb7799d9-c866s" event={"ID":"5cd9c342-f4f7-49d3-be86-3617d485feb5","Type":"ContainerStarted","Data":"ac9abc3fd601df82497a52a47f4554c5cdba82d78532407a549a51c35ab8c852"} Apr 22 15:08:30.739751 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:30.739723 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-865cb79987-pgmlp" event={"ID":"7f71ee10-0a7c-4fd9-80ee-56c493e3034e","Type":"ContainerStarted","Data":"398ddc656ffb48ff3071c2d1cb5794af1e9b50f1be1ccb97c1e2e984bf9981c3"} Apr 22 15:08:30.739906 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:30.739756 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-865cb79987-pgmlp" event={"ID":"7f71ee10-0a7c-4fd9-80ee-56c493e3034e","Type":"ContainerStarted","Data":"dcb627040cadc65b26f77fca3756fd56321cd722bad7530a9d72a14fed01e1d5"} Apr 22 15:08:30.741010 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:30.740990 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8l6g2" event={"ID":"fb9ab245-62d7-4d19-93cd-34d5f9595d3d","Type":"ContainerStarted","Data":"66b36c9a7ce9b62cc15438f6d0c4178190e804a01e2bbd097962e3dfa861d952"} Apr 22 15:08:30.762440 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:30.762392 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-74bb7799d9-c866s" podStartSLOduration=2.333396162 podStartE2EDuration="5.762379358s" podCreationTimestamp="2026-04-22 15:08:25 +0000 UTC" firstStartedPulling="2026-04-22 15:08:26.302432595 +0000 UTC m=+45.359170665" lastFinishedPulling="2026-04-22 15:08:29.731415799 +0000 UTC m=+48.788153861" observedRunningTime="2026-04-22 15:08:30.759787527 +0000 UTC m=+49.816525612" watchObservedRunningTime="2026-04-22 15:08:30.762379358 +0000 UTC m=+49.819117442" Apr 22 15:08:30.779980 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:30.779930 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8l6g2" podStartSLOduration=8.698371553 podStartE2EDuration="14.779917747s" podCreationTimestamp="2026-04-22 15:08:16 +0000 UTC" firstStartedPulling="2026-04-22 15:08:24.055571457 +0000 UTC m=+43.112309520" lastFinishedPulling="2026-04-22 15:08:30.137117637 +0000 UTC m=+49.193855714" observedRunningTime="2026-04-22 15:08:30.778647671 +0000 UTC m=+49.835385757" watchObservedRunningTime="2026-04-22 15:08:30.779917747 +0000 UTC m=+49.836655832" Apr 22 15:08:30.799190 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:30.799130 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-865cb79987-pgmlp" podStartSLOduration=2.799105883 podStartE2EDuration="2.799105883s" podCreationTimestamp="2026-04-22 15:08:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-22 15:08:30.799082711 +0000 UTC m=+49.855820792" watchObservedRunningTime="2026-04-22 15:08:30.799105883 +0000 UTC m=+49.855843967" Apr 22 15:08:34.175246 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:34.175200 2577 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-9d4b6777b-s79j7" Apr 22 15:08:34.175246 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:34.175242 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-9d4b6777b-s79j7" Apr 22 15:08:34.175797 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:34.175602 2577 scope.go:117] "RemoveContainer" containerID="167f84c1796ba5279002b3c191547a04e3ff7b10d9f1e05c9c9be3467b5f9d98" Apr 22 15:08:34.175797 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:08:34.175754 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-9d4b6777b-s79j7_openshift-console-operator(89e3aa95-bc39-4144-bc81-144160cf50eb)\"" pod="openshift-console-operator/console-operator-9d4b6777b-s79j7" podUID="89e3aa95-bc39-4144-bc81-144160cf50eb" Apr 22 15:08:39.646411 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:39.646383 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s6qsb" Apr 22 15:08:45.503878 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.503839 2577 scope.go:117] "RemoveContainer" containerID="167f84c1796ba5279002b3c191547a04e3ff7b10d9f1e05c9c9be3467b5f9d98" Apr 22 15:08:45.766093 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.765994 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a84180fa-5ffa-4cc6-80cc-0cd55558a345-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-75587bd455-29bzc\" (UID: \"a84180fa-5ffa-4cc6-80cc-0cd55558a345\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-29bzc" Apr 22 15:08:45.766093 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.766052 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/acbfa8e1-30e3-467a-b70e-c9e23ebddcbe-networking-console-plugin-cert\") pod \"networking-console-plugin-cb95c66f6-6ljs8\" (UID: \"acbfa8e1-30e3-467a-b70e-c9e23ebddcbe\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-6ljs8" Apr 22 15:08:45.766093 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.766074 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-tls\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:45.766093 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.766096 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3b905d0-a65d-4615-8789-ccfb785189a3-samples-operator-tls\") pod \"cluster-samples-operator-6dc5bdb6b4-k8gnz\" (UID: \"b3b905d0-a65d-4615-8789-ccfb785189a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-k8gnz" Apr 22 15:08:45.766454 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.766119 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc334588-fb2e-4ac6-bb34-460dad264804-metrics-tls\") pod \"dns-default-4zrz9\" (UID: \"dc334588-fb2e-4ac6-bb34-460dad264804\") " pod="openshift-dns/dns-default-4zrz9" Apr 22 15:08:45.766454 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.766146 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-tls\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:45.766454 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.766300 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19d55ae0-1958-4a44-a653-e45e72f6e6e2-metrics-certs\") pod \"router-default-594fffb56b-msk5s\" (UID: \"19d55ae0-1958-4a44-a653-e45e72f6e6e2\") " pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:45.766454 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.766436 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eb3d139a-27c4-4fa7-8809-a33c13551872-cert\") pod \"ingress-canary-4qmd5\" (UID: \"eb3d139a-27c4-4fa7-8809-a33c13551872\") " pod="openshift-ingress-canary/ingress-canary-4qmd5" Apr 22 15:08:45.766657 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.766470 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19d55ae0-1958-4a44-a653-e45e72f6e6e2-service-ca-bundle\") pod \"router-default-594fffb56b-msk5s\" (UID: \"19d55ae0-1958-4a44-a653-e45e72f6e6e2\") " pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:45.767256 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.767228 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19d55ae0-1958-4a44-a653-e45e72f6e6e2-service-ca-bundle\") pod \"router-default-594fffb56b-msk5s\" (UID: \"19d55ae0-1958-4a44-a653-e45e72f6e6e2\") " pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:45.769055 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.769000 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a84180fa-5ffa-4cc6-80cc-0cd55558a345-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-75587bd455-29bzc\" (UID: \"a84180fa-5ffa-4cc6-80cc-0cd55558a345\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-29bzc" Apr 22 15:08:45.769166 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.769094 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3b905d0-a65d-4615-8789-ccfb785189a3-samples-operator-tls\") pod \"cluster-samples-operator-6dc5bdb6b4-k8gnz\" (UID: \"b3b905d0-a65d-4615-8789-ccfb785189a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-k8gnz" Apr 22 15:08:45.769166 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.769117 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-tls\") pod \"image-registry-64fd688846-mkdtj\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:45.769384 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.769363 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-tls\") pod \"image-registry-8bc6dfb5-5vmjx\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:45.769458 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.769370 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/acbfa8e1-30e3-467a-b70e-c9e23ebddcbe-networking-console-plugin-cert\") pod \"networking-console-plugin-cb95c66f6-6ljs8\" (UID: \"acbfa8e1-30e3-467a-b70e-c9e23ebddcbe\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-6ljs8" Apr 22 15:08:45.770068 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.770049 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19d55ae0-1958-4a44-a653-e45e72f6e6e2-metrics-certs\") pod \"router-default-594fffb56b-msk5s\" (UID: \"19d55ae0-1958-4a44-a653-e45e72f6e6e2\") " pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:45.770373 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.770354 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eb3d139a-27c4-4fa7-8809-a33c13551872-cert\") pod \"ingress-canary-4qmd5\" (UID: \"eb3d139a-27c4-4fa7-8809-a33c13551872\") " pod="openshift-ingress-canary/ingress-canary-4qmd5" Apr 22 15:08:45.770878 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.770861 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc334588-fb2e-4ac6-bb34-460dad264804-metrics-tls\") pod \"dns-default-4zrz9\" (UID: \"dc334588-fb2e-4ac6-bb34-460dad264804\") " pod="openshift-dns/dns-default-4zrz9" Apr 22 15:08:45.792481 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.792461 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-s79j7_89e3aa95-bc39-4144-bc81-144160cf50eb/console-operator/1.log" Apr 22 15:08:45.792573 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.792510 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-9d4b6777b-s79j7" event={"ID":"89e3aa95-bc39-4144-bc81-144160cf50eb","Type":"ContainerStarted","Data":"10ca1ef8bd68519edaac85e999fce9ad7024086a520eea71e00880804799d996"} Apr 22 15:08:45.792777 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.792760 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-9d4b6777b-s79j7" Apr 22 15:08:45.817509 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.817467 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-9d4b6777b-s79j7" podStartSLOduration=37.467474308 podStartE2EDuration="46.817456062s" podCreationTimestamp="2026-04-22 15:07:59 +0000 UTC" firstStartedPulling="2026-04-22 15:08:14.431745038 +0000 UTC m=+33.488483101" lastFinishedPulling="2026-04-22 15:08:23.781726779 +0000 UTC m=+42.838464855" observedRunningTime="2026-04-22 15:08:45.815887902 +0000 UTC m=+64.872625999" watchObservedRunningTime="2026-04-22 15:08:45.817456062 +0000 UTC m=+64.874194146" Apr 22 15:08:45.921599 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.921572 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jdzns\"" Apr 22 15:08:45.929569 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.929546 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:45.941892 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.941867 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-k56g5\"" Apr 22 15:08:45.950121 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.950097 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-k8gnz" Apr 22 15:08:45.993167 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.992936 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-mcrp8\"" Apr 22 15:08:45.997428 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:45.997183 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:46.001289 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.000363 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-cb95c66f6-6ljs8" Apr 22 15:08:46.017199 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.016927 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-r6fws\"" Apr 22 15:08:46.026349 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.023377 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:46.026349 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.024103 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-j8ksr\"" Apr 22 15:08:46.028612 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.028405 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-62nkc\"" Apr 22 15:08:46.034065 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.032232 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-29bzc" Apr 22 15:08:46.037791 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.035265 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xqxh9\"" Apr 22 15:08:46.040176 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.040135 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4zrz9" Apr 22 15:08:46.044726 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.043484 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4qmd5" Apr 22 15:08:46.085983 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.085931 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8bc6dfb5-5vmjx"] Apr 22 15:08:46.121627 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.121505 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-k8gnz"] Apr 22 15:08:46.176217 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.175817 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/487f23b6-9470-41e9-abbd-295b6d317b10-metrics-certs\") pod \"network-metrics-daemon-z9ckr\" (UID: \"487f23b6-9470-41e9-abbd-295b6d317b10\") " pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:08:46.181101 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.180700 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Apr 22 15:08:46.192031 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.191964 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/487f23b6-9470-41e9-abbd-295b6d317b10-metrics-certs\") pod \"network-metrics-daemon-z9ckr\" (UID: \"487f23b6-9470-41e9-abbd-295b6d317b10\") " pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:08:46.290831 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.290722 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-594fffb56b-msk5s"] Apr 22 15:08:46.294888 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:08:46.294858 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19d55ae0_1958_4a44_a653_e45e72f6e6e2.slice/crio-0947eb0ee0a7781d1209091beced00727b2e46c5b6fc148de91f9df189cc59b4 WatchSource:0}: Error finding container 0947eb0ee0a7781d1209091beced00727b2e46c5b6fc148de91f9df189cc59b4: Status 404 returned error can't find the container with id 0947eb0ee0a7781d1209091beced00727b2e46c5b6fc148de91f9df189cc59b4 Apr 22 15:08:46.298290 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.297915 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-cb95c66f6-6ljs8"] Apr 22 15:08:46.301955 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:08:46.301929 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacbfa8e1_30e3_467a_b70e_c9e23ebddcbe.slice/crio-f7d4572f2ca5a05ce1bf0b765785f6f65b12461d509ddcd48b6a2790b23d1d1a WatchSource:0}: Error finding container f7d4572f2ca5a05ce1bf0b765785f6f65b12461d509ddcd48b6a2790b23d1d1a: Status 404 returned error can't find the container with id f7d4572f2ca5a05ce1bf0b765785f6f65b12461d509ddcd48b6a2790b23d1d1a Apr 22 15:08:46.313985 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.313485 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64fd688846-mkdtj"] Apr 22 15:08:46.319007 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:08:46.318954 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbd7337b_38ca_4006_bd9f_d822e5410b7c.slice/crio-c7b3d843624c74217d841200f657b955d1c8602bc4515b2e257b9285464b75f3 WatchSource:0}: Error finding container c7b3d843624c74217d841200f657b955d1c8602bc4515b2e257b9285464b75f3: Status 404 returned error can't find the container with id c7b3d843624c74217d841200f657b955d1c8602bc4515b2e257b9285464b75f3 Apr 22 15:08:46.360198 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.360170 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4qmd5"] Apr 22 15:08:46.363853 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:08:46.363817 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb3d139a_27c4_4fa7_8809_a33c13551872.slice/crio-41c426a2e919164269a3d2cc3b34e85c752d71c0fdb62a31f61720be06bf20f2 WatchSource:0}: Error finding container 41c426a2e919164269a3d2cc3b34e85c752d71c0fdb62a31f61720be06bf20f2: Status 404 returned error can't find the container with id 41c426a2e919164269a3d2cc3b34e85c752d71c0fdb62a31f61720be06bf20f2 Apr 22 15:08:46.435750 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.435561 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pdgcv\"" Apr 22 15:08:46.443835 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.443795 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z9ckr" Apr 22 15:08:46.546413 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.538116 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-75587bd455-29bzc"] Apr 22 15:08:46.546413 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.542904 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4zrz9"] Apr 22 15:08:46.555965 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:08:46.555914 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc334588_fb2e_4ac6_bb34_460dad264804.slice/crio-02e1a1dcf7c6880ff194c141f9d4e661c07526c3f56b04c2ed126fa1f021ad1e WatchSource:0}: Error finding container 02e1a1dcf7c6880ff194c141f9d4e661c07526c3f56b04c2ed126fa1f021ad1e: Status 404 returned error can't find the container with id 02e1a1dcf7c6880ff194c141f9d4e661c07526c3f56b04c2ed126fa1f021ad1e Apr 22 15:08:46.608596 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.608564 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-z9ckr"] Apr 22 15:08:46.615169 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:08:46.615136 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod487f23b6_9470_41e9_abbd_295b6d317b10.slice/crio-897ca3b74563e53ac1de366a47ad51c877b3e6e5de1118c9c4c0307ee93c94a8 WatchSource:0}: Error finding container 897ca3b74563e53ac1de366a47ad51c877b3e6e5de1118c9c4c0307ee93c94a8: Status 404 returned error can't find the container with id 897ca3b74563e53ac1de366a47ad51c877b3e6e5de1118c9c4c0307ee93c94a8 Apr 22 15:08:46.694698 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.694667 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-9d4b6777b-s79j7" Apr 22 15:08:46.796933 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.796856 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-z9ckr" event={"ID":"487f23b6-9470-41e9-abbd-295b6d317b10","Type":"ContainerStarted","Data":"897ca3b74563e53ac1de366a47ad51c877b3e6e5de1118c9c4c0307ee93c94a8"} Apr 22 15:08:46.797928 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.797900 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-k8gnz" event={"ID":"b3b905d0-a65d-4615-8789-ccfb785189a3","Type":"ContainerStarted","Data":"cecf5ec13043dbca53de095ab4c859f13e5c25ac366e615492a27436d9c5c26e"} Apr 22 15:08:46.799030 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.799007 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-29bzc" event={"ID":"a84180fa-5ffa-4cc6-80cc-0cd55558a345","Type":"ContainerStarted","Data":"57462569d4c156a0694972d42292f08bc6618f57bb2396678240a7b44bfc986c"} Apr 22 15:08:46.800026 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.800004 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4qmd5" event={"ID":"eb3d139a-27c4-4fa7-8809-a33c13551872","Type":"ContainerStarted","Data":"41c426a2e919164269a3d2cc3b34e85c752d71c0fdb62a31f61720be06bf20f2"} Apr 22 15:08:46.801116 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.801096 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4zrz9" event={"ID":"dc334588-fb2e-4ac6-bb34-460dad264804","Type":"ContainerStarted","Data":"02e1a1dcf7c6880ff194c141f9d4e661c07526c3f56b04c2ed126fa1f021ad1e"} Apr 22 15:08:46.802401 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.802354 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" event={"ID":"df8a3082-b089-41db-bbd6-1e6b09fd544b","Type":"ContainerStarted","Data":"4ffd314e0f29d8121f4889a51e31bb30188bec61c987c9b06a49cb1a947eae99"} Apr 22 15:08:46.802492 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.802416 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" event={"ID":"df8a3082-b089-41db-bbd6-1e6b09fd544b","Type":"ContainerStarted","Data":"286a2f668c0264dea55ca6877a278198fb19e73fa5919971aa205bc35e418aec"} Apr 22 15:08:46.802548 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.802528 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:08:46.808883 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.808840 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-594fffb56b-msk5s" event={"ID":"19d55ae0-1958-4a44-a653-e45e72f6e6e2","Type":"ContainerStarted","Data":"3f10b0af3722cf37797fa446d40bd5095c34c16855c38b16a240950761ceb32b"} Apr 22 15:08:46.808983 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.808892 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-594fffb56b-msk5s" event={"ID":"19d55ae0-1958-4a44-a653-e45e72f6e6e2","Type":"ContainerStarted","Data":"0947eb0ee0a7781d1209091beced00727b2e46c5b6fc148de91f9df189cc59b4"} Apr 22 15:08:46.809941 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.809919 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-cb95c66f6-6ljs8" event={"ID":"acbfa8e1-30e3-467a-b70e-c9e23ebddcbe","Type":"ContainerStarted","Data":"f7d4572f2ca5a05ce1bf0b765785f6f65b12461d509ddcd48b6a2790b23d1d1a"} Apr 22 15:08:46.811989 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.811927 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64fd688846-mkdtj" event={"ID":"bbd7337b-38ca-4006-bd9f-d822e5410b7c","Type":"ContainerStarted","Data":"e01e1a53ef2f63ebfb18438b89ed554775e3b18458541826ba612bd80cab37dc"} Apr 22 15:08:46.811989 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.811958 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64fd688846-mkdtj" event={"ID":"bbd7337b-38ca-4006-bd9f-d822e5410b7c","Type":"ContainerStarted","Data":"c7b3d843624c74217d841200f657b955d1c8602bc4515b2e257b9285464b75f3"} Apr 22 15:08:46.865865 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:46.865807 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" podStartSLOduration=64.865789571 podStartE2EDuration="1m4.865789571s" podCreationTimestamp="2026-04-22 15:07:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-22 15:08:46.865150124 +0000 UTC m=+65.921888208" watchObservedRunningTime="2026-04-22 15:08:46.865789571 +0000 UTC m=+65.922527658" Apr 22 15:08:47.023769 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:47.023530 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:47.026547 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:47.026525 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:47.255689 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:47.254981 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-64fd688846-mkdtj" podStartSLOduration=48.254960433 podStartE2EDuration="48.254960433s" podCreationTimestamp="2026-04-22 15:07:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-22 15:08:47.089757979 +0000 UTC m=+66.146496059" watchObservedRunningTime="2026-04-22 15:08:47.254960433 +0000 UTC m=+66.311698521" Apr 22 15:08:47.255689 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:47.255129 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/router-default-594fffb56b-msk5s" podStartSLOduration=48.255121472 podStartE2EDuration="48.255121472s" podCreationTimestamp="2026-04-22 15:07:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-22 15:08:47.245526068 +0000 UTC m=+66.302264155" watchObservedRunningTime="2026-04-22 15:08:47.255121472 +0000 UTC m=+66.311859560" Apr 22 15:08:47.817407 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:47.817360 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:47.818686 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:47.818108 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:08:47.818686 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:47.818412 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-594fffb56b-msk5s" Apr 22 15:08:49.820903 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:49.820863 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64fd688846-mkdtj"] Apr 22 15:08:49.832688 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:49.832662 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6bcc868b7-4jlxk"] Apr 22 15:08:49.847610 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:49.847584 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6bcc868b7-4jlxk" Apr 22 15:08:49.853434 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:49.853398 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Apr 22 15:08:49.853683 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:49.853665 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Apr 22 15:08:49.853918 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:49.853899 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-j8hh4\"" Apr 22 15:08:49.854115 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:49.854088 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6bcc868b7-4jlxk"] Apr 22 15:08:49.968776 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:49.968756 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-w5wkb"] Apr 22 15:08:49.979461 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:49.979440 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-w5wkb" Apr 22 15:08:49.986386 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:49.986363 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Apr 22 15:08:49.986493 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:49.986473 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Apr 22 15:08:49.986493 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:49.986487 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dmx2q\"" Apr 22 15:08:49.994099 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:49.994076 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-w5wkb"] Apr 22 15:08:50.009849 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.009823 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd9mv\" (UniqueName: \"kubernetes.io/projected/4836af87-a000-45a5-845e-cc0488640ef0-kube-api-access-vd9mv\") pod \"downloads-6bcc868b7-4jlxk\" (UID: \"4836af87-a000-45a5-845e-cc0488640ef0\") " pod="openshift-console/downloads-6bcc868b7-4jlxk" Apr 22 15:08:50.111770 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.111142 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d32dc35b-d560-45ad-90b9-f982fed7d7df-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w5wkb\" (UID: \"d32dc35b-d560-45ad-90b9-f982fed7d7df\") " pod="openshift-insights/insights-runtime-extractor-w5wkb" Apr 22 15:08:50.111770 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.111206 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chw6k\" (UniqueName: \"kubernetes.io/projected/d32dc35b-d560-45ad-90b9-f982fed7d7df-kube-api-access-chw6k\") pod \"insights-runtime-extractor-w5wkb\" (UID: \"d32dc35b-d560-45ad-90b9-f982fed7d7df\") " pod="openshift-insights/insights-runtime-extractor-w5wkb" Apr 22 15:08:50.111770 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.111246 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d32dc35b-d560-45ad-90b9-f982fed7d7df-data-volume\") pod \"insights-runtime-extractor-w5wkb\" (UID: \"d32dc35b-d560-45ad-90b9-f982fed7d7df\") " pod="openshift-insights/insights-runtime-extractor-w5wkb" Apr 22 15:08:50.111770 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.111288 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d32dc35b-d560-45ad-90b9-f982fed7d7df-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w5wkb\" (UID: \"d32dc35b-d560-45ad-90b9-f982fed7d7df\") " pod="openshift-insights/insights-runtime-extractor-w5wkb" Apr 22 15:08:50.111770 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.111323 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vd9mv\" (UniqueName: \"kubernetes.io/projected/4836af87-a000-45a5-845e-cc0488640ef0-kube-api-access-vd9mv\") pod \"downloads-6bcc868b7-4jlxk\" (UID: \"4836af87-a000-45a5-845e-cc0488640ef0\") " pod="openshift-console/downloads-6bcc868b7-4jlxk" Apr 22 15:08:50.111770 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.111368 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d32dc35b-d560-45ad-90b9-f982fed7d7df-crio-socket\") pod \"insights-runtime-extractor-w5wkb\" (UID: \"d32dc35b-d560-45ad-90b9-f982fed7d7df\") " pod="openshift-insights/insights-runtime-extractor-w5wkb" Apr 22 15:08:50.131093 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.131036 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd9mv\" (UniqueName: \"kubernetes.io/projected/4836af87-a000-45a5-845e-cc0488640ef0-kube-api-access-vd9mv\") pod \"downloads-6bcc868b7-4jlxk\" (UID: \"4836af87-a000-45a5-845e-cc0488640ef0\") " pod="openshift-console/downloads-6bcc868b7-4jlxk" Apr 22 15:08:50.162303 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.160620 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6bcc868b7-4jlxk" Apr 22 15:08:50.212413 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.212362 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d32dc35b-d560-45ad-90b9-f982fed7d7df-data-volume\") pod \"insights-runtime-extractor-w5wkb\" (UID: \"d32dc35b-d560-45ad-90b9-f982fed7d7df\") " pod="openshift-insights/insights-runtime-extractor-w5wkb" Apr 22 15:08:50.212589 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.212431 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d32dc35b-d560-45ad-90b9-f982fed7d7df-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w5wkb\" (UID: \"d32dc35b-d560-45ad-90b9-f982fed7d7df\") " pod="openshift-insights/insights-runtime-extractor-w5wkb" Apr 22 15:08:50.212589 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.212469 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d32dc35b-d560-45ad-90b9-f982fed7d7df-crio-socket\") pod \"insights-runtime-extractor-w5wkb\" (UID: \"d32dc35b-d560-45ad-90b9-f982fed7d7df\") " pod="openshift-insights/insights-runtime-extractor-w5wkb" Apr 22 15:08:50.212589 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.212531 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d32dc35b-d560-45ad-90b9-f982fed7d7df-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w5wkb\" (UID: \"d32dc35b-d560-45ad-90b9-f982fed7d7df\") " pod="openshift-insights/insights-runtime-extractor-w5wkb" Apr 22 15:08:50.212589 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.212579 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-chw6k\" (UniqueName: \"kubernetes.io/projected/d32dc35b-d560-45ad-90b9-f982fed7d7df-kube-api-access-chw6k\") pod \"insights-runtime-extractor-w5wkb\" (UID: \"d32dc35b-d560-45ad-90b9-f982fed7d7df\") " pod="openshift-insights/insights-runtime-extractor-w5wkb" Apr 22 15:08:50.213226 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.212996 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d32dc35b-d560-45ad-90b9-f982fed7d7df-crio-socket\") pod \"insights-runtime-extractor-w5wkb\" (UID: \"d32dc35b-d560-45ad-90b9-f982fed7d7df\") " pod="openshift-insights/insights-runtime-extractor-w5wkb" Apr 22 15:08:50.213467 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.213435 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d32dc35b-d560-45ad-90b9-f982fed7d7df-data-volume\") pod \"insights-runtime-extractor-w5wkb\" (UID: \"d32dc35b-d560-45ad-90b9-f982fed7d7df\") " pod="openshift-insights/insights-runtime-extractor-w5wkb" Apr 22 15:08:50.213597 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.213573 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d32dc35b-d560-45ad-90b9-f982fed7d7df-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w5wkb\" (UID: \"d32dc35b-d560-45ad-90b9-f982fed7d7df\") " pod="openshift-insights/insights-runtime-extractor-w5wkb" Apr 22 15:08:50.215230 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.215182 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d32dc35b-d560-45ad-90b9-f982fed7d7df-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w5wkb\" (UID: \"d32dc35b-d560-45ad-90b9-f982fed7d7df\") " pod="openshift-insights/insights-runtime-extractor-w5wkb" Apr 22 15:08:50.223657 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.223635 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chw6k\" (UniqueName: \"kubernetes.io/projected/d32dc35b-d560-45ad-90b9-f982fed7d7df-kube-api-access-chw6k\") pod \"insights-runtime-extractor-w5wkb\" (UID: \"d32dc35b-d560-45ad-90b9-f982fed7d7df\") " pod="openshift-insights/insights-runtime-extractor-w5wkb" Apr 22 15:08:50.297707 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.288410 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-w5wkb" Apr 22 15:08:50.365228 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.365175 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6bcc868b7-4jlxk"] Apr 22 15:08:50.372213 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:08:50.371758 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4836af87_a000_45a5_845e_cc0488640ef0.slice/crio-e22c61e8622a92a9ba3d65e2423fbafa51ae4537dd1dc35d14ec62156d94d33c WatchSource:0}: Error finding container e22c61e8622a92a9ba3d65e2423fbafa51ae4537dd1dc35d14ec62156d94d33c: Status 404 returned error can't find the container with id e22c61e8622a92a9ba3d65e2423fbafa51ae4537dd1dc35d14ec62156d94d33c Apr 22 15:08:50.557434 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:08:50.557379 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd32dc35b_d560_45ad_90b9_f982fed7d7df.slice/crio-fa39615cf70345ac6307f7c52c4acf71d5d5ff9014ce6954d0ea82eec2ca1cc6 WatchSource:0}: Error finding container fa39615cf70345ac6307f7c52c4acf71d5d5ff9014ce6954d0ea82eec2ca1cc6: Status 404 returned error can't find the container with id fa39615cf70345ac6307f7c52c4acf71d5d5ff9014ce6954d0ea82eec2ca1cc6 Apr 22 15:08:50.563348 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.563281 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-w5wkb"] Apr 22 15:08:50.827415 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.827381 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-k8gnz" event={"ID":"b3b905d0-a65d-4615-8789-ccfb785189a3","Type":"ContainerStarted","Data":"45158905f00b3b1816bf96bae1df815f2208e6e1ae88934f2051824985631340"} Apr 22 15:08:50.827835 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.827423 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-k8gnz" event={"ID":"b3b905d0-a65d-4615-8789-ccfb785189a3","Type":"ContainerStarted","Data":"66081d25dd440d968ab5b46acc69e154956ebc2a4e8a68a6e095b81f24664afe"} Apr 22 15:08:50.828496 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.828472 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6bcc868b7-4jlxk" event={"ID":"4836af87-a000-45a5-845e-cc0488640ef0","Type":"ContainerStarted","Data":"e22c61e8622a92a9ba3d65e2423fbafa51ae4537dd1dc35d14ec62156d94d33c"} Apr 22 15:08:50.829879 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.829847 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-29bzc" event={"ID":"a84180fa-5ffa-4cc6-80cc-0cd55558a345","Type":"ContainerStarted","Data":"27e1f68d30e2170773246dbcd6ccbb3fa0dc2dc61983b10a4e7ddbe3da42b422"} Apr 22 15:08:50.831123 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.831102 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4qmd5" event={"ID":"eb3d139a-27c4-4fa7-8809-a33c13551872","Type":"ContainerStarted","Data":"351ee85451592aa7509304e8932ba554d81da1cb081f50baf51837e00607a432"} Apr 22 15:08:50.832620 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.832596 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4zrz9" event={"ID":"dc334588-fb2e-4ac6-bb34-460dad264804","Type":"ContainerStarted","Data":"3b3c9cdea3532366449ac9d38d7616dd79163e0244ef00f3ddb09cd635a77612"} Apr 22 15:08:50.832719 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.832627 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4zrz9" event={"ID":"dc334588-fb2e-4ac6-bb34-460dad264804","Type":"ContainerStarted","Data":"9c1932a4a5e2f92fdd8adbb666939ea38e405882121dd35230be533e9a3c6da5"} Apr 22 15:08:50.832783 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.832737 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4zrz9" Apr 22 15:08:50.833837 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.833818 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w5wkb" event={"ID":"d32dc35b-d560-45ad-90b9-f982fed7d7df","Type":"ContainerStarted","Data":"a77b541a011472d953ac82f69c9ab3e6d30a480cd9f7ee9263e9af5a45296baf"} Apr 22 15:08:50.833939 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.833846 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w5wkb" event={"ID":"d32dc35b-d560-45ad-90b9-f982fed7d7df","Type":"ContainerStarted","Data":"fa39615cf70345ac6307f7c52c4acf71d5d5ff9014ce6954d0ea82eec2ca1cc6"} Apr 22 15:08:50.835384 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.835362 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-cb95c66f6-6ljs8" event={"ID":"acbfa8e1-30e3-467a-b70e-c9e23ebddcbe","Type":"ContainerStarted","Data":"7c49e067a909754bb2c2f6998e83d6046f1f4d1049ea4923c9fbf3d01b4d293d"} Apr 22 15:08:50.836887 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.836868 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-z9ckr" event={"ID":"487f23b6-9470-41e9-abbd-295b6d317b10","Type":"ContainerStarted","Data":"2e72c4181825724bd6f8218c291fdca8dd2758cddd56570840527589a727f53b"} Apr 22 15:08:50.836970 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.836891 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-z9ckr" event={"ID":"487f23b6-9470-41e9-abbd-295b6d317b10","Type":"ContainerStarted","Data":"8968177af45f30cfd19d18cc6396877b1f20514ba5e672ae43dd50c0bea27850"} Apr 22 15:08:50.855221 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.855120 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-k8gnz" podStartSLOduration=48.141024375 podStartE2EDuration="51.855109398s" podCreationTimestamp="2026-04-22 15:07:59 +0000 UTC" firstStartedPulling="2026-04-22 15:08:46.244623995 +0000 UTC m=+65.301362058" lastFinishedPulling="2026-04-22 15:08:49.958709017 +0000 UTC m=+69.015447081" observedRunningTime="2026-04-22 15:08:50.852695612 +0000 UTC m=+69.909433699" watchObservedRunningTime="2026-04-22 15:08:50.855109398 +0000 UTC m=+69.911847461" Apr 22 15:08:50.868265 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.868240 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-s9dtm"] Apr 22 15:08:50.876435 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.876398 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-cb95c66f6-6ljs8" podStartSLOduration=40.228231171 podStartE2EDuration="43.876387781s" podCreationTimestamp="2026-04-22 15:08:07 +0000 UTC" firstStartedPulling="2026-04-22 15:08:46.303937034 +0000 UTC m=+65.360675098" lastFinishedPulling="2026-04-22 15:08:49.952093644 +0000 UTC m=+69.008831708" observedRunningTime="2026-04-22 15:08:50.875809385 +0000 UTC m=+69.932547471" watchObservedRunningTime="2026-04-22 15:08:50.876387781 +0000 UTC m=+69.933125865" Apr 22 15:08:50.890949 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.890928 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-s9dtm"] Apr 22 15:08:50.891058 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.891020 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-s9dtm" Apr 22 15:08:50.896680 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.896659 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Apr 22 15:08:50.896680 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.896677 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-g64mr\"" Apr 22 15:08:50.904908 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.904873 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-4zrz9" podStartSLOduration=34.502782788 podStartE2EDuration="37.904860322s" podCreationTimestamp="2026-04-22 15:08:13 +0000 UTC" firstStartedPulling="2026-04-22 15:08:46.560456474 +0000 UTC m=+65.617194543" lastFinishedPulling="2026-04-22 15:08:49.962534014 +0000 UTC m=+69.019272077" observedRunningTime="2026-04-22 15:08:50.903915701 +0000 UTC m=+69.960653812" watchObservedRunningTime="2026-04-22 15:08:50.904860322 +0000 UTC m=+69.961598407" Apr 22 15:08:50.932286 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.932246 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4qmd5" podStartSLOduration=34.339893617 podStartE2EDuration="37.932232923s" podCreationTimestamp="2026-04-22 15:08:13 +0000 UTC" firstStartedPulling="2026-04-22 15:08:46.366010734 +0000 UTC m=+65.422748796" lastFinishedPulling="2026-04-22 15:08:49.958350031 +0000 UTC m=+69.015088102" observedRunningTime="2026-04-22 15:08:50.930174918 +0000 UTC m=+69.986913023" watchObservedRunningTime="2026-04-22 15:08:50.932232923 +0000 UTC m=+69.988971002" Apr 22 15:08:50.960242 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.960197 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-z9ckr" podStartSLOduration=66.620321393 podStartE2EDuration="1m9.960180122s" podCreationTimestamp="2026-04-22 15:07:41 +0000 UTC" firstStartedPulling="2026-04-22 15:08:46.617895104 +0000 UTC m=+65.674633170" lastFinishedPulling="2026-04-22 15:08:49.957753822 +0000 UTC m=+69.014491899" observedRunningTime="2026-04-22 15:08:50.957634832 +0000 UTC m=+70.014372917" watchObservedRunningTime="2026-04-22 15:08:50.960180122 +0000 UTC m=+70.016918250" Apr 22 15:08:50.988560 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:50.988213 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-29bzc" podStartSLOduration=48.519926132 podStartE2EDuration="51.988198786s" podCreationTimestamp="2026-04-22 15:07:59 +0000 UTC" firstStartedPulling="2026-04-22 15:08:46.552130661 +0000 UTC m=+65.608868730" lastFinishedPulling="2026-04-22 15:08:50.020403308 +0000 UTC m=+69.077141384" observedRunningTime="2026-04-22 15:08:50.987934419 +0000 UTC m=+70.044672506" watchObservedRunningTime="2026-04-22 15:08:50.988198786 +0000 UTC m=+70.044936872" Apr 22 15:08:51.019391 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:51.019358 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6f799b17-a43e-45ef-8c84-0e7851c9905f-tls-certificates\") pod \"prometheus-operator-admission-webhook-57cf98b594-s9dtm\" (UID: \"6f799b17-a43e-45ef-8c84-0e7851c9905f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-s9dtm" Apr 22 15:08:51.120444 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:51.120346 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6f799b17-a43e-45ef-8c84-0e7851c9905f-tls-certificates\") pod \"prometheus-operator-admission-webhook-57cf98b594-s9dtm\" (UID: \"6f799b17-a43e-45ef-8c84-0e7851c9905f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-s9dtm" Apr 22 15:08:51.123197 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:51.123171 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6f799b17-a43e-45ef-8c84-0e7851c9905f-tls-certificates\") pod \"prometheus-operator-admission-webhook-57cf98b594-s9dtm\" (UID: \"6f799b17-a43e-45ef-8c84-0e7851c9905f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-s9dtm" Apr 22 15:08:51.202002 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:51.201968 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-s9dtm" Apr 22 15:08:51.362973 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:51.362919 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-s9dtm"] Apr 22 15:08:51.518316 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:08:51.518280 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f799b17_a43e_45ef_8c84_0e7851c9905f.slice/crio-f799ae5cdc858cc1b99ff85d8584dacce6453bd209eb374a635f63e9089ea0b8 WatchSource:0}: Error finding container f799ae5cdc858cc1b99ff85d8584dacce6453bd209eb374a635f63e9089ea0b8: Status 404 returned error can't find the container with id f799ae5cdc858cc1b99ff85d8584dacce6453bd209eb374a635f63e9089ea0b8 Apr 22 15:08:51.842756 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:51.842673 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-s9dtm" event={"ID":"6f799b17-a43e-45ef-8c84-0e7851c9905f","Type":"ContainerStarted","Data":"f799ae5cdc858cc1b99ff85d8584dacce6453bd209eb374a635f63e9089ea0b8"} Apr 22 15:08:52.848080 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:52.848043 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w5wkb" event={"ID":"d32dc35b-d560-45ad-90b9-f982fed7d7df","Type":"ContainerStarted","Data":"ada200ecd6bf222c1525ead074450d72d5fc44cee07ee44ff493c713de4b96b6"} Apr 22 15:08:53.852547 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:53.852507 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-s9dtm" event={"ID":"6f799b17-a43e-45ef-8c84-0e7851c9905f","Type":"ContainerStarted","Data":"a0c15bba33f3c0fb9b7ad20a8161880759566c5427b64fc5cf8f4450abbb766e"} Apr 22 15:08:53.853002 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:53.852714 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-s9dtm" Apr 22 15:08:53.857572 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:53.857547 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-s9dtm" Apr 22 15:08:53.871609 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:53.871565 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-s9dtm" podStartSLOduration=2.26474636 podStartE2EDuration="3.871549516s" podCreationTimestamp="2026-04-22 15:08:50 +0000 UTC" firstStartedPulling="2026-04-22 15:08:51.520371916 +0000 UTC m=+70.577109981" lastFinishedPulling="2026-04-22 15:08:53.127175073 +0000 UTC m=+72.183913137" observedRunningTime="2026-04-22 15:08:53.870117797 +0000 UTC m=+72.926855885" watchObservedRunningTime="2026-04-22 15:08:53.871549516 +0000 UTC m=+72.928287600" Apr 22 15:08:54.857509 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:54.857467 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w5wkb" event={"ID":"d32dc35b-d560-45ad-90b9-f982fed7d7df","Type":"ContainerStarted","Data":"e86a9cc8a151df3619f9478fd32f563a3f673f584b40bc608ff7bbe776dadc41"} Apr 22 15:08:54.894068 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:54.894018 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-w5wkb" podStartSLOduration=2.369841253 podStartE2EDuration="5.894003414s" podCreationTimestamp="2026-04-22 15:08:49 +0000 UTC" firstStartedPulling="2026-04-22 15:08:50.644546494 +0000 UTC m=+69.701284569" lastFinishedPulling="2026-04-22 15:08:54.168708654 +0000 UTC m=+73.225446730" observedRunningTime="2026-04-22 15:08:54.891866707 +0000 UTC m=+73.948604802" watchObservedRunningTime="2026-04-22 15:08:54.894003414 +0000 UTC m=+73.950741499" Apr 22 15:08:56.725885 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:08:56.725857 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8lfsw" Apr 22 15:09:00.845589 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:00.845558 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4zrz9" Apr 22 15:09:01.537412 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.537065 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zlb8j"] Apr 22 15:09:01.577353 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.576693 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.583080 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.583038 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Apr 22 15:09:01.583726 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.583525 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2fn2s\"" Apr 22 15:09:01.583726 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.583584 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Apr 22 15:09:01.584947 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.583967 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Apr 22 15:09:01.584947 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.584127 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Apr 22 15:09:01.612349 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.612295 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/29275b0f-a5ac-4a8f-8ac0-bb243b489174-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zlb8j\" (UID: \"29275b0f-a5ac-4a8f-8ac0-bb243b489174\") " pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.612562 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.612504 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmqgz\" (UniqueName: \"kubernetes.io/projected/29275b0f-a5ac-4a8f-8ac0-bb243b489174-kube-api-access-hmqgz\") pod \"node-exporter-zlb8j\" (UID: \"29275b0f-a5ac-4a8f-8ac0-bb243b489174\") " pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.612663 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.612577 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29275b0f-a5ac-4a8f-8ac0-bb243b489174-metrics-client-ca\") pod \"node-exporter-zlb8j\" (UID: \"29275b0f-a5ac-4a8f-8ac0-bb243b489174\") " pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.612663 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.612624 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/29275b0f-a5ac-4a8f-8ac0-bb243b489174-node-exporter-textfile\") pod \"node-exporter-zlb8j\" (UID: \"29275b0f-a5ac-4a8f-8ac0-bb243b489174\") " pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.612768 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.612670 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/29275b0f-a5ac-4a8f-8ac0-bb243b489174-root\") pod \"node-exporter-zlb8j\" (UID: \"29275b0f-a5ac-4a8f-8ac0-bb243b489174\") " pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.612768 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.612694 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/29275b0f-a5ac-4a8f-8ac0-bb243b489174-node-exporter-wtmp\") pod \"node-exporter-zlb8j\" (UID: \"29275b0f-a5ac-4a8f-8ac0-bb243b489174\") " pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.612768 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.612719 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29275b0f-a5ac-4a8f-8ac0-bb243b489174-sys\") pod \"node-exporter-zlb8j\" (UID: \"29275b0f-a5ac-4a8f-8ac0-bb243b489174\") " pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.614089 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.612822 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/29275b0f-a5ac-4a8f-8ac0-bb243b489174-node-exporter-tls\") pod \"node-exporter-zlb8j\" (UID: \"29275b0f-a5ac-4a8f-8ac0-bb243b489174\") " pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.614089 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.612900 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/29275b0f-a5ac-4a8f-8ac0-bb243b489174-node-exporter-accelerators-collector-config\") pod \"node-exporter-zlb8j\" (UID: \"29275b0f-a5ac-4a8f-8ac0-bb243b489174\") " pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.714170 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.714137 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/29275b0f-a5ac-4a8f-8ac0-bb243b489174-root\") pod \"node-exporter-zlb8j\" (UID: \"29275b0f-a5ac-4a8f-8ac0-bb243b489174\") " pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.714170 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.714179 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/29275b0f-a5ac-4a8f-8ac0-bb243b489174-node-exporter-wtmp\") pod \"node-exporter-zlb8j\" (UID: \"29275b0f-a5ac-4a8f-8ac0-bb243b489174\") " pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.714451 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.714206 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29275b0f-a5ac-4a8f-8ac0-bb243b489174-sys\") pod \"node-exporter-zlb8j\" (UID: \"29275b0f-a5ac-4a8f-8ac0-bb243b489174\") " pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.714451 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.714244 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/29275b0f-a5ac-4a8f-8ac0-bb243b489174-node-exporter-tls\") pod \"node-exporter-zlb8j\" (UID: \"29275b0f-a5ac-4a8f-8ac0-bb243b489174\") " pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.714451 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.714240 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/29275b0f-a5ac-4a8f-8ac0-bb243b489174-root\") pod \"node-exporter-zlb8j\" (UID: \"29275b0f-a5ac-4a8f-8ac0-bb243b489174\") " pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.714451 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.714282 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/29275b0f-a5ac-4a8f-8ac0-bb243b489174-node-exporter-accelerators-collector-config\") pod \"node-exporter-zlb8j\" (UID: \"29275b0f-a5ac-4a8f-8ac0-bb243b489174\") " pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.714451 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.714304 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29275b0f-a5ac-4a8f-8ac0-bb243b489174-sys\") pod \"node-exporter-zlb8j\" (UID: \"29275b0f-a5ac-4a8f-8ac0-bb243b489174\") " pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.714451 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.714323 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/29275b0f-a5ac-4a8f-8ac0-bb243b489174-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zlb8j\" (UID: \"29275b0f-a5ac-4a8f-8ac0-bb243b489174\") " pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.714451 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.714366 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/29275b0f-a5ac-4a8f-8ac0-bb243b489174-node-exporter-wtmp\") pod \"node-exporter-zlb8j\" (UID: \"29275b0f-a5ac-4a8f-8ac0-bb243b489174\") " pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.714451 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.714381 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hmqgz\" (UniqueName: \"kubernetes.io/projected/29275b0f-a5ac-4a8f-8ac0-bb243b489174-kube-api-access-hmqgz\") pod \"node-exporter-zlb8j\" (UID: \"29275b0f-a5ac-4a8f-8ac0-bb243b489174\") " pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.714451 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.714441 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29275b0f-a5ac-4a8f-8ac0-bb243b489174-metrics-client-ca\") pod \"node-exporter-zlb8j\" (UID: \"29275b0f-a5ac-4a8f-8ac0-bb243b489174\") " pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.714869 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.714488 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/29275b0f-a5ac-4a8f-8ac0-bb243b489174-node-exporter-textfile\") pod \"node-exporter-zlb8j\" (UID: \"29275b0f-a5ac-4a8f-8ac0-bb243b489174\") " pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.714869 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.714778 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/29275b0f-a5ac-4a8f-8ac0-bb243b489174-node-exporter-textfile\") pod \"node-exporter-zlb8j\" (UID: \"29275b0f-a5ac-4a8f-8ac0-bb243b489174\") " pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.714938 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.714916 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/29275b0f-a5ac-4a8f-8ac0-bb243b489174-node-exporter-accelerators-collector-config\") pod \"node-exporter-zlb8j\" (UID: \"29275b0f-a5ac-4a8f-8ac0-bb243b489174\") " pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.715131 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.715110 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29275b0f-a5ac-4a8f-8ac0-bb243b489174-metrics-client-ca\") pod \"node-exporter-zlb8j\" (UID: \"29275b0f-a5ac-4a8f-8ac0-bb243b489174\") " pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.716914 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.716883 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/29275b0f-a5ac-4a8f-8ac0-bb243b489174-node-exporter-tls\") pod \"node-exporter-zlb8j\" (UID: \"29275b0f-a5ac-4a8f-8ac0-bb243b489174\") " pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.717190 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.717170 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/29275b0f-a5ac-4a8f-8ac0-bb243b489174-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zlb8j\" (UID: \"29275b0f-a5ac-4a8f-8ac0-bb243b489174\") " pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.724127 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.724105 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmqgz\" (UniqueName: \"kubernetes.io/projected/29275b0f-a5ac-4a8f-8ac0-bb243b489174-kube-api-access-hmqgz\") pod \"node-exporter-zlb8j\" (UID: \"29275b0f-a5ac-4a8f-8ac0-bb243b489174\") " pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.891814 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:01.891732 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zlb8j" Apr 22 15:09:01.901950 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:09:01.901921 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29275b0f_a5ac_4a8f_8ac0_bb243b489174.slice/crio-c9fdb5c8d257d025984c4eb642e871358ce7552d1b1dcd280101613d61de33af WatchSource:0}: Error finding container c9fdb5c8d257d025984c4eb642e871358ce7552d1b1dcd280101613d61de33af: Status 404 returned error can't find the container with id c9fdb5c8d257d025984c4eb642e871358ce7552d1b1dcd280101613d61de33af Apr 22 15:09:02.881047 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:02.881013 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zlb8j" event={"ID":"29275b0f-a5ac-4a8f-8ac0-bb243b489174","Type":"ContainerStarted","Data":"c9fdb5c8d257d025984c4eb642e871358ce7552d1b1dcd280101613d61de33af"} Apr 22 15:09:03.887978 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:03.887939 2577 generic.go:358] "Generic (PLEG): container finished" podID="29275b0f-a5ac-4a8f-8ac0-bb243b489174" containerID="281a39f99203f52952df1dc8fa8580bd02819c2e10de5829ce4432718efcf5c1" exitCode=0 Apr 22 15:09:03.888412 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:03.888013 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zlb8j" event={"ID":"29275b0f-a5ac-4a8f-8ac0-bb243b489174","Type":"ContainerDied","Data":"281a39f99203f52952df1dc8fa8580bd02819c2e10de5829ce4432718efcf5c1"} Apr 22 15:09:05.934910 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:05.934861 2577 patch_prober.go:28] interesting pod/image-registry-8bc6dfb5-5vmjx container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Apr 22 15:09:05.935341 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:05.934926 2577 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" podUID="df8a3082-b089-41db-bbd6-1e6b09fd544b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Apr 22 15:09:07.823315 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:07.823286 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:09:09.829031 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:09.828989 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:09:10.912826 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:10.912724 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6bcc868b7-4jlxk" event={"ID":"4836af87-a000-45a5-845e-cc0488640ef0","Type":"ContainerStarted","Data":"6dc7f10de35587deb8d2f613016edbf9becf7e501a52f945f15cbe696f0827a9"} Apr 22 15:09:10.913234 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:10.912923 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6bcc868b7-4jlxk" Apr 22 15:09:10.914499 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:10.914470 2577 patch_prober.go:28] interesting pod/downloads-6bcc868b7-4jlxk container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.22:8080/\": dial tcp 10.132.0.22:8080: connect: connection refused" start-of-body= Apr 22 15:09:10.914614 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:10.914518 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-6bcc868b7-4jlxk" podUID="4836af87-a000-45a5-845e-cc0488640ef0" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.22:8080/\": dial tcp 10.132.0.22:8080: connect: connection refused" Apr 22 15:09:10.915464 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:10.915440 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zlb8j" event={"ID":"29275b0f-a5ac-4a8f-8ac0-bb243b489174","Type":"ContainerStarted","Data":"df2165d8eabdcfdb74e2125c9a06ce7174471650c81571e72505e13db40fae3f"} Apr 22 15:09:10.915572 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:10.915472 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zlb8j" event={"ID":"29275b0f-a5ac-4a8f-8ac0-bb243b489174","Type":"ContainerStarted","Data":"9e04e9ed79c2e34f516a178c3359ad580d3b1aa6ea4624b0624f8fee8aea335c"} Apr 22 15:09:10.934510 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:10.934460 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-6bcc868b7-4jlxk" podStartSLOduration=1.609046475 podStartE2EDuration="21.934446798s" podCreationTimestamp="2026-04-22 15:08:49 +0000 UTC" firstStartedPulling="2026-04-22 15:08:50.390053874 +0000 UTC m=+69.446791937" lastFinishedPulling="2026-04-22 15:09:10.71545418 +0000 UTC m=+89.772192260" observedRunningTime="2026-04-22 15:09:10.9335142 +0000 UTC m=+89.990252298" watchObservedRunningTime="2026-04-22 15:09:10.934446798 +0000 UTC m=+89.991184883" Apr 22 15:09:10.952645 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:10.952593 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zlb8j" podStartSLOduration=9.044156945 podStartE2EDuration="9.95257623s" podCreationTimestamp="2026-04-22 15:09:01 +0000 UTC" firstStartedPulling="2026-04-22 15:09:01.903759962 +0000 UTC m=+80.960498025" lastFinishedPulling="2026-04-22 15:09:02.812179234 +0000 UTC m=+81.868917310" observedRunningTime="2026-04-22 15:09:10.951079682 +0000 UTC m=+90.007817766" watchObservedRunningTime="2026-04-22 15:09:10.95257623 +0000 UTC m=+90.009314317" Apr 22 15:09:11.936772 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:11.936741 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6bcc868b7-4jlxk" Apr 22 15:09:14.845088 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:14.845002 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-64fd688846-mkdtj" podUID="bbd7337b-38ca-4006-bd9f-d822e5410b7c" containerName="registry" containerID="cri-o://e01e1a53ef2f63ebfb18438b89ed554775e3b18458541826ba612bd80cab37dc" gracePeriod=30 Apr 22 15:09:15.121855 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.121830 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:09:15.232171 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.232135 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bbd7337b-38ca-4006-bd9f-d822e5410b7c-image-registry-private-configuration\") pod \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " Apr 22 15:09:15.232379 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.232189 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-certificates\") pod \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " Apr 22 15:09:15.232379 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.232247 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-tls\") pod \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " Apr 22 15:09:15.232379 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.232283 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bbd7337b-38ca-4006-bd9f-d822e5410b7c-installation-pull-secrets\") pod \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " Apr 22 15:09:15.232670 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.232633 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "bbd7337b-38ca-4006-bd9f-d822e5410b7c" (UID: "bbd7337b-38ca-4006-bd9f-d822e5410b7c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 22 15:09:15.232731 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.232716 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c5rdv\" (UniqueName: \"kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-kube-api-access-c5rdv\") pod \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " Apr 22 15:09:15.233280 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.232779 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bbd7337b-38ca-4006-bd9f-d822e5410b7c-ca-trust-extracted\") pod \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " Apr 22 15:09:15.233280 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.232818 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbd7337b-38ca-4006-bd9f-d822e5410b7c-trusted-ca\") pod \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " Apr 22 15:09:15.233280 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.232852 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-bound-sa-token\") pod \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\" (UID: \"bbd7337b-38ca-4006-bd9f-d822e5410b7c\") " Apr 22 15:09:15.233280 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.233082 2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-certificates\") on node \"ip-10-0-131-168.ec2.internal\" DevicePath \"\"" Apr 22 15:09:15.233280 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.233259 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbd7337b-38ca-4006-bd9f-d822e5410b7c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "bbd7337b-38ca-4006-bd9f-d822e5410b7c" (UID: "bbd7337b-38ca-4006-bd9f-d822e5410b7c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 22 15:09:15.234913 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.234869 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "bbd7337b-38ca-4006-bd9f-d822e5410b7c" (UID: "bbd7337b-38ca-4006-bd9f-d822e5410b7c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 22 15:09:15.234913 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.234876 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbd7337b-38ca-4006-bd9f-d822e5410b7c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "bbd7337b-38ca-4006-bd9f-d822e5410b7c" (UID: "bbd7337b-38ca-4006-bd9f-d822e5410b7c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 22 15:09:15.235097 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.234990 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbd7337b-38ca-4006-bd9f-d822e5410b7c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "bbd7337b-38ca-4006-bd9f-d822e5410b7c" (UID: "bbd7337b-38ca-4006-bd9f-d822e5410b7c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 22 15:09:15.235158 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.235120 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "bbd7337b-38ca-4006-bd9f-d822e5410b7c" (UID: "bbd7337b-38ca-4006-bd9f-d822e5410b7c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 22 15:09:15.235612 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.235587 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-kube-api-access-c5rdv" (OuterVolumeSpecName: "kube-api-access-c5rdv") pod "bbd7337b-38ca-4006-bd9f-d822e5410b7c" (UID: "bbd7337b-38ca-4006-bd9f-d822e5410b7c"). InnerVolumeSpecName "kube-api-access-c5rdv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 22 15:09:15.245298 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.245269 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbd7337b-38ca-4006-bd9f-d822e5410b7c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "bbd7337b-38ca-4006-bd9f-d822e5410b7c" (UID: "bbd7337b-38ca-4006-bd9f-d822e5410b7c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 22 15:09:15.334298 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.334263 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbd7337b-38ca-4006-bd9f-d822e5410b7c-trusted-ca\") on node \"ip-10-0-131-168.ec2.internal\" DevicePath \"\"" Apr 22 15:09:15.334298 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.334293 2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-bound-sa-token\") on node \"ip-10-0-131-168.ec2.internal\" DevicePath \"\"" Apr 22 15:09:15.334298 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.334304 2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bbd7337b-38ca-4006-bd9f-d822e5410b7c-image-registry-private-configuration\") on node \"ip-10-0-131-168.ec2.internal\" DevicePath \"\"" Apr 22 15:09:15.334574 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.334314 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-registry-tls\") on node \"ip-10-0-131-168.ec2.internal\" DevicePath \"\"" Apr 22 15:09:15.334574 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.334338 2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bbd7337b-38ca-4006-bd9f-d822e5410b7c-installation-pull-secrets\") on node \"ip-10-0-131-168.ec2.internal\" DevicePath \"\"" Apr 22 15:09:15.334574 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.334353 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5rdv\" (UniqueName: \"kubernetes.io/projected/bbd7337b-38ca-4006-bd9f-d822e5410b7c-kube-api-access-c5rdv\") on node \"ip-10-0-131-168.ec2.internal\" DevicePath \"\"" Apr 22 15:09:15.334574 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.334367 2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bbd7337b-38ca-4006-bd9f-d822e5410b7c-ca-trust-extracted\") on node \"ip-10-0-131-168.ec2.internal\" DevicePath \"\"" Apr 22 15:09:15.936687 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.936610 2577 generic.go:358] "Generic (PLEG): container finished" podID="bbd7337b-38ca-4006-bd9f-d822e5410b7c" containerID="e01e1a53ef2f63ebfb18438b89ed554775e3b18458541826ba612bd80cab37dc" exitCode=0 Apr 22 15:09:15.936687 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.936681 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64fd688846-mkdtj" Apr 22 15:09:15.937303 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.936697 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64fd688846-mkdtj" event={"ID":"bbd7337b-38ca-4006-bd9f-d822e5410b7c","Type":"ContainerDied","Data":"e01e1a53ef2f63ebfb18438b89ed554775e3b18458541826ba612bd80cab37dc"} Apr 22 15:09:15.937303 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.936734 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64fd688846-mkdtj" event={"ID":"bbd7337b-38ca-4006-bd9f-d822e5410b7c","Type":"ContainerDied","Data":"c7b3d843624c74217d841200f657b955d1c8602bc4515b2e257b9285464b75f3"} Apr 22 15:09:15.937303 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.936755 2577 scope.go:117] "RemoveContainer" containerID="e01e1a53ef2f63ebfb18438b89ed554775e3b18458541826ba612bd80cab37dc" Apr 22 15:09:15.951109 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.951085 2577 scope.go:117] "RemoveContainer" containerID="e01e1a53ef2f63ebfb18438b89ed554775e3b18458541826ba612bd80cab37dc" Apr 22 15:09:15.951427 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:09:15.951401 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e01e1a53ef2f63ebfb18438b89ed554775e3b18458541826ba612bd80cab37dc\": container with ID starting with e01e1a53ef2f63ebfb18438b89ed554775e3b18458541826ba612bd80cab37dc not found: ID does not exist" containerID="e01e1a53ef2f63ebfb18438b89ed554775e3b18458541826ba612bd80cab37dc" Apr 22 15:09:15.951522 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.951437 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e01e1a53ef2f63ebfb18438b89ed554775e3b18458541826ba612bd80cab37dc"} err="failed to get container status \"e01e1a53ef2f63ebfb18438b89ed554775e3b18458541826ba612bd80cab37dc\": rpc error: code = NotFound desc = could not find container \"e01e1a53ef2f63ebfb18438b89ed554775e3b18458541826ba612bd80cab37dc\": container with ID starting with e01e1a53ef2f63ebfb18438b89ed554775e3b18458541826ba612bd80cab37dc not found: ID does not exist" Apr 22 15:09:15.968766 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.968740 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64fd688846-mkdtj"] Apr 22 15:09:15.973080 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:15.973056 2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-64fd688846-mkdtj"] Apr 22 15:09:16.295079 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:16.295048 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8bc6dfb5-5vmjx"] Apr 22 15:09:17.508053 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:17.508013 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbd7337b-38ca-4006-bd9f-d822e5410b7c" path="/var/lib/kubelet/pods/bbd7337b-38ca-4006-bd9f-d822e5410b7c/volumes" Apr 22 15:09:34.994227 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:34.994193 2577 generic.go:358] "Generic (PLEG): container finished" podID="68da1e47-8873-416c-b2d9-d515d0985631" containerID="56b5e1edef4cccabab1a3457edd58092598b186b19dcc9a7f12d7b247689185c" exitCode=0 Apr 22 15:09:34.994628 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:34.994269 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-bd6gc" event={"ID":"68da1e47-8873-416c-b2d9-d515d0985631","Type":"ContainerDied","Data":"56b5e1edef4cccabab1a3457edd58092598b186b19dcc9a7f12d7b247689185c"} Apr 22 15:09:34.994628 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:34.994582 2577 scope.go:117] "RemoveContainer" containerID="56b5e1edef4cccabab1a3457edd58092598b186b19dcc9a7f12d7b247689185c" Apr 22 15:09:35.998961 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:35.998921 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-bd6gc" event={"ID":"68da1e47-8873-416c-b2d9-d515d0985631","Type":"ContainerStarted","Data":"1ef026a3e4b0ffc2866726949460aef524be42875f5db0086cf0e56e6ee41e07"} Apr 22 15:09:36.000253 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:36.000225 2577 generic.go:358] "Generic (PLEG): container finished" podID="6cd30837-92d3-44c3-ba6a-9c84a22a7c8b" containerID="0cd4621403ba3c8feb895b673b6405e184eefd9483492c07faecec39deb09138" exitCode=0 Apr 22 15:09:36.000368 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:36.000260 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-fxjmk" event={"ID":"6cd30837-92d3-44c3-ba6a-9c84a22a7c8b","Type":"ContainerDied","Data":"0cd4621403ba3c8feb895b673b6405e184eefd9483492c07faecec39deb09138"} Apr 22 15:09:36.000541 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:36.000529 2577 scope.go:117] "RemoveContainer" containerID="0cd4621403ba3c8feb895b673b6405e184eefd9483492c07faecec39deb09138" Apr 22 15:09:37.004083 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:37.004053 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-fxjmk" event={"ID":"6cd30837-92d3-44c3-ba6a-9c84a22a7c8b","Type":"ContainerStarted","Data":"78dc93e783cd23f3e21ad8542e80dd2230b016e511953111b526717c087fd8a4"} Apr 22 15:09:41.316320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:41.316279 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" podUID="df8a3082-b089-41db-bbd6-1e6b09fd544b" containerName="registry" containerID="cri-o://4ffd314e0f29d8121f4889a51e31bb30188bec61c987c9b06a49cb1a947eae99" gracePeriod=30 Apr 22 15:09:41.578888 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:41.578836 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:09:41.755184 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:41.755148 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/df8a3082-b089-41db-bbd6-1e6b09fd544b-ca-trust-extracted\") pod \"df8a3082-b089-41db-bbd6-1e6b09fd544b\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " Apr 22 15:09:41.755184 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:41.755187 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-tls\") pod \"df8a3082-b089-41db-bbd6-1e6b09fd544b\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " Apr 22 15:09:41.755540 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:41.755231 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrzpl\" (UniqueName: \"kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-kube-api-access-jrzpl\") pod \"df8a3082-b089-41db-bbd6-1e6b09fd544b\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " Apr 22 15:09:41.755540 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:41.755283 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/df8a3082-b089-41db-bbd6-1e6b09fd544b-trusted-ca\") pod \"df8a3082-b089-41db-bbd6-1e6b09fd544b\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " Apr 22 15:09:41.755540 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:41.755313 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-bound-sa-token\") pod \"df8a3082-b089-41db-bbd6-1e6b09fd544b\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " Apr 22 15:09:41.755540 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:41.755463 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/df8a3082-b089-41db-bbd6-1e6b09fd544b-installation-pull-secrets\") pod \"df8a3082-b089-41db-bbd6-1e6b09fd544b\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " Apr 22 15:09:41.755540 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:41.755531 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-certificates\") pod \"df8a3082-b089-41db-bbd6-1e6b09fd544b\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " Apr 22 15:09:41.755793 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:41.755559 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/df8a3082-b089-41db-bbd6-1e6b09fd544b-image-registry-private-configuration\") pod \"df8a3082-b089-41db-bbd6-1e6b09fd544b\" (UID: \"df8a3082-b089-41db-bbd6-1e6b09fd544b\") " Apr 22 15:09:41.756001 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:41.755947 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "df8a3082-b089-41db-bbd6-1e6b09fd544b" (UID: "df8a3082-b089-41db-bbd6-1e6b09fd544b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 22 15:09:41.756119 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:41.756101 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df8a3082-b089-41db-bbd6-1e6b09fd544b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "df8a3082-b089-41db-bbd6-1e6b09fd544b" (UID: "df8a3082-b089-41db-bbd6-1e6b09fd544b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 22 15:09:41.757941 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:41.757893 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-kube-api-access-jrzpl" (OuterVolumeSpecName: "kube-api-access-jrzpl") pod "df8a3082-b089-41db-bbd6-1e6b09fd544b" (UID: "df8a3082-b089-41db-bbd6-1e6b09fd544b"). InnerVolumeSpecName "kube-api-access-jrzpl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 22 15:09:41.758051 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:41.757944 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "df8a3082-b089-41db-bbd6-1e6b09fd544b" (UID: "df8a3082-b089-41db-bbd6-1e6b09fd544b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 22 15:09:41.758148 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:41.758121 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df8a3082-b089-41db-bbd6-1e6b09fd544b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "df8a3082-b089-41db-bbd6-1e6b09fd544b" (UID: "df8a3082-b089-41db-bbd6-1e6b09fd544b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 22 15:09:41.758264 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:41.758156 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "df8a3082-b089-41db-bbd6-1e6b09fd544b" (UID: "df8a3082-b089-41db-bbd6-1e6b09fd544b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 22 15:09:41.758264 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:41.758222 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df8a3082-b089-41db-bbd6-1e6b09fd544b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "df8a3082-b089-41db-bbd6-1e6b09fd544b" (UID: "df8a3082-b089-41db-bbd6-1e6b09fd544b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 22 15:09:41.763569 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:41.763544 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df8a3082-b089-41db-bbd6-1e6b09fd544b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "df8a3082-b089-41db-bbd6-1e6b09fd544b" (UID: "df8a3082-b089-41db-bbd6-1e6b09fd544b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 22 15:09:41.856370 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:41.856269 2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-bound-sa-token\") on node \"ip-10-0-131-168.ec2.internal\" DevicePath \"\"" Apr 22 15:09:41.856370 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:41.856297 2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/df8a3082-b089-41db-bbd6-1e6b09fd544b-installation-pull-secrets\") on node \"ip-10-0-131-168.ec2.internal\" DevicePath \"\"" Apr 22 15:09:41.856370 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:41.856307 2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-certificates\") on node \"ip-10-0-131-168.ec2.internal\" DevicePath \"\"" Apr 22 15:09:41.856370 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:41.856317 2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/df8a3082-b089-41db-bbd6-1e6b09fd544b-image-registry-private-configuration\") on node \"ip-10-0-131-168.ec2.internal\" DevicePath \"\"" Apr 22 15:09:41.856370 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:41.856347 2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/df8a3082-b089-41db-bbd6-1e6b09fd544b-ca-trust-extracted\") on node \"ip-10-0-131-168.ec2.internal\" DevicePath \"\"" Apr 22 15:09:41.856370 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:41.856358 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-registry-tls\") on node \"ip-10-0-131-168.ec2.internal\" DevicePath \"\"" Apr 22 15:09:41.856370 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:41.856366 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrzpl\" (UniqueName: \"kubernetes.io/projected/df8a3082-b089-41db-bbd6-1e6b09fd544b-kube-api-access-jrzpl\") on node \"ip-10-0-131-168.ec2.internal\" DevicePath \"\"" Apr 22 15:09:41.856370 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:41.856374 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/df8a3082-b089-41db-bbd6-1e6b09fd544b-trusted-ca\") on node \"ip-10-0-131-168.ec2.internal\" DevicePath \"\"" Apr 22 15:09:42.019410 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:42.019375 2577 generic.go:358] "Generic (PLEG): container finished" podID="df8a3082-b089-41db-bbd6-1e6b09fd544b" containerID="4ffd314e0f29d8121f4889a51e31bb30188bec61c987c9b06a49cb1a947eae99" exitCode=0 Apr 22 15:09:42.019567 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:42.019443 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" Apr 22 15:09:42.019567 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:42.019466 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" event={"ID":"df8a3082-b089-41db-bbd6-1e6b09fd544b","Type":"ContainerDied","Data":"4ffd314e0f29d8121f4889a51e31bb30188bec61c987c9b06a49cb1a947eae99"} Apr 22 15:09:42.019567 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:42.019503 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8bc6dfb5-5vmjx" event={"ID":"df8a3082-b089-41db-bbd6-1e6b09fd544b","Type":"ContainerDied","Data":"286a2f668c0264dea55ca6877a278198fb19e73fa5919971aa205bc35e418aec"} Apr 22 15:09:42.019567 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:42.019522 2577 scope.go:117] "RemoveContainer" containerID="4ffd314e0f29d8121f4889a51e31bb30188bec61c987c9b06a49cb1a947eae99" Apr 22 15:09:42.027300 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:42.027281 2577 scope.go:117] "RemoveContainer" containerID="4ffd314e0f29d8121f4889a51e31bb30188bec61c987c9b06a49cb1a947eae99" Apr 22 15:09:42.027575 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:09:42.027555 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ffd314e0f29d8121f4889a51e31bb30188bec61c987c9b06a49cb1a947eae99\": container with ID starting with 4ffd314e0f29d8121f4889a51e31bb30188bec61c987c9b06a49cb1a947eae99 not found: ID does not exist" containerID="4ffd314e0f29d8121f4889a51e31bb30188bec61c987c9b06a49cb1a947eae99" Apr 22 15:09:42.027646 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:42.027585 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ffd314e0f29d8121f4889a51e31bb30188bec61c987c9b06a49cb1a947eae99"} err="failed to get container status \"4ffd314e0f29d8121f4889a51e31bb30188bec61c987c9b06a49cb1a947eae99\": rpc error: code = NotFound desc = could not find container \"4ffd314e0f29d8121f4889a51e31bb30188bec61c987c9b06a49cb1a947eae99\": container with ID starting with 4ffd314e0f29d8121f4889a51e31bb30188bec61c987c9b06a49cb1a947eae99 not found: ID does not exist" Apr 22 15:09:42.041698 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:42.041677 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8bc6dfb5-5vmjx"] Apr 22 15:09:42.046052 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:42.046032 2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-8bc6dfb5-5vmjx"] Apr 22 15:09:43.507456 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:43.507420 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df8a3082-b089-41db-bbd6-1e6b09fd544b" path="/var/lib/kubelet/pods/df8a3082-b089-41db-bbd6-1e6b09fd544b/volumes" Apr 22 15:09:56.064919 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:56.064885 2577 generic.go:358] "Generic (PLEG): container finished" podID="c7b8b7c3-8b82-46c5-bb74-0c90f6448aff" containerID="49ec7cfdef76678998bfcceb426679b5d5ebb1f6cc510b35eb0a80986efde0f9" exitCode=0 Apr 22 15:09:56.065312 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:56.064957 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-585dfdc468-rxj2p" event={"ID":"c7b8b7c3-8b82-46c5-bb74-0c90f6448aff","Type":"ContainerDied","Data":"49ec7cfdef76678998bfcceb426679b5d5ebb1f6cc510b35eb0a80986efde0f9"} Apr 22 15:09:56.065382 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:56.065339 2577 scope.go:117] "RemoveContainer" containerID="49ec7cfdef76678998bfcceb426679b5d5ebb1f6cc510b35eb0a80986efde0f9" Apr 22 15:09:57.069130 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:09:57.069097 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-585dfdc468-rxj2p" event={"ID":"c7b8b7c3-8b82-46c5-bb74-0c90f6448aff","Type":"ContainerStarted","Data":"78bf9ad9b59bcf3d8f4b908f1d4797e5d0b5b0bad66c2ec76abb314f8eb71e76"} Apr 22 15:10:48.363421 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:48.363380 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c47c644-4kbbs"] Apr 22 15:10:48.363908 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:48.363681 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df8a3082-b089-41db-bbd6-1e6b09fd544b" containerName="registry" Apr 22 15:10:48.363908 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:48.363693 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="df8a3082-b089-41db-bbd6-1e6b09fd544b" containerName="registry" Apr 22 15:10:48.363908 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:48.363712 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbd7337b-38ca-4006-bd9f-d822e5410b7c" containerName="registry" Apr 22 15:10:48.363908 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:48.363717 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbd7337b-38ca-4006-bd9f-d822e5410b7c" containerName="registry" Apr 22 15:10:48.363908 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:48.363766 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="df8a3082-b089-41db-bbd6-1e6b09fd544b" containerName="registry" Apr 22 15:10:48.363908 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:48.363773 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="bbd7337b-38ca-4006-bd9f-d822e5410b7c" containerName="registry" Apr 22 15:10:48.367720 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:48.367697 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c47c644-4kbbs" Apr 22 15:10:48.370467 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:48.370435 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Apr 22 15:10:48.370644 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:48.370483 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Apr 22 15:10:48.370644 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:48.370624 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Apr 22 15:10:48.371641 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:48.371618 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Apr 22 15:10:48.381802 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:48.381769 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c47c644-4kbbs"] Apr 22 15:10:48.490257 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:48.490219 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4v9q\" (UniqueName: \"kubernetes.io/projected/2a284a79-20a9-489b-ae26-6e692bd50ca4-kube-api-access-w4v9q\") pod \"klusterlet-addon-workmgr-6c47c644-4kbbs\" (UID: \"2a284a79-20a9-489b-ae26-6e692bd50ca4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c47c644-4kbbs" Apr 22 15:10:48.490257 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:48.490263 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a284a79-20a9-489b-ae26-6e692bd50ca4-tmp\") pod \"klusterlet-addon-workmgr-6c47c644-4kbbs\" (UID: \"2a284a79-20a9-489b-ae26-6e692bd50ca4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c47c644-4kbbs" Apr 22 15:10:48.490505 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:48.490406 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/2a284a79-20a9-489b-ae26-6e692bd50ca4-klusterlet-config\") pod \"klusterlet-addon-workmgr-6c47c644-4kbbs\" (UID: \"2a284a79-20a9-489b-ae26-6e692bd50ca4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c47c644-4kbbs" Apr 22 15:10:48.591480 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:48.591425 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w4v9q\" (UniqueName: \"kubernetes.io/projected/2a284a79-20a9-489b-ae26-6e692bd50ca4-kube-api-access-w4v9q\") pod \"klusterlet-addon-workmgr-6c47c644-4kbbs\" (UID: \"2a284a79-20a9-489b-ae26-6e692bd50ca4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c47c644-4kbbs" Apr 22 15:10:48.591480 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:48.591487 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a284a79-20a9-489b-ae26-6e692bd50ca4-tmp\") pod \"klusterlet-addon-workmgr-6c47c644-4kbbs\" (UID: \"2a284a79-20a9-489b-ae26-6e692bd50ca4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c47c644-4kbbs" Apr 22 15:10:48.591694 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:48.591539 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/2a284a79-20a9-489b-ae26-6e692bd50ca4-klusterlet-config\") pod \"klusterlet-addon-workmgr-6c47c644-4kbbs\" (UID: \"2a284a79-20a9-489b-ae26-6e692bd50ca4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c47c644-4kbbs" Apr 22 15:10:48.591907 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:48.591885 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a284a79-20a9-489b-ae26-6e692bd50ca4-tmp\") pod \"klusterlet-addon-workmgr-6c47c644-4kbbs\" (UID: \"2a284a79-20a9-489b-ae26-6e692bd50ca4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c47c644-4kbbs" Apr 22 15:10:48.594127 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:48.594106 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/2a284a79-20a9-489b-ae26-6e692bd50ca4-klusterlet-config\") pod \"klusterlet-addon-workmgr-6c47c644-4kbbs\" (UID: \"2a284a79-20a9-489b-ae26-6e692bd50ca4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c47c644-4kbbs" Apr 22 15:10:48.600464 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:48.600439 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4v9q\" (UniqueName: \"kubernetes.io/projected/2a284a79-20a9-489b-ae26-6e692bd50ca4-kube-api-access-w4v9q\") pod \"klusterlet-addon-workmgr-6c47c644-4kbbs\" (UID: \"2a284a79-20a9-489b-ae26-6e692bd50ca4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c47c644-4kbbs" Apr 22 15:10:48.683558 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:48.683459 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c47c644-4kbbs" Apr 22 15:10:48.811488 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:48.811455 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c47c644-4kbbs"] Apr 22 15:10:48.814194 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:10:48.814164 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a284a79_20a9_489b_ae26_6e692bd50ca4.slice/crio-0a2c9b2587d82b79a36c92f9502dc8141c73309f1479050950eb0049a84d34f8 WatchSource:0}: Error finding container 0a2c9b2587d82b79a36c92f9502dc8141c73309f1479050950eb0049a84d34f8: Status 404 returned error can't find the container with id 0a2c9b2587d82b79a36c92f9502dc8141c73309f1479050950eb0049a84d34f8 Apr 22 15:10:49.220403 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:49.220364 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c47c644-4kbbs" event={"ID":"2a284a79-20a9-489b-ae26-6e692bd50ca4","Type":"ContainerStarted","Data":"0a2c9b2587d82b79a36c92f9502dc8141c73309f1479050950eb0049a84d34f8"} Apr 22 15:10:54.237820 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:54.237769 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c47c644-4kbbs" event={"ID":"2a284a79-20a9-489b-ae26-6e692bd50ca4","Type":"ContainerStarted","Data":"e2e70dace473a6d1adea898feb1a6d6ee28acdb42432ef3f5f6d081a7dd4fe27"} Apr 22 15:10:54.238298 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:54.238015 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c47c644-4kbbs" Apr 22 15:10:54.239811 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:54.239789 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c47c644-4kbbs" Apr 22 15:10:54.299321 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:10:54.299272 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c47c644-4kbbs" podStartSLOduration=1.537893849 podStartE2EDuration="6.299253767s" podCreationTimestamp="2026-04-22 15:10:48 +0000 UTC" firstStartedPulling="2026-04-22 15:10:48.815999447 +0000 UTC m=+187.872737510" lastFinishedPulling="2026-04-22 15:10:53.577359365 +0000 UTC m=+192.634097428" observedRunningTime="2026-04-22 15:10:54.27642059 +0000 UTC m=+193.333158673" watchObservedRunningTime="2026-04-22 15:10:54.299253767 +0000 UTC m=+193.355991850" Apr 22 15:11:05.041993 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:05.041957 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt"] Apr 22 15:11:05.048163 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:05.048140 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt" Apr 22 15:11:05.050927 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:05.050903 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-422gz\"" Apr 22 15:11:05.051139 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:05.051122 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Apr 22 15:11:05.051657 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:05.051639 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Apr 22 15:11:05.056352 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:05.056309 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt"] Apr 22 15:11:05.126253 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:05.126218 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8dlj\" (UniqueName: \"kubernetes.io/projected/7b587be6-0633-4518-a587-37c6dec337f0-kube-api-access-g8dlj\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt\" (UID: \"7b587be6-0633-4518-a587-37c6dec337f0\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt" Apr 22 15:11:05.126458 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:05.126276 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b587be6-0633-4518-a587-37c6dec337f0-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt\" (UID: \"7b587be6-0633-4518-a587-37c6dec337f0\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt" Apr 22 15:11:05.126458 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:05.126360 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b587be6-0633-4518-a587-37c6dec337f0-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt\" (UID: \"7b587be6-0633-4518-a587-37c6dec337f0\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt" Apr 22 15:11:05.227651 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:05.227612 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g8dlj\" (UniqueName: \"kubernetes.io/projected/7b587be6-0633-4518-a587-37c6dec337f0-kube-api-access-g8dlj\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt\" (UID: \"7b587be6-0633-4518-a587-37c6dec337f0\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt" Apr 22 15:11:05.227782 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:05.227751 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b587be6-0633-4518-a587-37c6dec337f0-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt\" (UID: \"7b587be6-0633-4518-a587-37c6dec337f0\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt" Apr 22 15:11:05.227846 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:05.227793 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b587be6-0633-4518-a587-37c6dec337f0-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt\" (UID: \"7b587be6-0633-4518-a587-37c6dec337f0\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt" Apr 22 15:11:05.228130 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:05.228110 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b587be6-0633-4518-a587-37c6dec337f0-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt\" (UID: \"7b587be6-0633-4518-a587-37c6dec337f0\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt" Apr 22 15:11:05.228210 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:05.228133 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b587be6-0633-4518-a587-37c6dec337f0-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt\" (UID: \"7b587be6-0633-4518-a587-37c6dec337f0\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt" Apr 22 15:11:05.238491 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:05.238466 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8dlj\" (UniqueName: \"kubernetes.io/projected/7b587be6-0633-4518-a587-37c6dec337f0-kube-api-access-g8dlj\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt\" (UID: \"7b587be6-0633-4518-a587-37c6dec337f0\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt" Apr 22 15:11:05.357857 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:05.357754 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt" Apr 22 15:11:05.478923 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:05.478892 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt"] Apr 22 15:11:05.482445 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:11:05.482416 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b587be6_0633_4518_a587_37c6dec337f0.slice/crio-36f696b8ae7a8f7e57736413f30332dda7b1e7cf0fee154ef5fe3b47bf0f02f1 WatchSource:0}: Error finding container 36f696b8ae7a8f7e57736413f30332dda7b1e7cf0fee154ef5fe3b47bf0f02f1: Status 404 returned error can't find the container with id 36f696b8ae7a8f7e57736413f30332dda7b1e7cf0fee154ef5fe3b47bf0f02f1 Apr 22 15:11:06.271573 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:06.271537 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt" event={"ID":"7b587be6-0633-4518-a587-37c6dec337f0","Type":"ContainerStarted","Data":"36f696b8ae7a8f7e57736413f30332dda7b1e7cf0fee154ef5fe3b47bf0f02f1"} Apr 22 15:11:15.297431 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:15.297395 2577 generic.go:358] "Generic (PLEG): container finished" podID="7b587be6-0633-4518-a587-37c6dec337f0" containerID="8bb2de2a26d7dab0c0d2f40bcbc1c701402b32b2212a983e5020ab36fd3d29f1" exitCode=0 Apr 22 15:11:15.297803 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:15.297481 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt" event={"ID":"7b587be6-0633-4518-a587-37c6dec337f0","Type":"ContainerDied","Data":"8bb2de2a26d7dab0c0d2f40bcbc1c701402b32b2212a983e5020ab36fd3d29f1"} Apr 22 15:11:17.305520 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:17.305479 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt" event={"ID":"7b587be6-0633-4518-a587-37c6dec337f0","Type":"ContainerStarted","Data":"5d98a4508bf9261455970427b97b9d6ec0af7fd8a7b727a19b252037c7ab2490"} Apr 22 15:11:18.309424 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:18.309392 2577 generic.go:358] "Generic (PLEG): container finished" podID="7b587be6-0633-4518-a587-37c6dec337f0" containerID="5d98a4508bf9261455970427b97b9d6ec0af7fd8a7b727a19b252037c7ab2490" exitCode=0 Apr 22 15:11:18.309863 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:18.309454 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt" event={"ID":"7b587be6-0633-4518-a587-37c6dec337f0","Type":"ContainerDied","Data":"5d98a4508bf9261455970427b97b9d6ec0af7fd8a7b727a19b252037c7ab2490"} Apr 22 15:11:25.333873 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:25.333839 2577 generic.go:358] "Generic (PLEG): container finished" podID="7b587be6-0633-4518-a587-37c6dec337f0" containerID="695fa4fa89bdc0b476ba4f0d34c77633e9d142192f5ed4ba7241977a25405147" exitCode=0 Apr 22 15:11:25.334256 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:25.333932 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt" event={"ID":"7b587be6-0633-4518-a587-37c6dec337f0","Type":"ContainerDied","Data":"695fa4fa89bdc0b476ba4f0d34c77633e9d142192f5ed4ba7241977a25405147"} Apr 22 15:11:26.459880 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:26.459853 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt" Apr 22 15:11:26.508356 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:26.508301 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b587be6-0633-4518-a587-37c6dec337f0-util\") pod \"7b587be6-0633-4518-a587-37c6dec337f0\" (UID: \"7b587be6-0633-4518-a587-37c6dec337f0\") " Apr 22 15:11:26.508522 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:26.508380 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b587be6-0633-4518-a587-37c6dec337f0-bundle\") pod \"7b587be6-0633-4518-a587-37c6dec337f0\" (UID: \"7b587be6-0633-4518-a587-37c6dec337f0\") " Apr 22 15:11:26.508522 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:26.508476 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8dlj\" (UniqueName: \"kubernetes.io/projected/7b587be6-0633-4518-a587-37c6dec337f0-kube-api-access-g8dlj\") pod \"7b587be6-0633-4518-a587-37c6dec337f0\" (UID: \"7b587be6-0633-4518-a587-37c6dec337f0\") " Apr 22 15:11:26.509072 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:26.509034 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b587be6-0633-4518-a587-37c6dec337f0-bundle" (OuterVolumeSpecName: "bundle") pod "7b587be6-0633-4518-a587-37c6dec337f0" (UID: "7b587be6-0633-4518-a587-37c6dec337f0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 22 15:11:26.510652 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:26.510622 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b587be6-0633-4518-a587-37c6dec337f0-kube-api-access-g8dlj" (OuterVolumeSpecName: "kube-api-access-g8dlj") pod "7b587be6-0633-4518-a587-37c6dec337f0" (UID: "7b587be6-0633-4518-a587-37c6dec337f0"). InnerVolumeSpecName "kube-api-access-g8dlj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 22 15:11:26.512865 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:26.512841 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b587be6-0633-4518-a587-37c6dec337f0-util" (OuterVolumeSpecName: "util") pod "7b587be6-0633-4518-a587-37c6dec337f0" (UID: "7b587be6-0633-4518-a587-37c6dec337f0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 22 15:11:26.609181 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:26.609075 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8dlj\" (UniqueName: \"kubernetes.io/projected/7b587be6-0633-4518-a587-37c6dec337f0-kube-api-access-g8dlj\") on node \"ip-10-0-131-168.ec2.internal\" DevicePath \"\"" Apr 22 15:11:26.609181 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:26.609122 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b587be6-0633-4518-a587-37c6dec337f0-util\") on node \"ip-10-0-131-168.ec2.internal\" DevicePath \"\"" Apr 22 15:11:26.609181 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:26.609139 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b587be6-0633-4518-a587-37c6dec337f0-bundle\") on node \"ip-10-0-131-168.ec2.internal\" DevicePath \"\"" Apr 22 15:11:27.341762 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:27.341731 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt" event={"ID":"7b587be6-0633-4518-a587-37c6dec337f0","Type":"ContainerDied","Data":"36f696b8ae7a8f7e57736413f30332dda7b1e7cf0fee154ef5fe3b47bf0f02f1"} Apr 22 15:11:27.341762 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:27.341764 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="36f696b8ae7a8f7e57736413f30332dda7b1e7cf0fee154ef5fe3b47bf0f02f1" Apr 22 15:11:27.341976 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:27.341795 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dwptmt" Apr 22 15:11:42.493941 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:42.493903 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-7kvxq"] Apr 22 15:11:42.494379 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:42.494215 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b587be6-0633-4518-a587-37c6dec337f0" containerName="extract" Apr 22 15:11:42.494379 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:42.494227 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b587be6-0633-4518-a587-37c6dec337f0" containerName="extract" Apr 22 15:11:42.494379 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:42.494242 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b587be6-0633-4518-a587-37c6dec337f0" containerName="pull" Apr 22 15:11:42.494379 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:42.494247 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b587be6-0633-4518-a587-37c6dec337f0" containerName="pull" Apr 22 15:11:42.494379 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:42.494261 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b587be6-0633-4518-a587-37c6dec337f0" containerName="util" Apr 22 15:11:42.494379 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:42.494267 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b587be6-0633-4518-a587-37c6dec337f0" containerName="util" Apr 22 15:11:42.494379 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:42.494316 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="7b587be6-0633-4518-a587-37c6dec337f0" containerName="extract" Apr 22 15:11:42.497121 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:42.497104 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-7kvxq" Apr 22 15:11:42.499466 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:42.499434 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Apr 22 15:11:42.499596 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:42.499524 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-7xsqn\"" Apr 22 15:11:42.499596 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:42.499532 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Apr 22 15:11:42.506712 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:42.506689 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-7kvxq"] Apr 22 15:11:42.644071 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:42.644031 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc619330-8fe9-42e9-96e0-5e0065162392-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-7kvxq\" (UID: \"bc619330-8fe9-42e9-96e0-5e0065162392\") " pod="cert-manager/cert-manager-cainjector-68b757865b-7kvxq" Apr 22 15:11:42.644253 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:42.644087 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfhc5\" (UniqueName: \"kubernetes.io/projected/bc619330-8fe9-42e9-96e0-5e0065162392-kube-api-access-sfhc5\") pod \"cert-manager-cainjector-68b757865b-7kvxq\" (UID: \"bc619330-8fe9-42e9-96e0-5e0065162392\") " pod="cert-manager/cert-manager-cainjector-68b757865b-7kvxq" Apr 22 15:11:42.745477 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:42.745375 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc619330-8fe9-42e9-96e0-5e0065162392-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-7kvxq\" (UID: \"bc619330-8fe9-42e9-96e0-5e0065162392\") " pod="cert-manager/cert-manager-cainjector-68b757865b-7kvxq" Apr 22 15:11:42.745477 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:42.745448 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sfhc5\" (UniqueName: \"kubernetes.io/projected/bc619330-8fe9-42e9-96e0-5e0065162392-kube-api-access-sfhc5\") pod \"cert-manager-cainjector-68b757865b-7kvxq\" (UID: \"bc619330-8fe9-42e9-96e0-5e0065162392\") " pod="cert-manager/cert-manager-cainjector-68b757865b-7kvxq" Apr 22 15:11:42.753185 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:42.753154 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc619330-8fe9-42e9-96e0-5e0065162392-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-7kvxq\" (UID: \"bc619330-8fe9-42e9-96e0-5e0065162392\") " pod="cert-manager/cert-manager-cainjector-68b757865b-7kvxq" Apr 22 15:11:42.753401 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:42.753377 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfhc5\" (UniqueName: \"kubernetes.io/projected/bc619330-8fe9-42e9-96e0-5e0065162392-kube-api-access-sfhc5\") pod \"cert-manager-cainjector-68b757865b-7kvxq\" (UID: \"bc619330-8fe9-42e9-96e0-5e0065162392\") " pod="cert-manager/cert-manager-cainjector-68b757865b-7kvxq" Apr 22 15:11:42.823861 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:42.823812 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-7kvxq" Apr 22 15:11:42.944627 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:42.944522 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-7kvxq"] Apr 22 15:11:42.947097 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:11:42.947068 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc619330_8fe9_42e9_96e0_5e0065162392.slice/crio-ded74c3f9ac8d3667f61776004204dfa1594ac2b172ceafbed8d860c7ebc3dc0 WatchSource:0}: Error finding container ded74c3f9ac8d3667f61776004204dfa1594ac2b172ceafbed8d860c7ebc3dc0: Status 404 returned error can't find the container with id ded74c3f9ac8d3667f61776004204dfa1594ac2b172ceafbed8d860c7ebc3dc0 Apr 22 15:11:43.389615 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:43.389581 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-7kvxq" event={"ID":"bc619330-8fe9-42e9-96e0-5e0065162392","Type":"ContainerStarted","Data":"ded74c3f9ac8d3667f61776004204dfa1594ac2b172ceafbed8d860c7ebc3dc0"} Apr 22 15:11:46.401675 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:46.401636 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-7kvxq" event={"ID":"bc619330-8fe9-42e9-96e0-5e0065162392","Type":"ContainerStarted","Data":"482935943ec3932913b93c4a4fbc542efcf81940df4f6277e2e5d8b878a16aef"} Apr 22 15:11:46.419305 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:46.419255 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-68b757865b-7kvxq" podStartSLOduration=1.5132563989999999 podStartE2EDuration="4.419239783s" podCreationTimestamp="2026-04-22 15:11:42 +0000 UTC" firstStartedPulling="2026-04-22 15:11:42.948979791 +0000 UTC m=+242.005717854" lastFinishedPulling="2026-04-22 15:11:45.854963175 +0000 UTC m=+244.911701238" observedRunningTime="2026-04-22 15:11:46.417693787 +0000 UTC m=+245.474431873" watchObservedRunningTime="2026-04-22 15:11:46.419239783 +0000 UTC m=+245.475977868" Apr 22 15:11:50.162986 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:50.162899 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2"] Apr 22 15:11:50.185572 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:50.185534 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2"] Apr 22 15:11:50.185740 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:50.185668 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2" Apr 22 15:11:50.188314 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:50.188289 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Apr 22 15:11:50.188314 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:50.188290 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Apr 22 15:11:50.189119 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:50.189094 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-422gz\"" Apr 22 15:11:50.309391 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:50.309355 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aefe72e2-a20e-4d8f-8536-431ccd565e82-util\") pod \"c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2\" (UID: \"aefe72e2-a20e-4d8f-8536-431ccd565e82\") " pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2" Apr 22 15:11:50.309588 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:50.309415 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czlrd\" (UniqueName: \"kubernetes.io/projected/aefe72e2-a20e-4d8f-8536-431ccd565e82-kube-api-access-czlrd\") pod \"c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2\" (UID: \"aefe72e2-a20e-4d8f-8536-431ccd565e82\") " pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2" Apr 22 15:11:50.309588 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:50.309447 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aefe72e2-a20e-4d8f-8536-431ccd565e82-bundle\") pod \"c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2\" (UID: \"aefe72e2-a20e-4d8f-8536-431ccd565e82\") " pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2" Apr 22 15:11:50.410561 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:50.410516 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-czlrd\" (UniqueName: \"kubernetes.io/projected/aefe72e2-a20e-4d8f-8536-431ccd565e82-kube-api-access-czlrd\") pod \"c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2\" (UID: \"aefe72e2-a20e-4d8f-8536-431ccd565e82\") " pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2" Apr 22 15:11:50.410561 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:50.410558 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aefe72e2-a20e-4d8f-8536-431ccd565e82-bundle\") pod \"c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2\" (UID: \"aefe72e2-a20e-4d8f-8536-431ccd565e82\") " pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2" Apr 22 15:11:50.410797 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:50.410624 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aefe72e2-a20e-4d8f-8536-431ccd565e82-util\") pod \"c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2\" (UID: \"aefe72e2-a20e-4d8f-8536-431ccd565e82\") " pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2" Apr 22 15:11:50.411067 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:50.411045 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aefe72e2-a20e-4d8f-8536-431ccd565e82-bundle\") pod \"c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2\" (UID: \"aefe72e2-a20e-4d8f-8536-431ccd565e82\") " pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2" Apr 22 15:11:50.411067 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:50.411058 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aefe72e2-a20e-4d8f-8536-431ccd565e82-util\") pod \"c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2\" (UID: \"aefe72e2-a20e-4d8f-8536-431ccd565e82\") " pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2" Apr 22 15:11:50.421500 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:50.421424 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czlrd\" (UniqueName: \"kubernetes.io/projected/aefe72e2-a20e-4d8f-8536-431ccd565e82-kube-api-access-czlrd\") pod \"c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2\" (UID: \"aefe72e2-a20e-4d8f-8536-431ccd565e82\") " pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2" Apr 22 15:11:50.495793 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:50.495751 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2" Apr 22 15:11:50.624445 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:50.624318 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2"] Apr 22 15:11:50.627200 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:11:50.627172 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaefe72e2_a20e_4d8f_8536_431ccd565e82.slice/crio-b25dce68c17c408a6046fb3b1f47b58fd6e182bab8aeff9d7c319d362c575c63 WatchSource:0}: Error finding container b25dce68c17c408a6046fb3b1f47b58fd6e182bab8aeff9d7c319d362c575c63: Status 404 returned error can't find the container with id b25dce68c17c408a6046fb3b1f47b58fd6e182bab8aeff9d7c319d362c575c63 Apr 22 15:11:51.419902 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:51.419865 2577 generic.go:358] "Generic (PLEG): container finished" podID="aefe72e2-a20e-4d8f-8536-431ccd565e82" containerID="c32e834600a5036f0fb6a211e5c7f6a0f87dd067379b883b3a2fd4ff33aa8878" exitCode=0 Apr 22 15:11:51.420298 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:51.419958 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2" event={"ID":"aefe72e2-a20e-4d8f-8536-431ccd565e82","Type":"ContainerDied","Data":"c32e834600a5036f0fb6a211e5c7f6a0f87dd067379b883b3a2fd4ff33aa8878"} Apr 22 15:11:51.420298 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:51.419991 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2" event={"ID":"aefe72e2-a20e-4d8f-8536-431ccd565e82","Type":"ContainerStarted","Data":"b25dce68c17c408a6046fb3b1f47b58fd6e182bab8aeff9d7c319d362c575c63"} Apr 22 15:11:54.431235 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:54.431200 2577 generic.go:358] "Generic (PLEG): container finished" podID="aefe72e2-a20e-4d8f-8536-431ccd565e82" containerID="551bc383c504e780bc9aa919cf987de535d8d5eae5f9a7c13318063d87ce3b4e" exitCode=0 Apr 22 15:11:54.431637 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:54.431291 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2" event={"ID":"aefe72e2-a20e-4d8f-8536-431ccd565e82","Type":"ContainerDied","Data":"551bc383c504e780bc9aa919cf987de535d8d5eae5f9a7c13318063d87ce3b4e"} Apr 22 15:11:55.436726 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:55.436684 2577 generic.go:358] "Generic (PLEG): container finished" podID="aefe72e2-a20e-4d8f-8536-431ccd565e82" containerID="bf9fb35d676766813e3fcafa4784e0be24395c5d2c5aed7c756fec6c2192eef8" exitCode=0 Apr 22 15:11:55.437129 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:55.436735 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2" event={"ID":"aefe72e2-a20e-4d8f-8536-431ccd565e82","Type":"ContainerDied","Data":"bf9fb35d676766813e3fcafa4784e0be24395c5d2c5aed7c756fec6c2192eef8"} Apr 22 15:11:56.559828 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:56.559803 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2" Apr 22 15:11:56.662261 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:56.662233 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aefe72e2-a20e-4d8f-8536-431ccd565e82-bundle\") pod \"aefe72e2-a20e-4d8f-8536-431ccd565e82\" (UID: \"aefe72e2-a20e-4d8f-8536-431ccd565e82\") " Apr 22 15:11:56.662451 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:56.662345 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czlrd\" (UniqueName: \"kubernetes.io/projected/aefe72e2-a20e-4d8f-8536-431ccd565e82-kube-api-access-czlrd\") pod \"aefe72e2-a20e-4d8f-8536-431ccd565e82\" (UID: \"aefe72e2-a20e-4d8f-8536-431ccd565e82\") " Apr 22 15:11:56.662451 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:56.662387 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aefe72e2-a20e-4d8f-8536-431ccd565e82-util\") pod \"aefe72e2-a20e-4d8f-8536-431ccd565e82\" (UID: \"aefe72e2-a20e-4d8f-8536-431ccd565e82\") " Apr 22 15:11:56.662662 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:56.662638 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aefe72e2-a20e-4d8f-8536-431ccd565e82-bundle" (OuterVolumeSpecName: "bundle") pod "aefe72e2-a20e-4d8f-8536-431ccd565e82" (UID: "aefe72e2-a20e-4d8f-8536-431ccd565e82"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 22 15:11:56.664345 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:56.664310 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aefe72e2-a20e-4d8f-8536-431ccd565e82-kube-api-access-czlrd" (OuterVolumeSpecName: "kube-api-access-czlrd") pod "aefe72e2-a20e-4d8f-8536-431ccd565e82" (UID: "aefe72e2-a20e-4d8f-8536-431ccd565e82"). InnerVolumeSpecName "kube-api-access-czlrd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 22 15:11:56.669411 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:56.669369 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aefe72e2-a20e-4d8f-8536-431ccd565e82-util" (OuterVolumeSpecName: "util") pod "aefe72e2-a20e-4d8f-8536-431ccd565e82" (UID: "aefe72e2-a20e-4d8f-8536-431ccd565e82"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 22 15:11:56.763222 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:56.763192 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aefe72e2-a20e-4d8f-8536-431ccd565e82-bundle\") on node \"ip-10-0-131-168.ec2.internal\" DevicePath \"\"" Apr 22 15:11:56.763222 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:56.763222 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czlrd\" (UniqueName: \"kubernetes.io/projected/aefe72e2-a20e-4d8f-8536-431ccd565e82-kube-api-access-czlrd\") on node \"ip-10-0-131-168.ec2.internal\" DevicePath \"\"" Apr 22 15:11:56.763426 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:56.763231 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aefe72e2-a20e-4d8f-8536-431ccd565e82-util\") on node \"ip-10-0-131-168.ec2.internal\" DevicePath \"\"" Apr 22 15:11:57.444204 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:57.444176 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2" Apr 22 15:11:57.444389 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:57.444179 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e5f5t2" event={"ID":"aefe72e2-a20e-4d8f-8536-431ccd565e82","Type":"ContainerDied","Data":"b25dce68c17c408a6046fb3b1f47b58fd6e182bab8aeff9d7c319d362c575c63"} Apr 22 15:11:57.444389 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:11:57.444289 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b25dce68c17c408a6046fb3b1f47b58fd6e182bab8aeff9d7c319d362c575c63" Apr 22 15:12:03.263693 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:12:03.263659 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-operator-747c5859c7-dsv7s"] Apr 22 15:12:03.264078 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:12:03.263976 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aefe72e2-a20e-4d8f-8536-431ccd565e82" containerName="util" Apr 22 15:12:03.264078 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:12:03.263988 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aefe72e2-a20e-4d8f-8536-431ccd565e82" containerName="util" Apr 22 15:12:03.264078 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:12:03.264012 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aefe72e2-a20e-4d8f-8536-431ccd565e82" containerName="pull" Apr 22 15:12:03.264078 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:12:03.264017 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aefe72e2-a20e-4d8f-8536-431ccd565e82" containerName="pull" Apr 22 15:12:03.264078 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:12:03.264029 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aefe72e2-a20e-4d8f-8536-431ccd565e82" containerName="extract" Apr 22 15:12:03.264078 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:12:03.264037 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aefe72e2-a20e-4d8f-8536-431ccd565e82" containerName="extract" Apr 22 15:12:03.264260 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:12:03.264082 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="aefe72e2-a20e-4d8f-8536-431ccd565e82" containerName="extract" Apr 22 15:12:03.269213 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:12:03.269194 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-747c5859c7-dsv7s" Apr 22 15:12:03.271658 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:12:03.271640 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-operator-dockercfg-dfjsf\"" Apr 22 15:12:03.271823 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:12:03.271810 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\"" Apr 22 15:12:03.272570 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:12:03.272551 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\"" Apr 22 15:12:03.276848 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:12:03.276586 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-747c5859c7-dsv7s"] Apr 22 15:12:03.420127 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:12:03.420092 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/818a8fea-af99-4edb-bc06-53b06aef5fe7-tmp\") pod \"jobset-operator-747c5859c7-dsv7s\" (UID: \"818a8fea-af99-4edb-bc06-53b06aef5fe7\") " pod="openshift-jobset-operator/jobset-operator-747c5859c7-dsv7s" Apr 22 15:12:03.420127 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:12:03.420131 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l44q9\" (UniqueName: \"kubernetes.io/projected/818a8fea-af99-4edb-bc06-53b06aef5fe7-kube-api-access-l44q9\") pod \"jobset-operator-747c5859c7-dsv7s\" (UID: \"818a8fea-af99-4edb-bc06-53b06aef5fe7\") " pod="openshift-jobset-operator/jobset-operator-747c5859c7-dsv7s" Apr 22 15:12:03.520833 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:12:03.520746 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/818a8fea-af99-4edb-bc06-53b06aef5fe7-tmp\") pod \"jobset-operator-747c5859c7-dsv7s\" (UID: \"818a8fea-af99-4edb-bc06-53b06aef5fe7\") " pod="openshift-jobset-operator/jobset-operator-747c5859c7-dsv7s" Apr 22 15:12:03.520833 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:12:03.520782 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l44q9\" (UniqueName: \"kubernetes.io/projected/818a8fea-af99-4edb-bc06-53b06aef5fe7-kube-api-access-l44q9\") pod \"jobset-operator-747c5859c7-dsv7s\" (UID: \"818a8fea-af99-4edb-bc06-53b06aef5fe7\") " pod="openshift-jobset-operator/jobset-operator-747c5859c7-dsv7s" Apr 22 15:12:03.521184 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:12:03.521166 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/818a8fea-af99-4edb-bc06-53b06aef5fe7-tmp\") pod \"jobset-operator-747c5859c7-dsv7s\" (UID: \"818a8fea-af99-4edb-bc06-53b06aef5fe7\") " pod="openshift-jobset-operator/jobset-operator-747c5859c7-dsv7s" Apr 22 15:12:03.530915 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:12:03.530885 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l44q9\" (UniqueName: \"kubernetes.io/projected/818a8fea-af99-4edb-bc06-53b06aef5fe7-kube-api-access-l44q9\") pod \"jobset-operator-747c5859c7-dsv7s\" (UID: \"818a8fea-af99-4edb-bc06-53b06aef5fe7\") " pod="openshift-jobset-operator/jobset-operator-747c5859c7-dsv7s" Apr 22 15:12:03.579745 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:12:03.579712 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-747c5859c7-dsv7s" Apr 22 15:12:03.700536 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:12:03.700514 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-747c5859c7-dsv7s"] Apr 22 15:12:03.703145 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:12:03.703123 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod818a8fea_af99_4edb_bc06_53b06aef5fe7.slice/crio-a9c1260d7aab1708b2e37ffadebdf5d83609d786abaab8accf284a785a92550e WatchSource:0}: Error finding container a9c1260d7aab1708b2e37ffadebdf5d83609d786abaab8accf284a785a92550e: Status 404 returned error can't find the container with id a9c1260d7aab1708b2e37ffadebdf5d83609d786abaab8accf284a785a92550e Apr 22 15:12:04.465617 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:12:04.465585 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-747c5859c7-dsv7s" event={"ID":"818a8fea-af99-4edb-bc06-53b06aef5fe7","Type":"ContainerStarted","Data":"a9c1260d7aab1708b2e37ffadebdf5d83609d786abaab8accf284a785a92550e"} Apr 22 15:12:06.474422 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:12:06.474380 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-747c5859c7-dsv7s" event={"ID":"818a8fea-af99-4edb-bc06-53b06aef5fe7","Type":"ContainerStarted","Data":"2255df46fed8f8dc1b7458c8dfa0be35138ce79333b2ecb6c24463a4cf96a102"} Apr 22 15:12:06.490449 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:12:06.490401 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-operator-747c5859c7-dsv7s" podStartSLOduration=1.264668887 podStartE2EDuration="3.490387698s" podCreationTimestamp="2026-04-22 15:12:03 +0000 UTC" firstStartedPulling="2026-04-22 15:12:03.704699189 +0000 UTC m=+262.761437253" lastFinishedPulling="2026-04-22 15:12:05.930417986 +0000 UTC m=+264.987156064" observedRunningTime="2026-04-22 15:12:06.488819142 +0000 UTC m=+265.545557227" watchObservedRunningTime="2026-04-22 15:12:06.490387698 +0000 UTC m=+265.547125783" Apr 22 15:12:41.409890 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:12:41.409860 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-s79j7_89e3aa95-bc39-4144-bc81-144160cf50eb/console-operator/1.log" Apr 22 15:12:41.409890 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:12:41.409887 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-s79j7_89e3aa95-bc39-4144-bc81-144160cf50eb/console-operator/1.log" Apr 22 15:12:41.419476 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:12:41.419457 2577 kubelet.go:1628] "Image garbage collection succeeded" Apr 22 15:14:30.060070 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:14:30.060038 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-gskjb/test-trainjob-pb95z-node-0-0-m6hwq"] Apr 22 15:14:30.062988 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:14:30.062971 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-gskjb/test-trainjob-pb95z-node-0-0-m6hwq" Apr 22 15:14:30.065312 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:14:30.065286 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-gskjb\"/\"default-dockercfg-qkbvq\"" Apr 22 15:14:30.065438 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:14:30.065361 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-gskjb\"/\"openshift-service-ca.crt\"" Apr 22 15:14:30.065438 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:14:30.065401 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-gskjb\"/\"kube-root-ca.crt\"" Apr 22 15:14:30.075269 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:14:30.075245 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-gskjb/test-trainjob-pb95z-node-0-0-m6hwq"] Apr 22 15:14:30.168177 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:14:30.168150 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lqnq\" (UniqueName: \"kubernetes.io/projected/0a37ff90-cfd8-4791-9a43-9069c09b98d6-kube-api-access-8lqnq\") pod \"test-trainjob-pb95z-node-0-0-m6hwq\" (UID: \"0a37ff90-cfd8-4791-9a43-9069c09b98d6\") " pod="test-ns-gskjb/test-trainjob-pb95z-node-0-0-m6hwq" Apr 22 15:14:30.269157 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:14:30.269123 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8lqnq\" (UniqueName: \"kubernetes.io/projected/0a37ff90-cfd8-4791-9a43-9069c09b98d6-kube-api-access-8lqnq\") pod \"test-trainjob-pb95z-node-0-0-m6hwq\" (UID: \"0a37ff90-cfd8-4791-9a43-9069c09b98d6\") " pod="test-ns-gskjb/test-trainjob-pb95z-node-0-0-m6hwq" Apr 22 15:14:30.277052 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:14:30.277023 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lqnq\" (UniqueName: \"kubernetes.io/projected/0a37ff90-cfd8-4791-9a43-9069c09b98d6-kube-api-access-8lqnq\") pod \"test-trainjob-pb95z-node-0-0-m6hwq\" (UID: \"0a37ff90-cfd8-4791-9a43-9069c09b98d6\") " pod="test-ns-gskjb/test-trainjob-pb95z-node-0-0-m6hwq" Apr 22 15:14:30.371935 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:14:30.371847 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-gskjb/test-trainjob-pb95z-node-0-0-m6hwq" Apr 22 15:14:30.493116 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:14:30.492966 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-gskjb/test-trainjob-pb95z-node-0-0-m6hwq"] Apr 22 15:14:30.496995 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:14:30.496970 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a37ff90_cfd8_4791_9a43_9069c09b98d6.slice/crio-f6786dd3ee0ba69e82bbfb69d448c14470f77bb44e0993a1597b73db4f20e594 WatchSource:0}: Error finding container f6786dd3ee0ba69e82bbfb69d448c14470f77bb44e0993a1597b73db4f20e594: Status 404 returned error can't find the container with id f6786dd3ee0ba69e82bbfb69d448c14470f77bb44e0993a1597b73db4f20e594 Apr 22 15:14:30.499074 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:14:30.499058 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 22 15:14:30.940568 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:14:30.940534 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-gskjb/test-trainjob-pb95z-node-0-0-m6hwq" event={"ID":"0a37ff90-cfd8-4791-9a43-9069c09b98d6","Type":"ContainerStarted","Data":"f6786dd3ee0ba69e82bbfb69d448c14470f77bb44e0993a1597b73db4f20e594"} Apr 22 15:17:41.436196 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:17:41.436167 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-s79j7_89e3aa95-bc39-4144-bc81-144160cf50eb/console-operator/1.log" Apr 22 15:17:41.437891 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:17:41.437871 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-s79j7_89e3aa95-bc39-4144-bc81-144160cf50eb/console-operator/1.log" Apr 22 15:20:16.148221 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:16.148102 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-gskjb/test-trainjob-pb95z-node-0-0-m6hwq" event={"ID":"0a37ff90-cfd8-4791-9a43-9069c09b98d6","Type":"ContainerStarted","Data":"9b4efa352432219e37658b1307674f4006999726ecfa247787c36ddf5297fc6f"} Apr 22 15:20:16.150739 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:16.150718 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-gskjb\"/\"default-dockercfg-qkbvq\"" Apr 22 15:20:16.171610 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:16.171549 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-gskjb/test-trainjob-pb95z-node-0-0-m6hwq" podStartSLOduration=0.877666699 podStartE2EDuration="5m46.171530459s" podCreationTimestamp="2026-04-22 15:14:30 +0000 UTC" firstStartedPulling="2026-04-22 15:14:30.49919242 +0000 UTC m=+409.555930483" lastFinishedPulling="2026-04-22 15:20:15.793056166 +0000 UTC m=+754.849794243" observedRunningTime="2026-04-22 15:20:16.170371438 +0000 UTC m=+755.227109527" watchObservedRunningTime="2026-04-22 15:20:16.171530459 +0000 UTC m=+755.228268544" Apr 22 15:20:16.268283 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:16.268247 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-gskjb\"/\"kube-root-ca.crt\"" Apr 22 15:20:16.278687 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:16.278661 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-gskjb\"/\"openshift-service-ca.crt\"" Apr 22 15:20:22.169206 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:22.169170 2577 generic.go:358] "Generic (PLEG): container finished" podID="0a37ff90-cfd8-4791-9a43-9069c09b98d6" containerID="9b4efa352432219e37658b1307674f4006999726ecfa247787c36ddf5297fc6f" exitCode=0 Apr 22 15:20:22.169649 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:22.169243 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-gskjb/test-trainjob-pb95z-node-0-0-m6hwq" event={"ID":"0a37ff90-cfd8-4791-9a43-9069c09b98d6","Type":"ContainerDied","Data":"9b4efa352432219e37658b1307674f4006999726ecfa247787c36ddf5297fc6f"} Apr 22 15:20:23.300423 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:23.300401 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-gskjb/test-trainjob-pb95z-node-0-0-m6hwq" Apr 22 15:20:23.422429 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:23.422349 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lqnq\" (UniqueName: \"kubernetes.io/projected/0a37ff90-cfd8-4791-9a43-9069c09b98d6-kube-api-access-8lqnq\") pod \"0a37ff90-cfd8-4791-9a43-9069c09b98d6\" (UID: \"0a37ff90-cfd8-4791-9a43-9069c09b98d6\") " Apr 22 15:20:23.424449 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:23.424419 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a37ff90-cfd8-4791-9a43-9069c09b98d6-kube-api-access-8lqnq" (OuterVolumeSpecName: "kube-api-access-8lqnq") pod "0a37ff90-cfd8-4791-9a43-9069c09b98d6" (UID: "0a37ff90-cfd8-4791-9a43-9069c09b98d6"). InnerVolumeSpecName "kube-api-access-8lqnq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 22 15:20:23.523624 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:23.523596 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lqnq\" (UniqueName: \"kubernetes.io/projected/0a37ff90-cfd8-4791-9a43-9069c09b98d6-kube-api-access-8lqnq\") on node \"ip-10-0-131-168.ec2.internal\" DevicePath \"\"" Apr 22 15:20:24.176002 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:24.175966 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-gskjb/test-trainjob-pb95z-node-0-0-m6hwq" event={"ID":"0a37ff90-cfd8-4791-9a43-9069c09b98d6","Type":"ContainerDied","Data":"f6786dd3ee0ba69e82bbfb69d448c14470f77bb44e0993a1597b73db4f20e594"} Apr 22 15:20:24.176002 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:24.176005 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f6786dd3ee0ba69e82bbfb69d448c14470f77bb44e0993a1597b73db4f20e594" Apr 22 15:20:24.176229 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:24.175978 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-gskjb/test-trainjob-pb95z-node-0-0-m6hwq" Apr 22 15:20:24.752865 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:24.752832 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-p4l27/test-trainjob-bn2mq-node-0-0-gck6z"] Apr 22 15:20:24.753382 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:24.753285 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a37ff90-cfd8-4791-9a43-9069c09b98d6" containerName="node" Apr 22 15:20:24.753382 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:24.753303 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a37ff90-cfd8-4791-9a43-9069c09b98d6" containerName="node" Apr 22 15:20:24.753515 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:24.753440 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="0a37ff90-cfd8-4791-9a43-9069c09b98d6" containerName="node" Apr 22 15:20:24.769499 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:24.769472 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-p4l27/test-trainjob-bn2mq-node-0-0-gck6z"] Apr 22 15:20:24.769619 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:24.769584 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-p4l27/test-trainjob-bn2mq-node-0-0-gck6z" Apr 22 15:20:24.772081 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:24.772058 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-p4l27\"/\"openshift-service-ca.crt\"" Apr 22 15:20:24.772200 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:24.772124 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-p4l27\"/\"kube-root-ca.crt\"" Apr 22 15:20:24.772412 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:24.772393 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-p4l27\"/\"default-dockercfg-cg62q\"" Apr 22 15:20:24.937564 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:24.937528 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl6bg\" (UniqueName: \"kubernetes.io/projected/9c911947-e5d7-48eb-b435-89efde55eb66-kube-api-access-cl6bg\") pod \"test-trainjob-bn2mq-node-0-0-gck6z\" (UID: \"9c911947-e5d7-48eb-b435-89efde55eb66\") " pod="test-ns-p4l27/test-trainjob-bn2mq-node-0-0-gck6z" Apr 22 15:20:25.038600 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:25.038520 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cl6bg\" (UniqueName: \"kubernetes.io/projected/9c911947-e5d7-48eb-b435-89efde55eb66-kube-api-access-cl6bg\") pod \"test-trainjob-bn2mq-node-0-0-gck6z\" (UID: \"9c911947-e5d7-48eb-b435-89efde55eb66\") " pod="test-ns-p4l27/test-trainjob-bn2mq-node-0-0-gck6z" Apr 22 15:20:25.049118 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:25.049083 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl6bg\" (UniqueName: \"kubernetes.io/projected/9c911947-e5d7-48eb-b435-89efde55eb66-kube-api-access-cl6bg\") pod \"test-trainjob-bn2mq-node-0-0-gck6z\" (UID: \"9c911947-e5d7-48eb-b435-89efde55eb66\") " pod="test-ns-p4l27/test-trainjob-bn2mq-node-0-0-gck6z" Apr 22 15:20:25.079170 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:25.079136 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-p4l27/test-trainjob-bn2mq-node-0-0-gck6z" Apr 22 15:20:25.219081 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:25.219048 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-p4l27/test-trainjob-bn2mq-node-0-0-gck6z"] Apr 22 15:20:25.222314 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:20:25.222283 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9c911947_e5d7_48eb_b435_89efde55eb66.slice/crio-62dfe72c4335efe4061605c6c16d9d328bc6c6d33d4da17f5d2d266de570c0aa WatchSource:0}: Error finding container 62dfe72c4335efe4061605c6c16d9d328bc6c6d33d4da17f5d2d266de570c0aa: Status 404 returned error can't find the container with id 62dfe72c4335efe4061605c6c16d9d328bc6c6d33d4da17f5d2d266de570c0aa Apr 22 15:20:25.224397 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:25.224380 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 22 15:20:26.192786 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:20:26.192728 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-p4l27/test-trainjob-bn2mq-node-0-0-gck6z" event={"ID":"9c911947-e5d7-48eb-b435-89efde55eb66","Type":"ContainerStarted","Data":"62dfe72c4335efe4061605c6c16d9d328bc6c6d33d4da17f5d2d266de570c0aa"} Apr 22 15:22:41.462341 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:22:41.462298 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-s79j7_89e3aa95-bc39-4144-bc81-144160cf50eb/console-operator/1.log" Apr 22 15:22:41.462857 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:22:41.462298 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-s79j7_89e3aa95-bc39-4144-bc81-144160cf50eb/console-operator/1.log" Apr 22 15:25:18.209344 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:25:18.209298 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-p4l27/test-trainjob-bn2mq-node-0-0-gck6z" event={"ID":"9c911947-e5d7-48eb-b435-89efde55eb66","Type":"ContainerStarted","Data":"298727b0cb0b730affe0cfcc4bb3902c72a43cecf7e6eda1cec506449df08ae9"} Apr 22 15:25:18.248125 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:25:18.248073 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-p4l27/test-trainjob-bn2mq-node-0-0-gck6z" podStartSLOduration=2.21563882 podStartE2EDuration="4m54.248057809s" podCreationTimestamp="2026-04-22 15:20:24 +0000 UTC" firstStartedPulling="2026-04-22 15:20:25.224545101 +0000 UTC m=+764.281283164" lastFinishedPulling="2026-04-22 15:25:17.25696409 +0000 UTC m=+1056.313702153" observedRunningTime="2026-04-22 15:25:18.24747419 +0000 UTC m=+1057.304212274" watchObservedRunningTime="2026-04-22 15:25:18.248057809 +0000 UTC m=+1057.304795893" Apr 22 15:25:23.227907 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:25:23.227828 2577 generic.go:358] "Generic (PLEG): container finished" podID="9c911947-e5d7-48eb-b435-89efde55eb66" containerID="298727b0cb0b730affe0cfcc4bb3902c72a43cecf7e6eda1cec506449df08ae9" exitCode=0 Apr 22 15:25:23.228284 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:25:23.227907 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-p4l27/test-trainjob-bn2mq-node-0-0-gck6z" event={"ID":"9c911947-e5d7-48eb-b435-89efde55eb66","Type":"ContainerDied","Data":"298727b0cb0b730affe0cfcc4bb3902c72a43cecf7e6eda1cec506449df08ae9"} Apr 22 15:25:24.395414 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:25:24.395388 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-p4l27/test-trainjob-bn2mq-node-0-0-gck6z" Apr 22 15:25:24.506667 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:25:24.506588 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cl6bg\" (UniqueName: \"kubernetes.io/projected/9c911947-e5d7-48eb-b435-89efde55eb66-kube-api-access-cl6bg\") pod \"9c911947-e5d7-48eb-b435-89efde55eb66\" (UID: \"9c911947-e5d7-48eb-b435-89efde55eb66\") " Apr 22 15:25:24.508744 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:25:24.508711 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c911947-e5d7-48eb-b435-89efde55eb66-kube-api-access-cl6bg" (OuterVolumeSpecName: "kube-api-access-cl6bg") pod "9c911947-e5d7-48eb-b435-89efde55eb66" (UID: "9c911947-e5d7-48eb-b435-89efde55eb66"). InnerVolumeSpecName "kube-api-access-cl6bg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 22 15:25:24.607441 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:25:24.607401 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cl6bg\" (UniqueName: \"kubernetes.io/projected/9c911947-e5d7-48eb-b435-89efde55eb66-kube-api-access-cl6bg\") on node \"ip-10-0-131-168.ec2.internal\" DevicePath \"\"" Apr 22 15:25:25.235593 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:25:25.235560 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-p4l27/test-trainjob-bn2mq-node-0-0-gck6z" event={"ID":"9c911947-e5d7-48eb-b435-89efde55eb66","Type":"ContainerDied","Data":"62dfe72c4335efe4061605c6c16d9d328bc6c6d33d4da17f5d2d266de570c0aa"} Apr 22 15:25:25.235593 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:25:25.235584 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-p4l27/test-trainjob-bn2mq-node-0-0-gck6z" Apr 22 15:25:25.235593 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:25:25.235594 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="62dfe72c4335efe4061605c6c16d9d328bc6c6d33d4da17f5d2d266de570c0aa" Apr 22 15:25:26.187738 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:25:26.187703 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-4ltzp/test-trainjob-jqpl7-node-0-0-nv7mg"] Apr 22 15:25:26.188111 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:25:26.188009 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c911947-e5d7-48eb-b435-89efde55eb66" containerName="node" Apr 22 15:25:26.188111 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:25:26.188020 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c911947-e5d7-48eb-b435-89efde55eb66" containerName="node" Apr 22 15:25:26.188111 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:25:26.188071 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="9c911947-e5d7-48eb-b435-89efde55eb66" containerName="node" Apr 22 15:25:26.310059 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:25:26.310016 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-4ltzp/test-trainjob-jqpl7-node-0-0-nv7mg"] Apr 22 15:25:26.310247 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:25:26.310167 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-4ltzp/test-trainjob-jqpl7-node-0-0-nv7mg" Apr 22 15:25:26.312955 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:25:26.312930 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-4ltzp\"/\"openshift-service-ca.crt\"" Apr 22 15:25:26.312955 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:25:26.312946 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-4ltzp\"/\"default-dockercfg-9h9ph\"" Apr 22 15:25:26.313911 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:25:26.313896 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-4ltzp\"/\"kube-root-ca.crt\"" Apr 22 15:25:26.422186 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:25:26.422148 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh9tk\" (UniqueName: \"kubernetes.io/projected/d4f80d2f-e21c-4696-9579-3c6bc9148b03-kube-api-access-rh9tk\") pod \"test-trainjob-jqpl7-node-0-0-nv7mg\" (UID: \"d4f80d2f-e21c-4696-9579-3c6bc9148b03\") " pod="test-ns-4ltzp/test-trainjob-jqpl7-node-0-0-nv7mg" Apr 22 15:25:26.523623 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:25:26.523588 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rh9tk\" (UniqueName: \"kubernetes.io/projected/d4f80d2f-e21c-4696-9579-3c6bc9148b03-kube-api-access-rh9tk\") pod \"test-trainjob-jqpl7-node-0-0-nv7mg\" (UID: \"d4f80d2f-e21c-4696-9579-3c6bc9148b03\") " pod="test-ns-4ltzp/test-trainjob-jqpl7-node-0-0-nv7mg" Apr 22 15:25:26.532578 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:25:26.532551 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh9tk\" (UniqueName: \"kubernetes.io/projected/d4f80d2f-e21c-4696-9579-3c6bc9148b03-kube-api-access-rh9tk\") pod \"test-trainjob-jqpl7-node-0-0-nv7mg\" (UID: \"d4f80d2f-e21c-4696-9579-3c6bc9148b03\") " pod="test-ns-4ltzp/test-trainjob-jqpl7-node-0-0-nv7mg" Apr 22 15:25:26.619680 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:25:26.619643 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-4ltzp/test-trainjob-jqpl7-node-0-0-nv7mg" Apr 22 15:25:26.749990 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:25:26.749957 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-4ltzp/test-trainjob-jqpl7-node-0-0-nv7mg"] Apr 22 15:25:26.753916 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:25:26.753891 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd4f80d2f_e21c_4696_9579_3c6bc9148b03.slice/crio-0f6560e97b50b85f691440512abc10c371261114ffed7a27359215914c67d03a WatchSource:0}: Error finding container 0f6560e97b50b85f691440512abc10c371261114ffed7a27359215914c67d03a: Status 404 returned error can't find the container with id 0f6560e97b50b85f691440512abc10c371261114ffed7a27359215914c67d03a Apr 22 15:25:26.755700 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:25:26.755684 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 22 15:25:27.243431 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:25:27.243399 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-4ltzp/test-trainjob-jqpl7-node-0-0-nv7mg" event={"ID":"d4f80d2f-e21c-4696-9579-3c6bc9148b03","Type":"ContainerStarted","Data":"0f6560e97b50b85f691440512abc10c371261114ffed7a27359215914c67d03a"} Apr 22 15:26:53.568397 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:26:53.568353 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-4ltzp/test-trainjob-jqpl7-node-0-0-nv7mg" event={"ID":"d4f80d2f-e21c-4696-9579-3c6bc9148b03","Type":"ContainerStarted","Data":"6e9a9ae56b885847dfe74839afdab2dfe1251119577f2e6aaaced27fc9559a9f"} Apr 22 15:26:53.587318 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:26:53.587270 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-4ltzp/test-trainjob-jqpl7-node-0-0-nv7mg" podStartSLOduration=1.481819244 podStartE2EDuration="1m27.587253277s" podCreationTimestamp="2026-04-22 15:25:26 +0000 UTC" firstStartedPulling="2026-04-22 15:25:26.755807457 +0000 UTC m=+1065.812545523" lastFinishedPulling="2026-04-22 15:26:52.861241487 +0000 UTC m=+1151.917979556" observedRunningTime="2026-04-22 15:26:53.584923147 +0000 UTC m=+1152.641661230" watchObservedRunningTime="2026-04-22 15:26:53.587253277 +0000 UTC m=+1152.643991360" Apr 22 15:26:55.576478 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:26:55.576446 2577 generic.go:358] "Generic (PLEG): container finished" podID="d4f80d2f-e21c-4696-9579-3c6bc9148b03" containerID="6e9a9ae56b885847dfe74839afdab2dfe1251119577f2e6aaaced27fc9559a9f" exitCode=0 Apr 22 15:26:55.576806 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:26:55.576522 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-4ltzp/test-trainjob-jqpl7-node-0-0-nv7mg" event={"ID":"d4f80d2f-e21c-4696-9579-3c6bc9148b03","Type":"ContainerDied","Data":"6e9a9ae56b885847dfe74839afdab2dfe1251119577f2e6aaaced27fc9559a9f"} Apr 22 15:26:56.725202 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:26:56.725176 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-4ltzp/test-trainjob-jqpl7-node-0-0-nv7mg" Apr 22 15:26:56.790777 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:26:56.790746 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rh9tk\" (UniqueName: \"kubernetes.io/projected/d4f80d2f-e21c-4696-9579-3c6bc9148b03-kube-api-access-rh9tk\") pod \"d4f80d2f-e21c-4696-9579-3c6bc9148b03\" (UID: \"d4f80d2f-e21c-4696-9579-3c6bc9148b03\") " Apr 22 15:26:56.792783 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:26:56.792758 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4f80d2f-e21c-4696-9579-3c6bc9148b03-kube-api-access-rh9tk" (OuterVolumeSpecName: "kube-api-access-rh9tk") pod "d4f80d2f-e21c-4696-9579-3c6bc9148b03" (UID: "d4f80d2f-e21c-4696-9579-3c6bc9148b03"). InnerVolumeSpecName "kube-api-access-rh9tk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 22 15:26:56.892164 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:26:56.892105 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rh9tk\" (UniqueName: \"kubernetes.io/projected/d4f80d2f-e21c-4696-9579-3c6bc9148b03-kube-api-access-rh9tk\") on node \"ip-10-0-131-168.ec2.internal\" DevicePath \"\"" Apr 22 15:26:57.582791 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:26:57.582759 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-4ltzp/test-trainjob-jqpl7-node-0-0-nv7mg" event={"ID":"d4f80d2f-e21c-4696-9579-3c6bc9148b03","Type":"ContainerDied","Data":"0f6560e97b50b85f691440512abc10c371261114ffed7a27359215914c67d03a"} Apr 22 15:26:57.582791 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:26:57.582790 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f6560e97b50b85f691440512abc10c371261114ffed7a27359215914c67d03a" Apr 22 15:26:57.582976 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:26:57.582797 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-4ltzp/test-trainjob-jqpl7-node-0-0-nv7mg" Apr 22 15:26:58.669612 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:26:58.669581 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-cwv8s/test-trainjob-nwlks-node-0-0-s86pr"] Apr 22 15:26:58.670037 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:26:58.669890 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4f80d2f-e21c-4696-9579-3c6bc9148b03" containerName="node" Apr 22 15:26:58.670037 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:26:58.669902 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4f80d2f-e21c-4696-9579-3c6bc9148b03" containerName="node" Apr 22 15:26:58.670037 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:26:58.669962 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="d4f80d2f-e21c-4696-9579-3c6bc9148b03" containerName="node" Apr 22 15:26:58.705950 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:26:58.705923 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-cwv8s/test-trainjob-nwlks-node-0-0-s86pr"] Apr 22 15:26:58.706085 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:26:58.706025 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-cwv8s/test-trainjob-nwlks-node-0-0-s86pr" Apr 22 15:26:58.708809 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:26:58.708786 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-cwv8s\"/\"kube-root-ca.crt\"" Apr 22 15:26:58.708938 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:26:58.708844 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-cwv8s\"/\"openshift-service-ca.crt\"" Apr 22 15:26:58.709866 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:26:58.709850 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-cwv8s\"/\"default-dockercfg-qf6sj\"" Apr 22 15:26:58.807441 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:26:58.807412 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvrdk\" (UniqueName: \"kubernetes.io/projected/6331ad26-270b-42c7-978a-1df2a7a73b4e-kube-api-access-nvrdk\") pod \"test-trainjob-nwlks-node-0-0-s86pr\" (UID: \"6331ad26-270b-42c7-978a-1df2a7a73b4e\") " pod="test-ns-cwv8s/test-trainjob-nwlks-node-0-0-s86pr" Apr 22 15:26:58.908704 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:26:58.908673 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nvrdk\" (UniqueName: \"kubernetes.io/projected/6331ad26-270b-42c7-978a-1df2a7a73b4e-kube-api-access-nvrdk\") pod \"test-trainjob-nwlks-node-0-0-s86pr\" (UID: \"6331ad26-270b-42c7-978a-1df2a7a73b4e\") " pod="test-ns-cwv8s/test-trainjob-nwlks-node-0-0-s86pr" Apr 22 15:26:58.917859 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:26:58.917822 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvrdk\" (UniqueName: \"kubernetes.io/projected/6331ad26-270b-42c7-978a-1df2a7a73b4e-kube-api-access-nvrdk\") pod \"test-trainjob-nwlks-node-0-0-s86pr\" (UID: \"6331ad26-270b-42c7-978a-1df2a7a73b4e\") " pod="test-ns-cwv8s/test-trainjob-nwlks-node-0-0-s86pr" Apr 22 15:26:59.018531 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:26:59.018499 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-cwv8s/test-trainjob-nwlks-node-0-0-s86pr" Apr 22 15:26:59.149757 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:26:59.149732 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-cwv8s/test-trainjob-nwlks-node-0-0-s86pr"] Apr 22 15:26:59.151677 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:26:59.151648 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6331ad26_270b_42c7_978a_1df2a7a73b4e.slice/crio-5df3c77cbc0451e06e595d6dbf70f73b19a6b50ffc85dcd9dbd7cfd31bd63202 WatchSource:0}: Error finding container 5df3c77cbc0451e06e595d6dbf70f73b19a6b50ffc85dcd9dbd7cfd31bd63202: Status 404 returned error can't find the container with id 5df3c77cbc0451e06e595d6dbf70f73b19a6b50ffc85dcd9dbd7cfd31bd63202 Apr 22 15:26:59.592292 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:26:59.592254 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-cwv8s/test-trainjob-nwlks-node-0-0-s86pr" event={"ID":"6331ad26-270b-42c7-978a-1df2a7a73b4e","Type":"ContainerStarted","Data":"5df3c77cbc0451e06e595d6dbf70f73b19a6b50ffc85dcd9dbd7cfd31bd63202"} Apr 22 15:27:41.485291 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:27:41.485255 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-s79j7_89e3aa95-bc39-4144-bc81-144160cf50eb/console-operator/1.log" Apr 22 15:27:41.487255 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:27:41.487230 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-s79j7_89e3aa95-bc39-4144-bc81-144160cf50eb/console-operator/1.log" Apr 22 15:32:41.510219 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:32:41.510193 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-s79j7_89e3aa95-bc39-4144-bc81-144160cf50eb/console-operator/1.log" Apr 22 15:32:41.514273 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:32:41.514247 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-s79j7_89e3aa95-bc39-4144-bc81-144160cf50eb/console-operator/1.log" Apr 22 15:34:48.211555 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:48.211515 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-cwv8s/test-trainjob-nwlks-node-0-0-s86pr" event={"ID":"6331ad26-270b-42c7-978a-1df2a7a73b4e","Type":"ContainerStarted","Data":"0c1d48fddeed78eb0cf07863837d599b0be904fce04ca43523984c60d9cf3e3b"} Apr 22 15:34:48.213696 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:48.213672 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-cwv8s\"/\"default-dockercfg-qf6sj\"" Apr 22 15:34:48.233534 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:48.233481 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-cwv8s/test-trainjob-nwlks-node-0-0-s86pr" podStartSLOduration=2.126493399 podStartE2EDuration="7m50.233463205s" podCreationTimestamp="2026-04-22 15:26:58 +0000 UTC" firstStartedPulling="2026-04-22 15:26:59.153740561 +0000 UTC m=+1158.210478623" lastFinishedPulling="2026-04-22 15:34:47.260710366 +0000 UTC m=+1626.317448429" observedRunningTime="2026-04-22 15:34:48.23175906 +0000 UTC m=+1627.288497153" watchObservedRunningTime="2026-04-22 15:34:48.233463205 +0000 UTC m=+1627.290201289" Apr 22 15:34:48.264382 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:48.264324 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-cwv8s\"/\"kube-root-ca.crt\"" Apr 22 15:34:48.274934 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:48.274911 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-cwv8s\"/\"openshift-service-ca.crt\"" Apr 22 15:34:51.222119 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:51.222084 2577 generic.go:358] "Generic (PLEG): container finished" podID="6331ad26-270b-42c7-978a-1df2a7a73b4e" containerID="0c1d48fddeed78eb0cf07863837d599b0be904fce04ca43523984c60d9cf3e3b" exitCode=0 Apr 22 15:34:51.222595 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:51.222158 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-cwv8s/test-trainjob-nwlks-node-0-0-s86pr" event={"ID":"6331ad26-270b-42c7-978a-1df2a7a73b4e","Type":"ContainerDied","Data":"0c1d48fddeed78eb0cf07863837d599b0be904fce04ca43523984c60d9cf3e3b"} Apr 22 15:34:52.373605 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:52.373582 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-cwv8s/test-trainjob-nwlks-node-0-0-s86pr" Apr 22 15:34:52.498417 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:52.498387 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvrdk\" (UniqueName: \"kubernetes.io/projected/6331ad26-270b-42c7-978a-1df2a7a73b4e-kube-api-access-nvrdk\") pod \"6331ad26-270b-42c7-978a-1df2a7a73b4e\" (UID: \"6331ad26-270b-42c7-978a-1df2a7a73b4e\") " Apr 22 15:34:52.500594 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:52.500562 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6331ad26-270b-42c7-978a-1df2a7a73b4e-kube-api-access-nvrdk" (OuterVolumeSpecName: "kube-api-access-nvrdk") pod "6331ad26-270b-42c7-978a-1df2a7a73b4e" (UID: "6331ad26-270b-42c7-978a-1df2a7a73b4e"). InnerVolumeSpecName "kube-api-access-nvrdk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 22 15:34:52.599381 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:52.599344 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvrdk\" (UniqueName: \"kubernetes.io/projected/6331ad26-270b-42c7-978a-1df2a7a73b4e-kube-api-access-nvrdk\") on node \"ip-10-0-131-168.ec2.internal\" DevicePath \"\"" Apr 22 15:34:53.231080 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:53.231044 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-cwv8s/test-trainjob-nwlks-node-0-0-s86pr" event={"ID":"6331ad26-270b-42c7-978a-1df2a7a73b4e","Type":"ContainerDied","Data":"5df3c77cbc0451e06e595d6dbf70f73b19a6b50ffc85dcd9dbd7cfd31bd63202"} Apr 22 15:34:53.231080 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:53.231078 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5df3c77cbc0451e06e595d6dbf70f73b19a6b50ffc85dcd9dbd7cfd31bd63202" Apr 22 15:34:53.231080 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:53.231054 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-cwv8s/test-trainjob-nwlks-node-0-0-s86pr" Apr 22 15:34:53.844762 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:53.844720 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-twqv9/test-trainjob-zkhcw-node-0-0-hcc9l"] Apr 22 15:34:53.845240 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:53.845193 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6331ad26-270b-42c7-978a-1df2a7a73b4e" containerName="node" Apr 22 15:34:53.845240 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:53.845210 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6331ad26-270b-42c7-978a-1df2a7a73b4e" containerName="node" Apr 22 15:34:53.845391 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:53.845286 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="6331ad26-270b-42c7-978a-1df2a7a73b4e" containerName="node" Apr 22 15:34:53.870050 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:53.870002 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-twqv9/test-trainjob-zkhcw-node-0-0-hcc9l"] Apr 22 15:34:53.870226 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:53.870131 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-twqv9/test-trainjob-zkhcw-node-0-0-hcc9l" Apr 22 15:34:53.872584 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:53.872555 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-twqv9\"/\"openshift-service-ca.crt\"" Apr 22 15:34:53.872754 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:53.872683 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-twqv9\"/\"kube-root-ca.crt\"" Apr 22 15:34:53.872843 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:53.872822 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-twqv9\"/\"default-dockercfg-vp7g7\"" Apr 22 15:34:54.010418 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:54.010371 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tm45s\" (UniqueName: \"kubernetes.io/projected/02a36763-cf9b-4414-9844-468006143fcd-kube-api-access-tm45s\") pod \"test-trainjob-zkhcw-node-0-0-hcc9l\" (UID: \"02a36763-cf9b-4414-9844-468006143fcd\") " pod="test-ns-twqv9/test-trainjob-zkhcw-node-0-0-hcc9l" Apr 22 15:34:54.111456 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:54.111364 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tm45s\" (UniqueName: \"kubernetes.io/projected/02a36763-cf9b-4414-9844-468006143fcd-kube-api-access-tm45s\") pod \"test-trainjob-zkhcw-node-0-0-hcc9l\" (UID: \"02a36763-cf9b-4414-9844-468006143fcd\") " pod="test-ns-twqv9/test-trainjob-zkhcw-node-0-0-hcc9l" Apr 22 15:34:54.119428 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:54.119405 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tm45s\" (UniqueName: \"kubernetes.io/projected/02a36763-cf9b-4414-9844-468006143fcd-kube-api-access-tm45s\") pod \"test-trainjob-zkhcw-node-0-0-hcc9l\" (UID: \"02a36763-cf9b-4414-9844-468006143fcd\") " pod="test-ns-twqv9/test-trainjob-zkhcw-node-0-0-hcc9l" Apr 22 15:34:54.181764 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:54.181732 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-twqv9/test-trainjob-zkhcw-node-0-0-hcc9l" Apr 22 15:34:54.301861 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:54.301833 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-twqv9/test-trainjob-zkhcw-node-0-0-hcc9l"] Apr 22 15:34:54.303535 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:34:54.303508 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02a36763_cf9b_4414_9844_468006143fcd.slice/crio-7648b0e6e82cfed05a735eaa3ffac04020ea070b9446258b244585a11133d832 WatchSource:0}: Error finding container 7648b0e6e82cfed05a735eaa3ffac04020ea070b9446258b244585a11133d832: Status 404 returned error can't find the container with id 7648b0e6e82cfed05a735eaa3ffac04020ea070b9446258b244585a11133d832 Apr 22 15:34:54.305646 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:54.305626 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 22 15:34:55.240669 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:34:55.240621 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-twqv9/test-trainjob-zkhcw-node-0-0-hcc9l" event={"ID":"02a36763-cf9b-4414-9844-468006143fcd","Type":"ContainerStarted","Data":"7648b0e6e82cfed05a735eaa3ffac04020ea070b9446258b244585a11133d832"} Apr 22 15:37:41.543055 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:37:41.542941 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-s79j7_89e3aa95-bc39-4144-bc81-144160cf50eb/console-operator/1.log" Apr 22 15:37:41.547083 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:37:41.547059 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-s79j7_89e3aa95-bc39-4144-bc81-144160cf50eb/console-operator/1.log" Apr 22 15:41:26.400067 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:41:26.400034 2577 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Apr 22 15:41:26.400631 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:41:26.400087 2577 container_gc.go:86] "Attempting to delete unused containers" Apr 22 15:41:26.401554 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:41:26.401529 2577 scope.go:117] "RemoveContainer" containerID="c32e834600a5036f0fb6a211e5c7f6a0f87dd067379b883b3a2fd4ff33aa8878" Apr 22 15:41:31.125867 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:41:31.125840 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-168.ec2.internal" event="NodeHasDiskPressure" Apr 22 15:43:16.161374 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:43:16.161305 2577 log.go:32] "ListImages with filter from image service failed" err="rpc error: code = DeadlineExceeded desc = context deadline exceeded" filter="nil" Apr 22 15:43:16.171832 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:43:16.161388 2577 kuberuntime_image.go:104] "Failed to list images" err="rpc error: code = DeadlineExceeded desc = context deadline exceeded" Apr 22 15:43:16.171832 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:43:16.161399 2577 image_gc_manager.go:230] "Failed to update image list" err="rpc error: code = DeadlineExceeded desc = context deadline exceeded" Apr 22 15:43:26.402552 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:43:26.402457 2577 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = DeadlineExceeded desc = context deadline exceeded" containerID="c32e834600a5036f0fb6a211e5c7f6a0f87dd067379b883b3a2fd4ff33aa8878" Apr 22 15:43:26.402552 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:43:26.402513 2577 kuberuntime_gc.go:150] "Failed to remove container" err="rpc error: code = DeadlineExceeded desc = context deadline exceeded" containerID="c32e834600a5036f0fb6a211e5c7f6a0f87dd067379b883b3a2fd4ff33aa8878" Apr 22 15:43:26.402552 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:43:26.402536 2577 scope.go:117] "RemoveContainer" containerID="8bb2de2a26d7dab0c0d2f40bcbc1c701402b32b2212a983e5020ab36fd3d29f1" Apr 22 15:44:41.552310 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:44:41.552272 2577 log.go:32] "ListImages with filter from image service failed" err="rpc error: code = DeadlineExceeded desc = context deadline exceeded" filter="nil" Apr 22 15:44:41.552310 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:44:41.552310 2577 kuberuntime_image.go:104] "Failed to list images" err="rpc error: code = DeadlineExceeded desc = context deadline exceeded" Apr 22 15:44:41.552799 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:44:41.552320 2577 image_gc_manager.go:222] "Failed to monitor images" err="rpc error: code = DeadlineExceeded desc = context deadline exceeded" Apr 22 15:44:41.558480 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:44:41.558459 2577 log.go:32] "ListImages with filter from image service failed" err="rpc error: code = DeadlineExceeded desc = context deadline exceeded" filter="nil" Apr 22 15:44:41.558530 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:44:41.558485 2577 kuberuntime_image.go:104] "Failed to list images" err="rpc error: code = DeadlineExceeded desc = context deadline exceeded" Apr 22 15:44:41.558530 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:44:41.558495 2577 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="rpc error: code = DeadlineExceeded desc = context deadline exceeded" Apr 22 15:45:26.403640 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:45:26.403588 2577 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = DeadlineExceeded desc = stream terminated by RST_STREAM with error code: CANCEL" containerID="8bb2de2a26d7dab0c0d2f40bcbc1c701402b32b2212a983e5020ab36fd3d29f1" Apr 22 15:45:26.403640 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:45:26.403647 2577 kuberuntime_gc.go:150] "Failed to remove container" err="rpc error: code = DeadlineExceeded desc = stream terminated by RST_STREAM with error code: CANCEL" containerID="8bb2de2a26d7dab0c0d2f40bcbc1c701402b32b2212a983e5020ab36fd3d29f1" Apr 22 15:45:26.404156 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:45:26.403670 2577 scope.go:117] "RemoveContainer" containerID="0c1d48fddeed78eb0cf07863837d599b0be904fce04ca43523984c60d9cf3e3b" Apr 22 15:45:29.984359 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:45:29.984151 2577 scope.go:117] "RemoveContainer" containerID="6e9a9ae56b885847dfe74839afdab2dfe1251119577f2e6aaaced27fc9559a9f" Apr 22 15:45:30.009479 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:45:30.009451 2577 scope.go:117] "RemoveContainer" containerID="551bc383c504e780bc9aa919cf987de535d8d5eae5f9a7c13318063d87ce3b4e" Apr 22 15:45:30.018198 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:45:30.018168 2577 scope.go:117] "RemoveContainer" containerID="695fa4fa89bdc0b476ba4f0d34c77633e9d142192f5ed4ba7241977a25405147" Apr 22 15:45:30.025847 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:45:30.025824 2577 scope.go:117] "RemoveContainer" containerID="298727b0cb0b730affe0cfcc4bb3902c72a43cecf7e6eda1cec506449df08ae9" Apr 22 15:45:30.042595 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:45:30.042572 2577 scope.go:117] "RemoveContainer" containerID="5d98a4508bf9261455970427b97b9d6ec0af7fd8a7b727a19b252037c7ab2490" Apr 22 15:45:30.050320 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:45:30.050297 2577 scope.go:117] "RemoveContainer" containerID="9b4efa352432219e37658b1307674f4006999726ecfa247787c36ddf5297fc6f" Apr 22 15:45:30.072219 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:45:30.072196 2577 scope.go:117] "RemoveContainer" containerID="bf9fb35d676766813e3fcafa4784e0be24395c5d2c5aed7c756fec6c2192eef8" Apr 22 15:45:30.107428 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:45:30.107405 2577 image_gc_manager.go:447] "Attempting to delete unused images" Apr 22 15:45:30.119203 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:45:30.119183 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-s79j7_89e3aa95-bc39-4144-bc81-144160cf50eb/console-operator/1.log" Apr 22 15:45:30.126055 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:45:30.126031 2577 image_gc_manager.go:514] "Removing image to free bytes" imageID="ac4be6c7a52584c773ae754a4ccfb9fb1db440f4c9d858ad0f78765a85625b4b" size=1065006420 runtimeHandler="" Apr 22 15:45:30.644959 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:45:30.644926 2577 image_gc_manager.go:514] "Removing image to free bytes" imageID="5151a6030289f6d1ef2c984ebd3e465632a3bf64de79db6f7b3d6e2e638b0557" size=1065600018 runtimeHandler="" Apr 22 15:45:30.662339 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:45:30.662282 2577 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: copying system image from manifest list: writing blob: adding layer with blob \"sha256:56a6a09b03e81131ca690210efc814701da032525ada07591ffe5d6c5d5a4906\"/\"\"/\"sha256:0832a7269a80e5ed5e1c5c749a8d30a6b248ce9b970d81f61a4f93a3b72673f9\": unpacking failed (error: exit status 1; output: write /opt/rocm-6.4.3/lib/librocprof-sys.so.1.0.2: no space left on device); artifact err: provided artifact is a container image" image="quay.io/opendatahub/odh-training-rocm64-torch29-py312@sha256:8a053c8ee3a4c326b745b2516a291c6b8a6e92defc5406ac2e9590bb742153f6" Apr 22 15:45:30.662532 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:45:30.662488 2577 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:node,Image:quay.io/opendatahub/odh-training-rocm64-torch29-py312@sha256:8a053c8ee3a4c326b745b2516a291c6b8a6e92defc5406ac2e9590bb742153f6,Command:[python -c import torch; print(f'PyTorch version: {torch.__version__}'); print('Training completed successfully')],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:,HostPort:0,ContainerPort:29500,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:PET_NNODES,Value:1,ValueFrom:nil,},EnvVar{Name:PET_NPROC_PER_NODE,Value:1,ValueFrom:nil,},EnvVar{Name:PET_NODE_RANK,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.annotations['batch.kubernetes.io/job-completion-index'],},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:PET_MASTER_ADDR,Value:test-trainjob-zkhcw-node-0-0.test-trainjob-zkhcw,ValueFrom:nil,},EnvVar{Name:PET_MASTER_PORT,Value:29500,ValueFrom:nil,},EnvVar{Name:JOB_COMPLETION_INDEX,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.labels['batch.kubernetes.io/job-completion-index'],},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kube-api-access-tm45s,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000710000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod test-trainjob-zkhcw-node-0-0-hcc9l_test-ns-twqv9(02a36763-cf9b-4414-9844-468006143fcd): ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: writing blob: adding layer with blob \"sha256:56a6a09b03e81131ca690210efc814701da032525ada07591ffe5d6c5d5a4906\"/\"\"/\"sha256:0832a7269a80e5ed5e1c5c749a8d30a6b248ce9b970d81f61a4f93a3b72673f9\": unpacking failed (error: exit status 1; output: write /opt/rocm-6.4.3/lib/librocprof-sys.so.1.0.2: no space left on device); artifact err: provided artifact is a container image" logger="UnhandledError" Apr 22 15:45:30.663687 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:45:30.663648 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"node\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: copying system image from manifest list: writing blob: adding layer with blob \\\"sha256:56a6a09b03e81131ca690210efc814701da032525ada07591ffe5d6c5d5a4906\\\"/\\\"\\\"/\\\"sha256:0832a7269a80e5ed5e1c5c749a8d30a6b248ce9b970d81f61a4f93a3b72673f9\\\": unpacking failed (error: exit status 1; output: write /opt/rocm-6.4.3/lib/librocprof-sys.so.1.0.2: no space left on device); artifact err: provided artifact is a container image\"" pod="test-ns-twqv9/test-trainjob-zkhcw-node-0-0-hcc9l" podUID="02a36763-cf9b-4414-9844-468006143fcd" Apr 22 15:45:30.701004 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:45:30.700976 2577 image_gc_manager.go:514] "Removing image to free bytes" imageID="ba0d5ab4eb24f99d84ae4923fefa85e3ab5042c1e554dcca3a41789529499633" size=107183730 runtimeHandler="" Apr 22 15:45:30.712537 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:45:30.712507 2577 image_gc_manager.go:514] "Removing image to free bytes" imageID="f9c8cb14b315efbe3847333b6de717d1a52318bb05b38cce743926641075fbb5" size=884076775 runtimeHandler="" Apr 22 15:45:30.735960 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:45:30.735928 2577 image_gc_manager.go:514] "Removing image to free bytes" imageID="df7311fe93e730dc6d3d65a73c992b1583cc3d49b2e20975439f4718eb9ac4f5" size=108503547 runtimeHandler="" Apr 22 15:45:30.748296 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:45:30.748238 2577 image_gc_manager.go:514] "Removing image to free bytes" imageID="ec845ac5d8f1d4c74cbd447a93360fa7b8b615723fab3a377882708da6009878" size=977364430 runtimeHandler="" Apr 22 15:45:30.771709 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:45:30.771680 2577 image_gc_manager.go:514] "Removing image to free bytes" imageID="bd2f0c6a473dfa650b536cfe1992446bf45305b3ace698398143f161694113a5" size=20806872103 runtimeHandler="" Apr 22 15:45:31.459784 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:45:31.459756 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-twqv9\"/\"default-dockercfg-vp7g7\"" Apr 22 15:45:31.587033 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:45:31.587001 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-twqv9\"/\"kube-root-ca.crt\"" Apr 22 15:45:31.598042 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:45:31.598015 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-twqv9\"/\"openshift-service-ca.crt\"" Apr 22 15:45:34.596970 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:45:34.596536 2577 image_gc_manager.go:514] "Removing image to free bytes" imageID="038c73cf9d35e89709b4f826c0ceb8dc783a3aa366d3139240c1a1da0ec1e546" size=7588072914 runtimeHandler="" Apr 22 15:45:34.596970 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:45:34.596819 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 22 15:45:34.597209 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:45:34.597061 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"node\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/opendatahub/odh-training-rocm64-torch29-py312@sha256:8a053c8ee3a4c326b745b2516a291c6b8a6e92defc5406ac2e9590bb742153f6\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: writing blob: adding layer with blob \\\"sha256:56a6a09b03e81131ca690210efc814701da032525ada07591ffe5d6c5d5a4906\\\"/\\\"\\\"/\\\"sha256:0832a7269a80e5ed5e1c5c749a8d30a6b248ce9b970d81f61a4f93a3b72673f9\\\": unpacking failed (error: exit status 1; output: write /opt/rocm-6.4.3/lib/librocprof-sys.so.1.0.2: no space left on device); artifact err: provided artifact is a container image\"" pod="test-ns-twqv9/test-trainjob-zkhcw-node-0-0-hcc9l" podUID="02a36763-cf9b-4414-9844-468006143fcd" Apr 22 15:45:37.677104 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:45:37.677066 2577 image_gc_manager.go:514] "Removing image to free bytes" imageID="7e65b8288e37c3f4fac04e8bf51240765caae34795b317d44d5399762a08b761" size=23201654702 runtimeHandler="" Apr 22 15:45:41.771004 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:45:41.770971 2577 image_gc_manager.go:514] "Removing image to free bytes" imageID="819e15fdec92d846e6d5de4b1b2988adcb74f6d3046689fe03c655b03a67975d" size=18873458221 runtimeHandler="" Apr 22 15:45:44.583322 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:45:44.583295 2577 eviction_manager.go:383] "Eviction manager: able to reduce resource pressure without evicting pods." resourceName="ephemeral-storage" Apr 22 15:46:38.059254 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:46:38.059221 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-168.ec2.internal" event="NodeHasNoDiskPressure" Apr 22 15:49:41.565579 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:49:41.565553 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-s79j7_89e3aa95-bc39-4144-bc81-144160cf50eb/console-operator/1.log" Apr 22 15:49:41.570161 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:49:41.570137 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-s79j7_89e3aa95-bc39-4144-bc81-144160cf50eb/console-operator/1.log" Apr 22 15:49:41.578073 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:49:41.578055 2577 kubelet.go:1628] "Image garbage collection succeeded" Apr 22 15:54:00.053421 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:54:00.053310 2577 log.go:32] "ListImages with filter from image service failed" err="rpc error: code = DeadlineExceeded desc = context deadline exceeded" filter="nil" Apr 22 15:54:00.053421 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:54:00.053373 2577 kuberuntime_image.go:104] "Failed to list images" err="rpc error: code = DeadlineExceeded desc = context deadline exceeded" Apr 22 15:54:00.053421 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:54:00.053385 2577 image_gc_manager.go:230] "Failed to update image list" err="rpc error: code = DeadlineExceeded desc = context deadline exceeded" Apr 22 15:54:53.828909 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:54:53.828866 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c1d48fddeed78eb0cf07863837d599b0be904fce04ca43523984c60d9cf3e3b\": container with ID starting with 0c1d48fddeed78eb0cf07863837d599b0be904fce04ca43523984c60d9cf3e3b not found: ID does not exist" containerID="0c1d48fddeed78eb0cf07863837d599b0be904fce04ca43523984c60d9cf3e3b" Apr 22 15:54:53.926161 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:54:53.926125 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e9a9ae56b885847dfe74839afdab2dfe1251119577f2e6aaaced27fc9559a9f\": container with ID starting with 6e9a9ae56b885847dfe74839afdab2dfe1251119577f2e6aaaced27fc9559a9f not found: ID does not exist" containerID="6e9a9ae56b885847dfe74839afdab2dfe1251119577f2e6aaaced27fc9559a9f" Apr 22 15:54:54.027080 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:54:54.027046 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"298727b0cb0b730affe0cfcc4bb3902c72a43cecf7e6eda1cec506449df08ae9\": container with ID starting with 298727b0cb0b730affe0cfcc4bb3902c72a43cecf7e6eda1cec506449df08ae9 not found: ID does not exist" containerID="298727b0cb0b730affe0cfcc4bb3902c72a43cecf7e6eda1cec506449df08ae9" Apr 22 15:54:54.521587 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:54:54.521549 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b4efa352432219e37658b1307674f4006999726ecfa247787c36ddf5297fc6f\": container with ID starting with 9b4efa352432219e37658b1307674f4006999726ecfa247787c36ddf5297fc6f not found: ID does not exist" containerID="9b4efa352432219e37658b1307674f4006999726ecfa247787c36ddf5297fc6f" Apr 22 15:54:58.768269 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:54:58.768233 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-twqv9/test-trainjob-zkhcw-node-0-0-hcc9l"] Apr 22 15:54:58.868591 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:54:58.868555 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-cwv8s/test-trainjob-nwlks-node-0-0-s86pr"] Apr 22 15:54:58.871346 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:54:58.871307 2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-cwv8s/test-trainjob-nwlks-node-0-0-s86pr"] Apr 22 15:54:58.968932 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:54:58.968896 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-4ltzp/test-trainjob-jqpl7-node-0-0-nv7mg"] Apr 22 15:54:58.970481 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:54:58.970455 2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-4ltzp/test-trainjob-jqpl7-node-0-0-nv7mg"] Apr 22 15:54:59.241583 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:54:59.241547 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-p4l27/test-trainjob-bn2mq-node-0-0-gck6z"] Apr 22 15:54:59.245530 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:54:59.245504 2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-p4l27/test-trainjob-bn2mq-node-0-0-gck6z"] Apr 22 15:54:59.507000 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:54:59.506919 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6331ad26-270b-42c7-978a-1df2a7a73b4e" path="/var/lib/kubelet/pods/6331ad26-270b-42c7-978a-1df2a7a73b4e/volumes" Apr 22 15:54:59.507260 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:54:59.507248 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c911947-e5d7-48eb-b435-89efde55eb66" path="/var/lib/kubelet/pods/9c911947-e5d7-48eb-b435-89efde55eb66/volumes" Apr 22 15:54:59.507575 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:54:59.507563 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4f80d2f-e21c-4696-9579-3c6bc9148b03" path="/var/lib/kubelet/pods/d4f80d2f-e21c-4696-9579-3c6bc9148b03/volumes" Apr 22 15:54:59.790151 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:54:59.790071 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-gskjb/test-trainjob-pb95z-node-0-0-m6hwq"] Apr 22 15:54:59.793715 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:54:59.793692 2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-gskjb/test-trainjob-pb95z-node-0-0-m6hwq"] Apr 22 15:55:01.507887 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:55:01.507849 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a37ff90-cfd8-4791-9a43-9069c09b98d6" path="/var/lib/kubelet/pods/0a37ff90-cfd8-4791-9a43-9069c09b98d6/volumes" Apr 22 15:55:47.429796 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:55:47.429757 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xjqzk/must-gather-fm5d2"] Apr 22 15:55:47.444679 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:55:47.444640 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xjqzk/must-gather-fm5d2"] Apr 22 15:55:47.444860 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:55:47.444759 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xjqzk/must-gather-fm5d2" Apr 22 15:55:47.447363 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:55:47.447313 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xjqzk\"/\"openshift-service-ca.crt\"" Apr 22 15:55:47.447518 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:55:47.447386 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xjqzk\"/\"default-dockercfg-ct867\"" Apr 22 15:55:47.448284 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:55:47.448241 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xjqzk\"/\"kube-root-ca.crt\"" Apr 22 15:55:47.571520 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:55:47.571481 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a1cc4a89-91d6-46a0-b35b-51c29d6751d9-must-gather-output\") pod \"must-gather-fm5d2\" (UID: \"a1cc4a89-91d6-46a0-b35b-51c29d6751d9\") " pod="openshift-must-gather-xjqzk/must-gather-fm5d2" Apr 22 15:55:47.571686 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:55:47.571558 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg8d9\" (UniqueName: \"kubernetes.io/projected/a1cc4a89-91d6-46a0-b35b-51c29d6751d9-kube-api-access-xg8d9\") pod \"must-gather-fm5d2\" (UID: \"a1cc4a89-91d6-46a0-b35b-51c29d6751d9\") " pod="openshift-must-gather-xjqzk/must-gather-fm5d2" Apr 22 15:55:47.672800 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:55:47.672763 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xg8d9\" (UniqueName: \"kubernetes.io/projected/a1cc4a89-91d6-46a0-b35b-51c29d6751d9-kube-api-access-xg8d9\") pod \"must-gather-fm5d2\" (UID: \"a1cc4a89-91d6-46a0-b35b-51c29d6751d9\") " pod="openshift-must-gather-xjqzk/must-gather-fm5d2" Apr 22 15:55:47.672999 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:55:47.672829 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a1cc4a89-91d6-46a0-b35b-51c29d6751d9-must-gather-output\") pod \"must-gather-fm5d2\" (UID: \"a1cc4a89-91d6-46a0-b35b-51c29d6751d9\") " pod="openshift-must-gather-xjqzk/must-gather-fm5d2" Apr 22 15:55:47.673254 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:55:47.673220 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a1cc4a89-91d6-46a0-b35b-51c29d6751d9-must-gather-output\") pod \"must-gather-fm5d2\" (UID: \"a1cc4a89-91d6-46a0-b35b-51c29d6751d9\") " pod="openshift-must-gather-xjqzk/must-gather-fm5d2" Apr 22 15:55:47.680478 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:55:47.680411 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg8d9\" (UniqueName: \"kubernetes.io/projected/a1cc4a89-91d6-46a0-b35b-51c29d6751d9-kube-api-access-xg8d9\") pod \"must-gather-fm5d2\" (UID: \"a1cc4a89-91d6-46a0-b35b-51c29d6751d9\") " pod="openshift-must-gather-xjqzk/must-gather-fm5d2" Apr 22 15:55:47.757232 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:55:47.757190 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xjqzk/must-gather-fm5d2" Apr 22 15:56:02.200688 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:02.200657 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-s79j7_89e3aa95-bc39-4144-bc81-144160cf50eb/console-operator/1.log" Apr 22 15:56:02.201735 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:02.201623 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-s79j7_89e3aa95-bc39-4144-bc81-144160cf50eb/console-operator/1.log" Apr 22 15:56:02.311522 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:02.311447 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xjqzk/must-gather-fm5d2"] Apr 22 15:56:02.313897 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:56:02.313873 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1cc4a89_91d6_46a0_b35b_51c29d6751d9.slice/crio-2f5ca1ba147f253ccc5e6a96b72c66d38836acf5ed87b19aebf85c476bf19c65 WatchSource:0}: Error finding container 2f5ca1ba147f253ccc5e6a96b72c66d38836acf5ed87b19aebf85c476bf19c65: Status 404 returned error can't find the container with id 2f5ca1ba147f253ccc5e6a96b72c66d38836acf5ed87b19aebf85c476bf19c65 Apr 22 15:56:02.315963 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:02.315943 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 22 15:56:02.521360 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:02.521284 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xjqzk/must-gather-fm5d2" event={"ID":"a1cc4a89-91d6-46a0-b35b-51c29d6751d9","Type":"ContainerStarted","Data":"2f5ca1ba147f253ccc5e6a96b72c66d38836acf5ed87b19aebf85c476bf19c65"} Apr 22 15:56:04.530114 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:04.530077 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-twqv9/test-trainjob-zkhcw-node-0-0-hcc9l" event={"ID":"02a36763-cf9b-4414-9844-468006143fcd","Type":"ContainerStarted","Data":"6161ea7c77169ccbe0e68b989823d15da46bcb0445989bd75de57d66c56d4859"} Apr 22 15:56:04.530595 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:04.530090 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-twqv9/test-trainjob-zkhcw-node-0-0-hcc9l" podUID="02a36763-cf9b-4414-9844-468006143fcd" containerName="node" containerID="cri-o://6161ea7c77169ccbe0e68b989823d15da46bcb0445989bd75de57d66c56d4859" gracePeriod=30 Apr 22 15:56:04.531845 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:04.531819 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xjqzk/must-gather-fm5d2" event={"ID":"a1cc4a89-91d6-46a0-b35b-51c29d6751d9","Type":"ContainerStarted","Data":"290ee48971c02df314e44f3cf133e086cd9a0788c6dfb938f9c664e1d82c9e77"} Apr 22 15:56:04.544158 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:04.544115 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-twqv9/test-trainjob-zkhcw-node-0-0-hcc9l" podStartSLOduration=2.301324568 podStartE2EDuration="21m11.544104063s" podCreationTimestamp="2026-04-22 15:34:53 +0000 UTC" firstStartedPulling="2026-04-22 15:34:54.305799134 +0000 UTC m=+1633.362537196" lastFinishedPulling="2026-04-22 15:56:03.548578628 +0000 UTC m=+2902.605316691" observedRunningTime="2026-04-22 15:56:04.54289679 +0000 UTC m=+2903.599634887" watchObservedRunningTime="2026-04-22 15:56:04.544104063 +0000 UTC m=+2903.600842167" Apr 22 15:56:05.536222 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:05.536187 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xjqzk/must-gather-fm5d2" event={"ID":"a1cc4a89-91d6-46a0-b35b-51c29d6751d9","Type":"ContainerStarted","Data":"4adeaa56677d3e8087d90e9bca056b2bc6f5c7d92d33d286b4a9ad65696b3f96"} Apr 22 15:56:05.551310 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:05.551242 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-xjqzk/must-gather-fm5d2" podStartSLOduration=16.783359313 podStartE2EDuration="18.551222593s" podCreationTimestamp="2026-04-22 15:55:47 +0000 UTC" firstStartedPulling="2026-04-22 15:56:02.31613388 +0000 UTC m=+2901.372871944" lastFinishedPulling="2026-04-22 15:56:04.083996954 +0000 UTC m=+2903.140735224" observedRunningTime="2026-04-22 15:56:05.549482506 +0000 UTC m=+2904.606220591" watchObservedRunningTime="2026-04-22 15:56:05.551222593 +0000 UTC m=+2904.607960679" Apr 22 15:56:06.702920 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:06.702888 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8l6g2_fb9ab245-62d7-4d19-93cd-34d5f9595d3d/global-pull-secret-syncer/0.log" Apr 22 15:56:06.830971 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:06.830936 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mzm96_ab8e8378-2063-4f8e-bb2a-cc3cd4ca56d1/konnectivity-agent/0.log" Apr 22 15:56:06.905915 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:06.905884 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-168.ec2.internal_5e64b9b4046044f766b93ef1fd3f3a00/haproxy/0.log" Apr 22 15:56:10.490952 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:10.490923 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-75587bd455-29bzc_a84180fa-5ffa-4cc6-80cc-0cd55558a345/cluster-monitoring-operator/0.log" Apr 22 15:56:10.795180 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:10.795097 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zlb8j_29275b0f-a5ac-4a8f-8ac0-bb243b489174/node-exporter/0.log" Apr 22 15:56:10.819977 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:10.819950 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zlb8j_29275b0f-a5ac-4a8f-8ac0-bb243b489174/kube-rbac-proxy/0.log" Apr 22 15:56:10.846375 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:10.846347 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zlb8j_29275b0f-a5ac-4a8f-8ac0-bb243b489174/init-textfile/0.log" Apr 22 15:56:11.132639 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:11.132566 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-57cf98b594-s9dtm_6f799b17-a43e-45ef-8c84-0e7851c9905f/prometheus-operator-admission-webhook/0.log" Apr 22 15:56:12.398467 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:12.398438 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-cb95c66f6-6ljs8_acbfa8e1-30e3-467a-b70e-c9e23ebddcbe/networking-console-plugin/0.log" Apr 22 15:56:12.937207 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:12.937180 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-s79j7_89e3aa95-bc39-4144-bc81-144160cf50eb/console-operator/1.log" Apr 22 15:56:12.943514 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:12.943490 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-s79j7_89e3aa95-bc39-4144-bc81-144160cf50eb/console-operator/2.log" Apr 22 15:56:13.330397 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:13.330371 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6bcc868b7-4jlxk_4836af87-a000-45a5-845e-cc0488640ef0/download-server/0.log" Apr 22 15:56:13.697101 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:13.696994 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-7c6cbb6c87-lrs9z_4ccb076c-b0c3-40bd-b647-ef47ba1d8681/volume-data-source-validator/0.log" Apr 22 15:56:14.293076 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:14.293046 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4zrz9_dc334588-fb2e-4ac6-bb34-460dad264804/dns/0.log" Apr 22 15:56:14.312785 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:14.312758 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4zrz9_dc334588-fb2e-4ac6-bb34-460dad264804/kube-rbac-proxy/0.log" Apr 22 15:56:14.463647 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:14.463622 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xlxs5_74c26902-8853-4def-9827-b429513b3de8/dns-node-resolver/0.log" Apr 22 15:56:14.905892 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:14.905858 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g8wmb_9dc3cd64-355a-4b4b-a5e1-5d165978d8a3/node-ca/0.log" Apr 22 15:56:15.106799 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:15.106755 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xjqzk/perf-node-gather-daemonset-v5k7j"] Apr 22 15:56:15.232021 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:15.231983 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xjqzk/perf-node-gather-daemonset-v5k7j"] Apr 22 15:56:15.232185 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:15.232166 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xjqzk/perf-node-gather-daemonset-v5k7j" Apr 22 15:56:15.292154 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:15.292112 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1-podres\") pod \"perf-node-gather-daemonset-v5k7j\" (UID: \"ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1\") " pod="openshift-must-gather-xjqzk/perf-node-gather-daemonset-v5k7j" Apr 22 15:56:15.292468 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:15.292422 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1-proc\") pod \"perf-node-gather-daemonset-v5k7j\" (UID: \"ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1\") " pod="openshift-must-gather-xjqzk/perf-node-gather-daemonset-v5k7j" Apr 22 15:56:15.292620 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:15.292600 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1-sys\") pod \"perf-node-gather-daemonset-v5k7j\" (UID: \"ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1\") " pod="openshift-must-gather-xjqzk/perf-node-gather-daemonset-v5k7j" Apr 22 15:56:15.292767 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:15.292751 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh2d2\" (UniqueName: \"kubernetes.io/projected/ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1-kube-api-access-lh2d2\") pod \"perf-node-gather-daemonset-v5k7j\" (UID: \"ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1\") " pod="openshift-must-gather-xjqzk/perf-node-gather-daemonset-v5k7j" Apr 22 15:56:15.292961 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:15.292943 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1-lib-modules\") pod \"perf-node-gather-daemonset-v5k7j\" (UID: \"ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1\") " pod="openshift-must-gather-xjqzk/perf-node-gather-daemonset-v5k7j" Apr 22 15:56:15.394399 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:15.394366 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1-lib-modules\") pod \"perf-node-gather-daemonset-v5k7j\" (UID: \"ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1\") " pod="openshift-must-gather-xjqzk/perf-node-gather-daemonset-v5k7j" Apr 22 15:56:15.394587 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:15.394428 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1-podres\") pod \"perf-node-gather-daemonset-v5k7j\" (UID: \"ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1\") " pod="openshift-must-gather-xjqzk/perf-node-gather-daemonset-v5k7j" Apr 22 15:56:15.394587 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:15.394455 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1-proc\") pod \"perf-node-gather-daemonset-v5k7j\" (UID: \"ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1\") " pod="openshift-must-gather-xjqzk/perf-node-gather-daemonset-v5k7j" Apr 22 15:56:15.394587 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:15.394471 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1-sys\") pod \"perf-node-gather-daemonset-v5k7j\" (UID: \"ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1\") " pod="openshift-must-gather-xjqzk/perf-node-gather-daemonset-v5k7j" Apr 22 15:56:15.394587 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:15.394491 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lh2d2\" (UniqueName: \"kubernetes.io/projected/ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1-kube-api-access-lh2d2\") pod \"perf-node-gather-daemonset-v5k7j\" (UID: \"ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1\") " pod="openshift-must-gather-xjqzk/perf-node-gather-daemonset-v5k7j" Apr 22 15:56:15.394587 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:15.394568 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1-proc\") pod \"perf-node-gather-daemonset-v5k7j\" (UID: \"ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1\") " pod="openshift-must-gather-xjqzk/perf-node-gather-daemonset-v5k7j" Apr 22 15:56:15.394799 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:15.394566 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1-lib-modules\") pod \"perf-node-gather-daemonset-v5k7j\" (UID: \"ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1\") " pod="openshift-must-gather-xjqzk/perf-node-gather-daemonset-v5k7j" Apr 22 15:56:15.394799 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:15.394572 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1-sys\") pod \"perf-node-gather-daemonset-v5k7j\" (UID: \"ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1\") " pod="openshift-must-gather-xjqzk/perf-node-gather-daemonset-v5k7j" Apr 22 15:56:15.394799 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:15.394651 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1-podres\") pod \"perf-node-gather-daemonset-v5k7j\" (UID: \"ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1\") " pod="openshift-must-gather-xjqzk/perf-node-gather-daemonset-v5k7j" Apr 22 15:56:15.401978 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:15.401827 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh2d2\" (UniqueName: \"kubernetes.io/projected/ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1-kube-api-access-lh2d2\") pod \"perf-node-gather-daemonset-v5k7j\" (UID: \"ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1\") " pod="openshift-must-gather-xjqzk/perf-node-gather-daemonset-v5k7j" Apr 22 15:56:15.544385 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:15.544297 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xjqzk/perf-node-gather-daemonset-v5k7j" Apr 22 15:56:15.599158 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:15.599085 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-594fffb56b-msk5s_19d55ae0-1958-4a44-a653-e45e72f6e6e2/router/0.log" Apr 22 15:56:15.692768 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:15.692738 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xjqzk/perf-node-gather-daemonset-v5k7j"] Apr 22 15:56:15.695531 ip-10-0-131-168 kubenswrapper[2577]: W0422 15:56:15.695492 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podae5aaf27_dda5_47b1_81d3_a2ee3a5915c1.slice/crio-9b588203385667884f597a0eeeb77ea7bd69a8c8b79c2236035c901cac8db2f7 WatchSource:0}: Error finding container 9b588203385667884f597a0eeeb77ea7bd69a8c8b79c2236035c901cac8db2f7: Status 404 returned error can't find the container with id 9b588203385667884f597a0eeeb77ea7bd69a8c8b79c2236035c901cac8db2f7 Apr 22 15:56:15.923141 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:15.923113 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4qmd5_eb3d139a-27c4-4fa7-8809-a33c13551872/serve-healthcheck-canary/0.log" Apr 22 15:56:16.290826 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:16.290791 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-585dfdc468-rxj2p_c7b8b7c3-8b82-46c5-bb74-0c90f6448aff/insights-operator/1.log" Apr 22 15:56:16.308868 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:16.308828 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-585dfdc468-rxj2p_c7b8b7c3-8b82-46c5-bb74-0c90f6448aff/insights-operator/0.log" Apr 22 15:56:16.389176 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:16.389147 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w5wkb_d32dc35b-d560-45ad-90b9-f982fed7d7df/kube-rbac-proxy/0.log" Apr 22 15:56:16.409732 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:16.409710 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w5wkb_d32dc35b-d560-45ad-90b9-f982fed7d7df/exporter/0.log" Apr 22 15:56:16.430467 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:16.430443 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w5wkb_d32dc35b-d560-45ad-90b9-f982fed7d7df/extractor/0.log" Apr 22 15:56:16.582564 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:16.582495 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xjqzk/perf-node-gather-daemonset-v5k7j" event={"ID":"ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1","Type":"ContainerStarted","Data":"812e95c87e1e082bed3f8f6990e5175e966b39faa4ebde089d2169360186f564"} Apr 22 15:56:16.582564 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:16.582528 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xjqzk/perf-node-gather-daemonset-v5k7j" event={"ID":"ae5aaf27-dda5-47b1-81d3-a2ee3a5915c1","Type":"ContainerStarted","Data":"9b588203385667884f597a0eeeb77ea7bd69a8c8b79c2236035c901cac8db2f7"} Apr 22 15:56:16.582746 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:16.582627 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xjqzk/perf-node-gather-daemonset-v5k7j" Apr 22 15:56:16.596755 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:16.596717 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-xjqzk/perf-node-gather-daemonset-v5k7j" podStartSLOduration=1.596704218 podStartE2EDuration="1.596704218s" podCreationTimestamp="2026-04-22 15:56:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-22 15:56:16.594727978 +0000 UTC m=+2915.651466062" watchObservedRunningTime="2026-04-22 15:56:16.596704218 +0000 UTC m=+2915.653442336" Apr 22 15:56:18.131151 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:18.131121 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-jobset-operator_jobset-operator-747c5859c7-dsv7s_818a8fea-af99-4edb-bc06-53b06aef5fe7/jobset-operator/0.log" Apr 22 15:56:21.114970 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:21.114942 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-74bb7799d9-c866s_5cd9c342-f4f7-49d3-be86-3617d485feb5/migrator/0.log" Apr 22 15:56:21.133471 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:21.133447 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-74bb7799d9-c866s_5cd9c342-f4f7-49d3-be86-3617d485feb5/graceful-termination/0.log" Apr 22 15:56:21.398004 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:21.397913 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6769c5d45-fxjmk_6cd30837-92d3-44c3-ba6a-9c84a22a7c8b/kube-storage-version-migrator-operator/1.log" Apr 22 15:56:21.401967 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:21.401942 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6769c5d45-fxjmk_6cd30837-92d3-44c3-ba6a-9c84a22a7c8b/kube-storage-version-migrator-operator/0.log" Apr 22 15:56:22.214672 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:22.214638 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2jf6l_6b9559b7-6b5b-4ecf-9655-4379447b15c7/kube-multus-additional-cni-plugins/0.log" Apr 22 15:56:22.240140 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:22.240105 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2jf6l_6b9559b7-6b5b-4ecf-9655-4379447b15c7/egress-router-binary-copy/0.log" Apr 22 15:56:22.279942 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:22.279908 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2jf6l_6b9559b7-6b5b-4ecf-9655-4379447b15c7/cni-plugins/0.log" Apr 22 15:56:22.311903 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:22.311867 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2jf6l_6b9559b7-6b5b-4ecf-9655-4379447b15c7/bond-cni-plugin/0.log" Apr 22 15:56:22.341615 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:22.341587 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2jf6l_6b9559b7-6b5b-4ecf-9655-4379447b15c7/routeoverride-cni/0.log" Apr 22 15:56:22.397153 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:22.397116 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2jf6l_6b9559b7-6b5b-4ecf-9655-4379447b15c7/whereabouts-cni-bincopy/0.log" Apr 22 15:56:22.428446 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:22.428413 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2jf6l_6b9559b7-6b5b-4ecf-9655-4379447b15c7/whereabouts-cni/0.log" Apr 22 15:56:22.595746 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:22.595698 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xjqzk/perf-node-gather-daemonset-v5k7j" Apr 22 15:56:22.764627 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:22.764599 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-h5xww_0c332b2a-7ac7-4b26-b9d4-272932e01b8e/kube-multus/0.log" Apr 22 15:56:22.884871 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:22.884800 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-z9ckr_487f23b6-9470-41e9-abbd-295b6d317b10/network-metrics-daemon/0.log" Apr 22 15:56:22.903179 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:22.903150 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-z9ckr_487f23b6-9470-41e9-abbd-295b6d317b10/kube-rbac-proxy/0.log" Apr 22 15:56:24.310068 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:24.310026 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6qsb_bdcb6dd7-b076-455a-8529-a6c7a1eeae89/ovn-controller/0.log" Apr 22 15:56:24.343453 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:56:24.343406 2577 log.go:32] "Failed when parsing line in log file" err="unexpected timestamp format \"2006-01-02T15:04:05.999999999Z07:00\": parsing time \"2026-04-22T15:42026-04-22T15:45:31.677358487+00:00\" as \"2006-01-02T15:04:05.999999999Z07:00\": cannot parse \"026-04-22T15:45:31.677358487+00:00\" as \":\"" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6qsb_bdcb6dd7-b076-455a-8529-a6c7a1eeae89/ovn-acl-logging/0.log" line="2026-04-22T15:42026-04-22T15:45:31.677358487+00:00 stderr F + true\n" Apr 22 15:56:24.348470 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:24.348436 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6qsb_bdcb6dd7-b076-455a-8529-a6c7a1eeae89/ovn-acl-logging/0.log" Apr 22 15:56:24.368008 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:24.367970 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6qsb_bdcb6dd7-b076-455a-8529-a6c7a1eeae89/kube-rbac-proxy-node/0.log" Apr 22 15:56:24.396434 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:24.396405 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6qsb_bdcb6dd7-b076-455a-8529-a6c7a1eeae89/kube-rbac-proxy-ovn-metrics/0.log" Apr 22 15:56:24.416292 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:24.416250 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6qsb_bdcb6dd7-b076-455a-8529-a6c7a1eeae89/northd/0.log" Apr 22 15:56:24.436698 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:24.436662 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6qsb_bdcb6dd7-b076-455a-8529-a6c7a1eeae89/nbdb/0.log" Apr 22 15:56:24.458181 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:24.458148 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6qsb_bdcb6dd7-b076-455a-8529-a6c7a1eeae89/sbdb/0.log" Apr 22 15:56:24.646710 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:24.646635 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6qsb_bdcb6dd7-b076-455a-8529-a6c7a1eeae89/ovnkube-controller/0.log" Apr 22 15:56:25.657841 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:25.657817 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-8894fc9bd-nmq7h_3d4eaf4b-2b28-4ad4-866f-ac541d15b00b/check-endpoints/0.log" Apr 22 15:56:25.679097 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:25.679078 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-8lfsw_68a481e1-e027-433e-84ea-cb8d42406d8f/network-check-target-container/0.log" Apr 22 15:56:26.383242 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:26.383219 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-twqv9/test-trainjob-zkhcw-node-0-0-hcc9l" Apr 22 15:56:26.491761 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:26.491732 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tm45s\" (UniqueName: \"kubernetes.io/projected/02a36763-cf9b-4414-9844-468006143fcd-kube-api-access-tm45s\") pod \"02a36763-cf9b-4414-9844-468006143fcd\" (UID: \"02a36763-cf9b-4414-9844-468006143fcd\") " Apr 22 15:56:26.493980 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:26.493948 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02a36763-cf9b-4414-9844-468006143fcd-kube-api-access-tm45s" (OuterVolumeSpecName: "kube-api-access-tm45s") pod "02a36763-cf9b-4414-9844-468006143fcd" (UID: "02a36763-cf9b-4414-9844-468006143fcd"). InnerVolumeSpecName "kube-api-access-tm45s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 22 15:56:26.593054 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:26.593024 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tm45s\" (UniqueName: \"kubernetes.io/projected/02a36763-cf9b-4414-9844-468006143fcd-kube-api-access-tm45s\") on node \"ip-10-0-131-168.ec2.internal\" DevicePath \"\"" Apr 22 15:56:26.617627 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:26.617600 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-rrrjp_727c0127-0608-49d1-a2a3-aa2d700b9898/iptables-alerter/0.log" Apr 22 15:56:26.619345 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:26.619296 2577 generic.go:358] "Generic (PLEG): container finished" podID="02a36763-cf9b-4414-9844-468006143fcd" containerID="6161ea7c77169ccbe0e68b989823d15da46bcb0445989bd75de57d66c56d4859" exitCode=0 Apr 22 15:56:26.619471 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:26.619367 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-twqv9/test-trainjob-zkhcw-node-0-0-hcc9l" event={"ID":"02a36763-cf9b-4414-9844-468006143fcd","Type":"ContainerDied","Data":"6161ea7c77169ccbe0e68b989823d15da46bcb0445989bd75de57d66c56d4859"} Apr 22 15:56:26.619471 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:26.619407 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-twqv9/test-trainjob-zkhcw-node-0-0-hcc9l" event={"ID":"02a36763-cf9b-4414-9844-468006143fcd","Type":"ContainerDied","Data":"7648b0e6e82cfed05a735eaa3ffac04020ea070b9446258b244585a11133d832"} Apr 22 15:56:26.619471 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:26.619427 2577 scope.go:117] "RemoveContainer" containerID="6161ea7c77169ccbe0e68b989823d15da46bcb0445989bd75de57d66c56d4859" Apr 22 15:56:26.619471 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:26.619377 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-twqv9/test-trainjob-zkhcw-node-0-0-hcc9l" Apr 22 15:56:26.635997 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:26.635817 2577 scope.go:117] "RemoveContainer" containerID="6161ea7c77169ccbe0e68b989823d15da46bcb0445989bd75de57d66c56d4859" Apr 22 15:56:26.636200 ip-10-0-131-168 kubenswrapper[2577]: E0422 15:56:26.636158 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6161ea7c77169ccbe0e68b989823d15da46bcb0445989bd75de57d66c56d4859\": container with ID starting with 6161ea7c77169ccbe0e68b989823d15da46bcb0445989bd75de57d66c56d4859 not found: ID does not exist" containerID="6161ea7c77169ccbe0e68b989823d15da46bcb0445989bd75de57d66c56d4859" Apr 22 15:56:26.636270 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:26.636209 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6161ea7c77169ccbe0e68b989823d15da46bcb0445989bd75de57d66c56d4859"} err="failed to get container status \"6161ea7c77169ccbe0e68b989823d15da46bcb0445989bd75de57d66c56d4859\": rpc error: code = NotFound desc = could not find container \"6161ea7c77169ccbe0e68b989823d15da46bcb0445989bd75de57d66c56d4859\": container with ID starting with 6161ea7c77169ccbe0e68b989823d15da46bcb0445989bd75de57d66c56d4859 not found: ID does not exist" Apr 22 15:56:26.648059 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:26.648033 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-twqv9/test-trainjob-zkhcw-node-0-0-hcc9l"] Apr 22 15:56:26.650588 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:26.650559 2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-twqv9/test-trainjob-zkhcw-node-0-0-hcc9l"] Apr 22 15:56:27.210905 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:27.210841 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-hz7vw_be504427-e741-4041-98dc-92cc1db1293a/tuned/0.log" Apr 22 15:56:27.508596 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:27.508553 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02a36763-cf9b-4414-9844-468006143fcd" path="/var/lib/kubelet/pods/02a36763-cf9b-4414-9844-468006143fcd/volumes" Apr 22 15:56:28.849923 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:28.849885 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-6dc5bdb6b4-k8gnz_b3b905d0-a65d-4615-8789-ccfb785189a3/cluster-samples-operator/0.log" Apr 22 15:56:28.865594 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:28.865570 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-6dc5bdb6b4-k8gnz_b3b905d0-a65d-4615-8789-ccfb785189a3/cluster-samples-operator-watch/0.log" Apr 22 15:56:29.717143 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:29.717113 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca-operator_service-ca-operator-d6fc45fc5-bd6gc_68da1e47-8873-416c-b2d9-d515d0985631/service-ca-operator/1.log" Apr 22 15:56:29.784249 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:29.784213 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca-operator_service-ca-operator-d6fc45fc5-bd6gc_68da1e47-8873-416c-b2d9-d515d0985631/service-ca-operator/0.log" Apr 22 15:56:30.067706 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:30.067675 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca_service-ca-865cb79987-pgmlp_7f71ee10-0a7c-4fd9-80ee-56c493e3034e/service-ca-controller/0.log" Apr 22 15:56:30.574253 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:30.574229 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-vmfs2_0510159c-4726-4872-8eb0-5d435faa0b32/csi-driver/0.log" Apr 22 15:56:30.593834 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:30.593809 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-vmfs2_0510159c-4726-4872-8eb0-5d435faa0b32/csi-node-driver-registrar/0.log" Apr 22 15:56:30.614099 ip-10-0-131-168 kubenswrapper[2577]: I0422 15:56:30.614073 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-vmfs2_0510159c-4726-4872-8eb0-5d435faa0b32/csi-liveness-probe/0.log"