Apr 22 21:06:34.241426 ip-10-0-136-20 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Apr 22 21:06:34.241440 ip-10-0-136-20 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Apr 22 21:06:34.241449 ip-10-0-136-20 systemd[1]: kubelet.service: Failed with result 'resources'. Apr 22 21:06:34.241707 ip-10-0-136-20 systemd[1]: Failed to start Kubernetes Kubelet. Apr 22 21:06:44.473429 ip-10-0-136-20 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Apr 22 21:06:44.473447 ip-10-0-136-20 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 862efd02cd1b4d32bfb89134179636a5 -- Apr 22 21:09:19.155135 ip-10-0-136-20 systemd[1]: Starting Kubernetes Kubelet... Apr 22 21:09:19.599628 ip-10-0-136-20 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Apr 22 21:09:19.599628 ip-10-0-136-20 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Apr 22 21:09:19.599628 ip-10-0-136-20 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Apr 22 21:09:19.599628 ip-10-0-136-20 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Apr 22 21:09:19.599628 ip-10-0-136-20 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Apr 22 21:09:19.602953 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.602862 2580 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Apr 22 21:09:19.606826 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606810 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 22 21:09:19.606826 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606826 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 22 21:09:19.606902 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606830 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 22 21:09:19.606902 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606833 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 22 21:09:19.606902 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606837 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 22 21:09:19.606902 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606840 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 22 21:09:19.606902 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606843 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 22 21:09:19.606902 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606846 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 22 21:09:19.606902 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606863 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 22 21:09:19.606902 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606867 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 22 21:09:19.606902 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606871 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 22 21:09:19.606902 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606874 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 22 21:09:19.606902 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606876 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 22 21:09:19.606902 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606879 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 22 21:09:19.606902 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606881 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 22 21:09:19.606902 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606884 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 22 21:09:19.606902 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606886 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 22 21:09:19.606902 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606889 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 22 21:09:19.606902 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606891 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 22 21:09:19.606902 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606894 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 22 21:09:19.606902 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606897 2580 feature_gate.go:328] unrecognized feature gate: Example2 Apr 22 21:09:19.607341 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606900 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 22 21:09:19.607341 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606904 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 22 21:09:19.607341 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606908 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 22 21:09:19.607341 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606911 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 22 21:09:19.607341 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606914 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 22 21:09:19.607341 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606917 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 22 21:09:19.607341 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606920 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 22 21:09:19.607341 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606923 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 22 21:09:19.607341 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606926 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 22 21:09:19.607341 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606929 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 22 21:09:19.607341 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606932 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 22 21:09:19.607341 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606935 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 22 21:09:19.607341 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606938 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 22 21:09:19.607341 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606941 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 22 21:09:19.607341 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606944 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 22 21:09:19.607341 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606946 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 22 21:09:19.607341 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606948 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 22 21:09:19.607341 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606951 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 22 21:09:19.607341 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606954 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 22 21:09:19.607789 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606956 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 22 21:09:19.607789 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606958 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 22 21:09:19.607789 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606961 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 22 21:09:19.607789 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606963 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 22 21:09:19.607789 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606966 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 22 21:09:19.607789 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606968 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 22 21:09:19.607789 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606971 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 22 21:09:19.607789 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606973 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 22 21:09:19.607789 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606975 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 22 21:09:19.607789 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606978 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 22 21:09:19.607789 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606980 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 22 21:09:19.607789 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606983 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 22 21:09:19.607789 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606986 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 22 21:09:19.607789 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606988 2580 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 22 21:09:19.607789 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606991 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 22 21:09:19.607789 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606994 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 22 21:09:19.607789 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606997 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 22 21:09:19.607789 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.606999 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 22 21:09:19.607789 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607002 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 22 21:09:19.608268 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607005 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 22 21:09:19.608268 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607007 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 22 21:09:19.608268 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607010 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 22 21:09:19.608268 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607013 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 22 21:09:19.608268 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607015 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 22 21:09:19.608268 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607019 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 22 21:09:19.608268 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607021 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 22 21:09:19.608268 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607024 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 22 21:09:19.608268 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607026 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 22 21:09:19.608268 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607028 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 22 21:09:19.608268 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607031 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 22 21:09:19.608268 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607033 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 22 21:09:19.608268 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607035 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 22 21:09:19.608268 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607038 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 22 21:09:19.608268 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607040 2580 feature_gate.go:328] unrecognized feature gate: Example Apr 22 21:09:19.608268 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607043 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 22 21:09:19.608268 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607045 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 22 21:09:19.608268 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607048 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 22 21:09:19.608268 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607051 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 22 21:09:19.608268 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607054 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 22 21:09:19.608747 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607056 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 22 21:09:19.608747 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607059 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 22 21:09:19.608747 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607062 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 22 21:09:19.608747 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607065 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 22 21:09:19.608747 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607067 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 22 21:09:19.608747 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607070 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 22 21:09:19.608747 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607073 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 22 21:09:19.608747 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607484 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 22 21:09:19.608747 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607489 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 22 21:09:19.608747 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607492 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 22 21:09:19.608747 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607495 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 22 21:09:19.608747 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607499 2580 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 22 21:09:19.608747 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607502 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 22 21:09:19.608747 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607505 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 22 21:09:19.608747 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607509 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 22 21:09:19.608747 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607512 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 22 21:09:19.608747 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607514 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 22 21:09:19.608747 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607517 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 22 21:09:19.608747 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607520 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 22 21:09:19.608747 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607523 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 22 21:09:19.609240 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607525 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 22 21:09:19.609240 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607528 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 22 21:09:19.609240 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607530 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 22 21:09:19.609240 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607533 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 22 21:09:19.609240 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607536 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 22 21:09:19.609240 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607538 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 22 21:09:19.609240 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607541 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 22 21:09:19.609240 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607544 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 22 21:09:19.609240 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607546 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 22 21:09:19.609240 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607549 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 22 21:09:19.609240 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607552 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 22 21:09:19.609240 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607554 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 22 21:09:19.609240 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607557 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 22 21:09:19.609240 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607560 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 22 21:09:19.609240 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607563 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 22 21:09:19.609240 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607566 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 22 21:09:19.609240 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607569 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 22 21:09:19.609240 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607571 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 22 21:09:19.609240 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607574 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 22 21:09:19.609240 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607577 2580 feature_gate.go:328] unrecognized feature gate: Example2 Apr 22 21:09:19.609738 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607580 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 22 21:09:19.609738 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607582 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 22 21:09:19.609738 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607585 2580 feature_gate.go:328] unrecognized feature gate: Example Apr 22 21:09:19.609738 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607587 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 22 21:09:19.609738 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607589 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 22 21:09:19.609738 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607593 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 22 21:09:19.609738 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607595 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 22 21:09:19.609738 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607598 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 22 21:09:19.609738 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607601 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 22 21:09:19.609738 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607603 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 22 21:09:19.609738 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607606 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 22 21:09:19.609738 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607608 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 22 21:09:19.609738 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607610 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 22 21:09:19.609738 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607613 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 22 21:09:19.609738 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607615 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 22 21:09:19.609738 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607618 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 22 21:09:19.609738 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607620 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 22 21:09:19.609738 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607623 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 22 21:09:19.609738 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607625 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 22 21:09:19.609738 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607628 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 22 21:09:19.609738 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607630 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 22 21:09:19.610269 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607633 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 22 21:09:19.610269 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607635 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 22 21:09:19.610269 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607637 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 22 21:09:19.610269 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607640 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 22 21:09:19.610269 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607643 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 22 21:09:19.610269 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607645 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 22 21:09:19.610269 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607648 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 22 21:09:19.610269 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607651 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 22 21:09:19.610269 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607655 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 22 21:09:19.610269 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607658 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 22 21:09:19.610269 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607660 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 22 21:09:19.610269 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607663 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 22 21:09:19.610269 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607666 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 22 21:09:19.610269 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607669 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 22 21:09:19.610269 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607671 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 22 21:09:19.610269 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607673 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 22 21:09:19.610269 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607676 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 22 21:09:19.610269 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607678 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 22 21:09:19.610725 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607681 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 22 21:09:19.610725 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607683 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 22 21:09:19.610725 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607687 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 22 21:09:19.610725 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607689 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 22 21:09:19.610725 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607692 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 22 21:09:19.610725 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607694 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 22 21:09:19.610725 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607697 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 22 21:09:19.610725 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607699 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 22 21:09:19.610725 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607703 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 22 21:09:19.610725 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607706 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 22 21:09:19.610725 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607709 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 22 21:09:19.610725 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607712 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 22 21:09:19.610725 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607714 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 22 21:09:19.610725 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.607716 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 22 21:09:19.610725 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607813 2580 flags.go:64] FLAG: --address="0.0.0.0" Apr 22 21:09:19.610725 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607839 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Apr 22 21:09:19.610725 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607845 2580 flags.go:64] FLAG: --anonymous-auth="true" Apr 22 21:09:19.610725 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607862 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Apr 22 21:09:19.610725 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607868 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Apr 22 21:09:19.610725 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607871 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Apr 22 21:09:19.610725 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607876 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Apr 22 21:09:19.611251 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607880 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Apr 22 21:09:19.611251 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607883 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Apr 22 21:09:19.611251 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607887 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Apr 22 21:09:19.611251 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607890 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Apr 22 21:09:19.611251 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607893 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Apr 22 21:09:19.611251 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607897 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Apr 22 21:09:19.611251 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607899 2580 flags.go:64] FLAG: --cgroup-root="" Apr 22 21:09:19.611251 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607902 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Apr 22 21:09:19.611251 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607905 2580 flags.go:64] FLAG: --client-ca-file="" Apr 22 21:09:19.611251 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607908 2580 flags.go:64] FLAG: --cloud-config="" Apr 22 21:09:19.611251 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607911 2580 flags.go:64] FLAG: --cloud-provider="external" Apr 22 21:09:19.611251 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607914 2580 flags.go:64] FLAG: --cluster-dns="[]" Apr 22 21:09:19.611251 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607920 2580 flags.go:64] FLAG: --cluster-domain="" Apr 22 21:09:19.611251 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607923 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Apr 22 21:09:19.611251 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607926 2580 flags.go:64] FLAG: --config-dir="" Apr 22 21:09:19.611251 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607929 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Apr 22 21:09:19.611251 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607932 2580 flags.go:64] FLAG: --container-log-max-files="5" Apr 22 21:09:19.611251 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607936 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Apr 22 21:09:19.611251 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607939 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Apr 22 21:09:19.611251 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607944 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Apr 22 21:09:19.611251 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607947 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Apr 22 21:09:19.611251 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607950 2580 flags.go:64] FLAG: --contention-profiling="false" Apr 22 21:09:19.611251 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607953 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Apr 22 21:09:19.611251 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607956 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Apr 22 21:09:19.611870 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607959 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Apr 22 21:09:19.611870 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607962 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Apr 22 21:09:19.611870 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607966 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Apr 22 21:09:19.611870 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607969 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Apr 22 21:09:19.611870 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607972 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Apr 22 21:09:19.611870 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607975 2580 flags.go:64] FLAG: --enable-load-reader="false" Apr 22 21:09:19.611870 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607978 2580 flags.go:64] FLAG: --enable-server="true" Apr 22 21:09:19.611870 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607981 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Apr 22 21:09:19.611870 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607986 2580 flags.go:64] FLAG: --event-burst="100" Apr 22 21:09:19.611870 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607989 2580 flags.go:64] FLAG: --event-qps="50" Apr 22 21:09:19.611870 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607992 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Apr 22 21:09:19.611870 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607995 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Apr 22 21:09:19.611870 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.607998 2580 flags.go:64] FLAG: --eviction-hard="" Apr 22 21:09:19.611870 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608002 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Apr 22 21:09:19.611870 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608005 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Apr 22 21:09:19.611870 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608008 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Apr 22 21:09:19.611870 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608011 2580 flags.go:64] FLAG: --eviction-soft="" Apr 22 21:09:19.611870 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608014 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Apr 22 21:09:19.611870 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608017 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Apr 22 21:09:19.611870 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608020 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Apr 22 21:09:19.611870 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608023 2580 flags.go:64] FLAG: --experimental-mounter-path="" Apr 22 21:09:19.611870 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608026 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Apr 22 21:09:19.611870 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608029 2580 flags.go:64] FLAG: --fail-swap-on="true" Apr 22 21:09:19.611870 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608032 2580 flags.go:64] FLAG: --feature-gates="" Apr 22 21:09:19.611870 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608036 2580 flags.go:64] FLAG: --file-check-frequency="20s" Apr 22 21:09:19.612475 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608039 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Apr 22 21:09:19.612475 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608043 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Apr 22 21:09:19.612475 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608047 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Apr 22 21:09:19.612475 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608050 2580 flags.go:64] FLAG: --healthz-port="10248" Apr 22 21:09:19.612475 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608053 2580 flags.go:64] FLAG: --help="false" Apr 22 21:09:19.612475 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608056 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-136-20.ec2.internal" Apr 22 21:09:19.612475 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608059 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Apr 22 21:09:19.612475 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608062 2580 flags.go:64] FLAG: --http-check-frequency="20s" Apr 22 21:09:19.612475 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608065 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Apr 22 21:09:19.612475 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608068 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Apr 22 21:09:19.612475 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608071 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Apr 22 21:09:19.612475 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608074 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Apr 22 21:09:19.612475 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608080 2580 flags.go:64] FLAG: --image-service-endpoint="" Apr 22 21:09:19.612475 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608083 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Apr 22 21:09:19.612475 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608086 2580 flags.go:64] FLAG: --kube-api-burst="100" Apr 22 21:09:19.612475 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608089 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Apr 22 21:09:19.612475 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608093 2580 flags.go:64] FLAG: --kube-api-qps="50" Apr 22 21:09:19.612475 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608096 2580 flags.go:64] FLAG: --kube-reserved="" Apr 22 21:09:19.612475 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608099 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Apr 22 21:09:19.612475 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608101 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Apr 22 21:09:19.612475 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608104 2580 flags.go:64] FLAG: --kubelet-cgroups="" Apr 22 21:09:19.612475 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608107 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Apr 22 21:09:19.612475 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608110 2580 flags.go:64] FLAG: --lock-file="" Apr 22 21:09:19.612475 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608113 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Apr 22 21:09:19.613061 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608116 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Apr 22 21:09:19.613061 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608119 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Apr 22 21:09:19.613061 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608124 2580 flags.go:64] FLAG: --log-json-split-stream="false" Apr 22 21:09:19.613061 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608127 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Apr 22 21:09:19.613061 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608130 2580 flags.go:64] FLAG: --log-text-split-stream="false" Apr 22 21:09:19.613061 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608133 2580 flags.go:64] FLAG: --logging-format="text" Apr 22 21:09:19.613061 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608135 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Apr 22 21:09:19.613061 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608139 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Apr 22 21:09:19.613061 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608142 2580 flags.go:64] FLAG: --manifest-url="" Apr 22 21:09:19.613061 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608144 2580 flags.go:64] FLAG: --manifest-url-header="" Apr 22 21:09:19.613061 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608150 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Apr 22 21:09:19.613061 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608153 2580 flags.go:64] FLAG: --max-open-files="1000000" Apr 22 21:09:19.613061 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608158 2580 flags.go:64] FLAG: --max-pods="110" Apr 22 21:09:19.613061 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608161 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Apr 22 21:09:19.613061 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608164 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Apr 22 21:09:19.613061 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608167 2580 flags.go:64] FLAG: --memory-manager-policy="None" Apr 22 21:09:19.613061 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608169 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Apr 22 21:09:19.613061 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608172 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Apr 22 21:09:19.613061 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608175 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Apr 22 21:09:19.613061 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608178 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Apr 22 21:09:19.613061 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608187 2580 flags.go:64] FLAG: --node-status-max-images="50" Apr 22 21:09:19.613061 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608190 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Apr 22 21:09:19.613061 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608193 2580 flags.go:64] FLAG: --oom-score-adj="-999" Apr 22 21:09:19.613061 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608196 2580 flags.go:64] FLAG: --pod-cidr="" Apr 22 21:09:19.613666 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608198 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c8cfe89231412ff3ee8cb6207fa0be33cad0f08e88c9c0f1e9f7e8c6f14d6715" Apr 22 21:09:19.613666 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608204 2580 flags.go:64] FLAG: --pod-manifest-path="" Apr 22 21:09:19.613666 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608207 2580 flags.go:64] FLAG: --pod-max-pids="-1" Apr 22 21:09:19.613666 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608210 2580 flags.go:64] FLAG: --pods-per-core="0" Apr 22 21:09:19.613666 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608213 2580 flags.go:64] FLAG: --port="10250" Apr 22 21:09:19.613666 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608216 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Apr 22 21:09:19.613666 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608219 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-092096de882b0a61f" Apr 22 21:09:19.613666 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608222 2580 flags.go:64] FLAG: --qos-reserved="" Apr 22 21:09:19.613666 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608225 2580 flags.go:64] FLAG: --read-only-port="10255" Apr 22 21:09:19.613666 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608228 2580 flags.go:64] FLAG: --register-node="true" Apr 22 21:09:19.613666 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608231 2580 flags.go:64] FLAG: --register-schedulable="true" Apr 22 21:09:19.613666 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608234 2580 flags.go:64] FLAG: --register-with-taints="" Apr 22 21:09:19.613666 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608238 2580 flags.go:64] FLAG: --registry-burst="10" Apr 22 21:09:19.613666 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608241 2580 flags.go:64] FLAG: --registry-qps="5" Apr 22 21:09:19.613666 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608244 2580 flags.go:64] FLAG: --reserved-cpus="" Apr 22 21:09:19.613666 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608247 2580 flags.go:64] FLAG: --reserved-memory="" Apr 22 21:09:19.613666 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608251 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Apr 22 21:09:19.613666 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608254 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Apr 22 21:09:19.613666 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608259 2580 flags.go:64] FLAG: --rotate-certificates="false" Apr 22 21:09:19.613666 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608262 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Apr 22 21:09:19.613666 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608265 2580 flags.go:64] FLAG: --runonce="false" Apr 22 21:09:19.613666 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608267 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Apr 22 21:09:19.613666 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608270 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Apr 22 21:09:19.613666 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608273 2580 flags.go:64] FLAG: --seccomp-default="false" Apr 22 21:09:19.613666 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608277 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Apr 22 21:09:19.614310 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608280 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Apr 22 21:09:19.614310 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608284 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Apr 22 21:09:19.614310 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608287 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Apr 22 21:09:19.614310 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608290 2580 flags.go:64] FLAG: --storage-driver-password="root" Apr 22 21:09:19.614310 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608294 2580 flags.go:64] FLAG: --storage-driver-secure="false" Apr 22 21:09:19.614310 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608297 2580 flags.go:64] FLAG: --storage-driver-table="stats" Apr 22 21:09:19.614310 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608300 2580 flags.go:64] FLAG: --storage-driver-user="root" Apr 22 21:09:19.614310 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608303 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Apr 22 21:09:19.614310 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608306 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Apr 22 21:09:19.614310 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608309 2580 flags.go:64] FLAG: --system-cgroups="" Apr 22 21:09:19.614310 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608311 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Apr 22 21:09:19.614310 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608316 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Apr 22 21:09:19.614310 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608319 2580 flags.go:64] FLAG: --tls-cert-file="" Apr 22 21:09:19.614310 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608322 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Apr 22 21:09:19.614310 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608326 2580 flags.go:64] FLAG: --tls-min-version="" Apr 22 21:09:19.614310 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608329 2580 flags.go:64] FLAG: --tls-private-key-file="" Apr 22 21:09:19.614310 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608332 2580 flags.go:64] FLAG: --topology-manager-policy="none" Apr 22 21:09:19.614310 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608335 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Apr 22 21:09:19.614310 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608337 2580 flags.go:64] FLAG: --topology-manager-scope="container" Apr 22 21:09:19.614310 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608340 2580 flags.go:64] FLAG: --v="2" Apr 22 21:09:19.614310 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608345 2580 flags.go:64] FLAG: --version="false" Apr 22 21:09:19.614310 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608349 2580 flags.go:64] FLAG: --vmodule="" Apr 22 21:09:19.614310 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608354 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Apr 22 21:09:19.614310 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.608357 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Apr 22 21:09:19.614310 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608445 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 22 21:09:19.615234 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608449 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 22 21:09:19.615234 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608452 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 22 21:09:19.615234 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608455 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 22 21:09:19.615234 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608458 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 22 21:09:19.615234 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608461 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 22 21:09:19.615234 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608464 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 22 21:09:19.615234 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608467 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 22 21:09:19.615234 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608470 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 22 21:09:19.615234 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608473 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 22 21:09:19.615234 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608475 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 22 21:09:19.615234 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608478 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 22 21:09:19.615234 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608484 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 22 21:09:19.615234 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608487 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 22 21:09:19.615234 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608490 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 22 21:09:19.615234 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608493 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 22 21:09:19.615234 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608496 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 22 21:09:19.615234 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608499 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 22 21:09:19.615234 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608501 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 22 21:09:19.615234 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608504 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 22 21:09:19.616035 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608507 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 22 21:09:19.616035 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608510 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 22 21:09:19.616035 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608513 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 22 21:09:19.616035 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608516 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 22 21:09:19.616035 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608519 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 22 21:09:19.616035 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608521 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 22 21:09:19.616035 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608524 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 22 21:09:19.616035 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608526 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 22 21:09:19.616035 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608529 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 22 21:09:19.616035 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608531 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 22 21:09:19.616035 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608534 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 22 21:09:19.616035 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608536 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 22 21:09:19.616035 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608539 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 22 21:09:19.616035 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608542 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 22 21:09:19.616035 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608545 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 22 21:09:19.616035 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608547 2580 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 22 21:09:19.616035 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608550 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 22 21:09:19.616035 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608554 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 22 21:09:19.616035 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608557 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 22 21:09:19.616823 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608559 2580 feature_gate.go:328] unrecognized feature gate: Example Apr 22 21:09:19.616823 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608561 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 22 21:09:19.616823 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608564 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 22 21:09:19.616823 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608566 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 22 21:09:19.616823 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608569 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 22 21:09:19.616823 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608573 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 22 21:09:19.616823 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608575 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 22 21:09:19.616823 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608578 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 22 21:09:19.616823 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608581 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 22 21:09:19.616823 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608583 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 22 21:09:19.616823 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608586 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 22 21:09:19.616823 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608588 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 22 21:09:19.616823 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608591 2580 feature_gate.go:328] unrecognized feature gate: Example2 Apr 22 21:09:19.616823 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608593 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 22 21:09:19.616823 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608595 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 22 21:09:19.616823 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608598 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 22 21:09:19.616823 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608601 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 22 21:09:19.616823 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608603 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 22 21:09:19.616823 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608605 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 22 21:09:19.616823 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608608 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 22 21:09:19.617535 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608611 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 22 21:09:19.617535 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608613 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 22 21:09:19.617535 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608616 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 22 21:09:19.617535 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608618 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 22 21:09:19.617535 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608621 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 22 21:09:19.617535 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608623 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 22 21:09:19.617535 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608627 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 22 21:09:19.617535 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608630 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 22 21:09:19.617535 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608632 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 22 21:09:19.617535 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608634 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 22 21:09:19.617535 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608637 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 22 21:09:19.617535 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608640 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 22 21:09:19.617535 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608642 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 22 21:09:19.617535 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608645 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 22 21:09:19.617535 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608647 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 22 21:09:19.617535 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608650 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 22 21:09:19.617535 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608652 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 22 21:09:19.617535 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608656 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 22 21:09:19.617535 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608658 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 22 21:09:19.617535 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608661 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 22 21:09:19.618419 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608663 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 22 21:09:19.618419 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608666 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 22 21:09:19.618419 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608668 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 22 21:09:19.618419 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608671 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 22 21:09:19.618419 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608673 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 22 21:09:19.618419 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608675 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 22 21:09:19.618419 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.608678 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 22 21:09:19.618419 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.609492 2580 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Apr 22 21:09:19.618419 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.618018 2580 server.go:530] "Kubelet version" kubeletVersion="v1.33.9" Apr 22 21:09:19.618419 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.618040 2580 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Apr 22 21:09:19.618419 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618115 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 22 21:09:19.618419 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618122 2580 feature_gate.go:328] unrecognized feature gate: Example Apr 22 21:09:19.618419 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618127 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 22 21:09:19.618419 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618132 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 22 21:09:19.618419 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618137 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 22 21:09:19.618419 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618141 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 22 21:09:19.619254 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618146 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 22 21:09:19.619254 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618149 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 22 21:09:19.619254 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618154 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 22 21:09:19.619254 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618158 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 22 21:09:19.619254 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618162 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 22 21:09:19.619254 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618166 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 22 21:09:19.619254 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618170 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 22 21:09:19.619254 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618175 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 22 21:09:19.619254 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618179 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 22 21:09:19.619254 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618183 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 22 21:09:19.619254 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618187 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 22 21:09:19.619254 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618191 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 22 21:09:19.619254 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618195 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 22 21:09:19.619254 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618199 2580 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 22 21:09:19.619254 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618203 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 22 21:09:19.619254 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618207 2580 feature_gate.go:328] unrecognized feature gate: Example2 Apr 22 21:09:19.619254 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618211 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 22 21:09:19.619254 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618215 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 22 21:09:19.619254 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618219 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 22 21:09:19.619254 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618223 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 22 21:09:19.619746 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618228 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 22 21:09:19.619746 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618233 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 22 21:09:19.619746 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618237 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 22 21:09:19.619746 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618241 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 22 21:09:19.619746 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618245 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 22 21:09:19.619746 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618250 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 22 21:09:19.619746 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618255 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 22 21:09:19.619746 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618259 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 22 21:09:19.619746 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618263 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 22 21:09:19.619746 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618267 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 22 21:09:19.619746 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618271 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 22 21:09:19.619746 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618276 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 22 21:09:19.619746 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618280 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 22 21:09:19.619746 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618284 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 22 21:09:19.619746 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618288 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 22 21:09:19.619746 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618292 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 22 21:09:19.619746 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618296 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 22 21:09:19.619746 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618300 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 22 21:09:19.619746 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618303 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 22 21:09:19.619746 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618308 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 22 21:09:19.620362 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618312 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 22 21:09:19.620362 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618316 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 22 21:09:19.620362 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618320 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 22 21:09:19.620362 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618324 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 22 21:09:19.620362 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618328 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 22 21:09:19.620362 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618334 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 22 21:09:19.620362 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618341 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 22 21:09:19.620362 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618345 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 22 21:09:19.620362 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618349 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 22 21:09:19.620362 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618354 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 22 21:09:19.620362 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618358 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 22 21:09:19.620362 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618362 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 22 21:09:19.620362 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618366 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 22 21:09:19.620362 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618370 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 22 21:09:19.620362 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618374 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 22 21:09:19.620362 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618378 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 22 21:09:19.620362 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618383 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 22 21:09:19.620362 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618387 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 22 21:09:19.620362 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618392 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 22 21:09:19.621102 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618396 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 22 21:09:19.621102 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618400 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 22 21:09:19.621102 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618404 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 22 21:09:19.621102 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618408 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 22 21:09:19.621102 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618413 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 22 21:09:19.621102 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618417 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 22 21:09:19.621102 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618420 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 22 21:09:19.621102 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618424 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 22 21:09:19.621102 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618428 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 22 21:09:19.621102 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618431 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 22 21:09:19.621102 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618435 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 22 21:09:19.621102 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618440 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 22 21:09:19.621102 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618444 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 22 21:09:19.621102 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618449 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 22 21:09:19.621102 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618454 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 22 21:09:19.621102 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618460 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 22 21:09:19.621102 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618467 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 22 21:09:19.621102 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618472 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 22 21:09:19.621102 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618477 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 22 21:09:19.621571 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618481 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 22 21:09:19.621571 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618486 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 22 21:09:19.621571 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.618494 2580 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Apr 22 21:09:19.621571 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618646 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 22 21:09:19.621571 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618655 2580 feature_gate.go:328] unrecognized feature gate: Example2 Apr 22 21:09:19.621571 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618660 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 22 21:09:19.621571 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618665 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 22 21:09:19.621571 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618670 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 22 21:09:19.621571 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618675 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 22 21:09:19.621571 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618679 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 22 21:09:19.621571 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618683 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 22 21:09:19.621571 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618688 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 22 21:09:19.621571 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618692 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 22 21:09:19.621571 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618696 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 22 21:09:19.621571 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618700 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 22 21:09:19.621571 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618705 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 22 21:09:19.622174 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618709 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 22 21:09:19.622174 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618713 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 22 21:09:19.622174 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618717 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 22 21:09:19.622174 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618721 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 22 21:09:19.622174 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618726 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 22 21:09:19.622174 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618730 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 22 21:09:19.622174 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618734 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 22 21:09:19.622174 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618739 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 22 21:09:19.622174 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618743 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 22 21:09:19.622174 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618748 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 22 21:09:19.622174 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618752 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 22 21:09:19.622174 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618757 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 22 21:09:19.622174 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618762 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 22 21:09:19.622174 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618766 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 22 21:09:19.622174 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618770 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 22 21:09:19.622174 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618774 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 22 21:09:19.622174 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618778 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 22 21:09:19.622174 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618783 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 22 21:09:19.622174 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618787 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 22 21:09:19.622174 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618791 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 22 21:09:19.622952 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618795 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 22 21:09:19.622952 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618799 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 22 21:09:19.622952 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618803 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 22 21:09:19.622952 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618808 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 22 21:09:19.622952 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618812 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 22 21:09:19.622952 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618816 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 22 21:09:19.622952 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618821 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 22 21:09:19.622952 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618825 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 22 21:09:19.622952 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618829 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 22 21:09:19.622952 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618833 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 22 21:09:19.622952 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618837 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 22 21:09:19.622952 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618841 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 22 21:09:19.622952 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618846 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 22 21:09:19.622952 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618871 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 22 21:09:19.622952 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618875 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 22 21:09:19.622952 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618879 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 22 21:09:19.622952 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618884 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 22 21:09:19.622952 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618887 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 22 21:09:19.622952 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618891 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 22 21:09:19.622952 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618894 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 22 21:09:19.623548 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618898 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 22 21:09:19.623548 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618901 2580 feature_gate.go:328] unrecognized feature gate: Example Apr 22 21:09:19.623548 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618906 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 22 21:09:19.623548 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618910 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 22 21:09:19.623548 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618915 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 22 21:09:19.623548 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618920 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 22 21:09:19.623548 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618925 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 22 21:09:19.623548 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618929 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 22 21:09:19.623548 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618932 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 22 21:09:19.623548 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618936 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 22 21:09:19.623548 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618940 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 22 21:09:19.623548 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618943 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 22 21:09:19.623548 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618947 2580 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 22 21:09:19.623548 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618951 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 22 21:09:19.623548 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618955 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 22 21:09:19.623548 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618958 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 22 21:09:19.623548 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618962 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 22 21:09:19.623548 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618966 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 22 21:09:19.623548 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618969 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 22 21:09:19.624063 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618974 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 22 21:09:19.624063 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618978 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 22 21:09:19.624063 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618982 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 22 21:09:19.624063 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618986 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 22 21:09:19.624063 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618990 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 22 21:09:19.624063 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618994 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 22 21:09:19.624063 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.618998 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 22 21:09:19.624063 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.619003 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 22 21:09:19.624063 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.619007 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 22 21:09:19.624063 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.619011 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 22 21:09:19.624063 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.619015 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 22 21:09:19.624063 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.619019 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 22 21:09:19.624063 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.619023 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 22 21:09:19.624063 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:19.619029 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 22 21:09:19.624063 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.619037 2580 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Apr 22 21:09:19.624512 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.619971 2580 server.go:962] "Client rotation is on, will bootstrap in background" Apr 22 21:09:19.624512 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.623385 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Apr 22 21:09:19.624512 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.624369 2580 server.go:1019] "Starting client certificate rotation" Apr 22 21:09:19.624512 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.624464 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Apr 22 21:09:19.624512 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.624505 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Apr 22 21:09:19.651791 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.651768 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Apr 22 21:09:19.654279 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.654262 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Apr 22 21:09:19.670290 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.670273 2580 log.go:25] "Validated CRI v1 runtime API" Apr 22 21:09:19.676136 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.675714 2580 log.go:25] "Validated CRI v1 image API" Apr 22 21:09:19.677381 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.677364 2580 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Apr 22 21:09:19.681454 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.681422 2580 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Apr 22 21:09:19.681644 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.681618 2580 fs.go:135] Filesystem UUIDs: map[03481b27-7345-416d-8aa2-08a5e27d51c2:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 fc772440-6cee-4834-86fe-3e610103465c:/dev/nvme0n1p4] Apr 22 21:09:19.681682 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.681649 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Apr 22 21:09:19.688100 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.687984 2580 manager.go:217] Machine: {Timestamp:2026-04-22 21:09:19.6859889 +0000 UTC m=+0.411885491 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3091124 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27eb79f0e7617af91b42bc8ac16cb4 SystemUUID:ec27eb79-f0e7-617a-f91b-42bc8ac16cb4 BootID:862efd02-cd1b-4d32-bfb8-9134179636a5 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:62:0f:dd:75:5b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:62:0f:dd:75:5b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4e:95:09:c4:b0:01 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Apr 22 21:09:19.688100 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.688089 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Apr 22 21:09:19.688231 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.688214 2580 manager.go:233] Version: {KernelVersion:5.14.0-570.107.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260414-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Apr 22 21:09:19.689143 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.689116 2580 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Apr 22 21:09:19.689306 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.689147 2580 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-20.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Apr 22 21:09:19.689389 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.689321 2580 topology_manager.go:138] "Creating topology manager with none policy" Apr 22 21:09:19.689389 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.689332 2580 container_manager_linux.go:306] "Creating device plugin manager" Apr 22 21:09:19.689389 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.689350 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Apr 22 21:09:19.690053 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.690040 2580 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Apr 22 21:09:19.691269 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.691255 2580 state_mem.go:36] "Initialized new in-memory state store" Apr 22 21:09:19.691393 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.691382 2580 server.go:1267] "Using root directory" path="/var/lib/kubelet" Apr 22 21:09:19.693874 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.693862 2580 kubelet.go:491] "Attempting to sync node with API server" Apr 22 21:09:19.693939 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.693883 2580 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Apr 22 21:09:19.693939 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.693899 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Apr 22 21:09:19.693939 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.693914 2580 kubelet.go:397] "Adding apiserver pod source" Apr 22 21:09:19.693939 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.693927 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Apr 22 21:09:19.695112 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.695099 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Apr 22 21:09:19.695187 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.695121 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Apr 22 21:09:19.697923 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.697893 2580 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.10-2.rhaos4.20.gita4d0894.el9" apiVersion="v1" Apr 22 21:09:19.698514 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.698496 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8vf26" Apr 22 21:09:19.699123 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.699110 2580 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Apr 22 21:09:19.700807 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.700797 2580 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Apr 22 21:09:19.700869 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.700814 2580 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Apr 22 21:09:19.700869 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.700820 2580 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Apr 22 21:09:19.700869 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.700826 2580 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Apr 22 21:09:19.700869 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.700831 2580 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Apr 22 21:09:19.700869 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.700837 2580 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Apr 22 21:09:19.700869 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.700844 2580 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Apr 22 21:09:19.700869 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.700871 2580 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Apr 22 21:09:19.701054 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.700881 2580 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Apr 22 21:09:19.701054 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.700887 2580 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Apr 22 21:09:19.701054 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.700896 2580 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Apr 22 21:09:19.701054 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.700905 2580 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Apr 22 21:09:19.702743 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.702726 2580 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Apr 22 21:09:19.702743 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.702743 2580 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Apr 22 21:09:19.703249 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:19.703226 2580 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Apr 22 21:09:19.703317 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:19.703303 2580 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-20.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Apr 22 21:09:19.705704 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.705676 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8vf26" Apr 22 21:09:19.706478 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.706466 2580 watchdog_linux.go:99] "Systemd watchdog is not enabled" Apr 22 21:09:19.706539 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.706501 2580 server.go:1295] "Started kubelet" Apr 22 21:09:19.707107 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.706597 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Apr 22 21:09:19.707205 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.707075 2580 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Apr 22 21:09:19.707205 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.707143 2580 server_v1.go:47] "podresources" method="list" useActivePods=true Apr 22 21:09:19.707426 ip-10-0-136-20 systemd[1]: Started Kubernetes Kubelet. Apr 22 21:09:19.708597 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.708577 2580 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Apr 22 21:09:19.709273 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.709260 2580 server.go:317] "Adding debug handlers to kubelet server" Apr 22 21:09:19.713451 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.713430 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Apr 22 21:09:19.714111 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.714092 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Apr 22 21:09:19.714757 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.714733 2580 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Apr 22 21:09:19.714846 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.714735 2580 volume_manager.go:295] "The desired_state_of_world populator starts" Apr 22 21:09:19.714846 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.714783 2580 volume_manager.go:297] "Starting Kubelet Volume Manager" Apr 22 21:09:19.714968 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.714940 2580 reconstruct.go:97] "Volume reconstruction finished" Apr 22 21:09:19.714968 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.714948 2580 reconciler.go:26] "Reconciler: start to sync state" Apr 22 21:09:19.715053 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:19.715009 2580 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-20.ec2.internal\" not found" Apr 22 21:09:19.716003 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.715981 2580 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Apr 22 21:09:19.716154 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.716138 2580 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-136-20.ec2.internal" not found Apr 22 21:09:19.717018 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.717002 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Apr 22 21:09:19.717018 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.717020 2580 factory.go:55] Registering systemd factory Apr 22 21:09:19.717151 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.717030 2580 factory.go:223] Registration of the systemd container factory successfully Apr 22 21:09:19.717316 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.717301 2580 factory.go:153] Registering CRI-O factory Apr 22 21:09:19.717389 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.717321 2580 factory.go:223] Registration of the crio container factory successfully Apr 22 21:09:19.717389 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.717350 2580 factory.go:103] Registering Raw factory Apr 22 21:09:19.717389 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.717363 2580 manager.go:1196] Started watching for new ooms in manager Apr 22 21:09:19.717749 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.717735 2580 manager.go:319] Starting recovery of all containers Apr 22 21:09:19.717813 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:19.717783 2580 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-20.ec2.internal\" not found" node="ip-10-0-136-20.ec2.internal" Apr 22 21:09:19.720205 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:19.720165 2580 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Apr 22 21:09:19.725212 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.725194 2580 manager.go:324] Recovery completed Apr 22 21:09:19.730183 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.730167 2580 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 22 21:09:19.730541 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.730528 2580 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-136-20.ec2.internal" not found Apr 22 21:09:19.732903 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.732889 2580 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasSufficientMemory" Apr 22 21:09:19.732982 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.732916 2580 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasNoDiskPressure" Apr 22 21:09:19.732982 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.732932 2580 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasSufficientPID" Apr 22 21:09:19.733410 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.733395 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Apr 22 21:09:19.733477 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.733410 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Apr 22 21:09:19.733477 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.733430 2580 state_mem.go:36] "Initialized new in-memory state store" Apr 22 21:09:19.735369 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.735353 2580 policy_none.go:49] "None policy: Start" Apr 22 21:09:19.735369 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.735370 2580 memory_manager.go:186] "Starting memorymanager" policy="None" Apr 22 21:09:19.735676 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.735380 2580 state_mem.go:35] "Initializing new in-memory state store" Apr 22 21:09:19.780411 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.780394 2580 manager.go:341] "Starting Device Plugin manager" Apr 22 21:09:19.786497 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:19.780425 2580 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Apr 22 21:09:19.786497 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.780434 2580 server.go:85] "Starting device plugin registration server" Apr 22 21:09:19.786497 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.780684 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Apr 22 21:09:19.786497 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.780699 2580 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Apr 22 21:09:19.786497 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.780802 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Apr 22 21:09:19.786497 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.780895 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Apr 22 21:09:19.786497 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.780904 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Apr 22 21:09:19.786497 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:19.781413 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Apr 22 21:09:19.786497 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:19.781457 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-20.ec2.internal\" not found" Apr 22 21:09:19.792508 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.792494 2580 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-136-20.ec2.internal" not found Apr 22 21:09:19.856380 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.856304 2580 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Apr 22 21:09:19.857784 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.857760 2580 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Apr 22 21:09:19.857873 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.857800 2580 status_manager.go:230] "Starting to sync pod status with apiserver" Apr 22 21:09:19.857873 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.857823 2580 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Apr 22 21:09:19.857873 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.857833 2580 kubelet.go:2451] "Starting kubelet main sync loop" Apr 22 21:09:19.858004 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:19.857948 2580 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Apr 22 21:09:19.861433 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.861412 2580 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Apr 22 21:09:19.880957 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.880940 2580 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 22 21:09:19.882142 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.882128 2580 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasSufficientMemory" Apr 22 21:09:19.882209 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.882155 2580 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasNoDiskPressure" Apr 22 21:09:19.882209 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.882166 2580 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasSufficientPID" Apr 22 21:09:19.882209 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.882187 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-20.ec2.internal" Apr 22 21:09:19.890259 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.890247 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-20.ec2.internal" Apr 22 21:09:19.890317 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:19.890266 2580 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-20.ec2.internal\": node \"ip-10-0-136-20.ec2.internal\" not found" Apr 22 21:09:19.907195 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:19.907176 2580 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-20.ec2.internal\" not found" Apr 22 21:09:19.958472 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.958433 2580 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-20.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal"] Apr 22 21:09:19.958549 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.958532 2580 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 22 21:09:19.959368 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.959344 2580 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasSufficientMemory" Apr 22 21:09:19.959482 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.959377 2580 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasNoDiskPressure" Apr 22 21:09:19.959482 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.959387 2580 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasSufficientPID" Apr 22 21:09:19.960458 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.960446 2580 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 22 21:09:19.960616 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.960600 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-20.ec2.internal" Apr 22 21:09:19.960683 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.960635 2580 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 22 21:09:19.961160 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.961143 2580 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasSufficientMemory" Apr 22 21:09:19.961240 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.961174 2580 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasNoDiskPressure" Apr 22 21:09:19.961240 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.961187 2580 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasSufficientPID" Apr 22 21:09:19.961240 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.961195 2580 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasSufficientMemory" Apr 22 21:09:19.961240 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.961214 2580 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasNoDiskPressure" Apr 22 21:09:19.961240 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.961223 2580 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasSufficientPID" Apr 22 21:09:19.962412 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.962397 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal" Apr 22 21:09:19.962489 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.962433 2580 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 22 21:09:19.963125 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.963110 2580 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasSufficientMemory" Apr 22 21:09:19.963224 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.963142 2580 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasNoDiskPressure" Apr 22 21:09:19.963224 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:19.963156 2580 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasSufficientPID" Apr 22 21:09:19.996538 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:19.996521 2580 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-20.ec2.internal\" not found" node="ip-10-0-136-20.ec2.internal" Apr 22 21:09:20.000861 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:20.000836 2580 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-20.ec2.internal\" not found" node="ip-10-0-136-20.ec2.internal" Apr 22 21:09:20.007366 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:20.007352 2580 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-20.ec2.internal\" not found" Apr 22 21:09:20.016491 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.016475 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7702f8c6e9277cd7f5964f802e4caebb-config\") pod \"kube-apiserver-proxy-ip-10-0-136-20.ec2.internal\" (UID: \"7702f8c6e9277cd7f5964f802e4caebb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-20.ec2.internal" Apr 22 21:09:20.016536 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.016507 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f511a0a82c52fc6840a29f94334af07f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal\" (UID: \"f511a0a82c52fc6840a29f94334af07f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal" Apr 22 21:09:20.016536 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.016528 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f511a0a82c52fc6840a29f94334af07f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal\" (UID: \"f511a0a82c52fc6840a29f94334af07f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal" Apr 22 21:09:20.107658 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:20.107583 2580 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-20.ec2.internal\" not found" Apr 22 21:09:20.116968 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.116947 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7702f8c6e9277cd7f5964f802e4caebb-config\") pod \"kube-apiserver-proxy-ip-10-0-136-20.ec2.internal\" (UID: \"7702f8c6e9277cd7f5964f802e4caebb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-20.ec2.internal" Apr 22 21:09:20.117028 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.116975 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f511a0a82c52fc6840a29f94334af07f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal\" (UID: \"f511a0a82c52fc6840a29f94334af07f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal" Apr 22 21:09:20.117028 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.116991 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f511a0a82c52fc6840a29f94334af07f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal\" (UID: \"f511a0a82c52fc6840a29f94334af07f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal" Apr 22 21:09:20.117096 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.117044 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f511a0a82c52fc6840a29f94334af07f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal\" (UID: \"f511a0a82c52fc6840a29f94334af07f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal" Apr 22 21:09:20.117096 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.117063 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f511a0a82c52fc6840a29f94334af07f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal\" (UID: \"f511a0a82c52fc6840a29f94334af07f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal" Apr 22 21:09:20.117096 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.117061 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7702f8c6e9277cd7f5964f802e4caebb-config\") pod \"kube-apiserver-proxy-ip-10-0-136-20.ec2.internal\" (UID: \"7702f8c6e9277cd7f5964f802e4caebb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-20.ec2.internal" Apr 22 21:09:20.208383 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:20.208347 2580 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-20.ec2.internal\" not found" Apr 22 21:09:20.298994 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.298972 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-20.ec2.internal" Apr 22 21:09:20.303500 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.303484 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal" Apr 22 21:09:20.308981 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:20.308962 2580 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-20.ec2.internal\" not found" Apr 22 21:09:20.409446 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:20.409424 2580 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-20.ec2.internal\" not found" Apr 22 21:09:20.509970 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:20.509950 2580 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-20.ec2.internal\" not found" Apr 22 21:09:20.610464 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:20.610433 2580 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-20.ec2.internal\" not found" Apr 22 21:09:20.623924 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.623909 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Apr 22 21:09:20.624043 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.624028 2580 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Apr 22 21:09:20.624079 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.624066 2580 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Apr 22 21:09:20.707580 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.707508 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-04-21 21:04:19 +0000 UTC" deadline="2028-02-01 10:08:37.467058922 +0000 UTC" Apr 22 21:09:20.707580 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.707539 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15588h59m16.759521987s" Apr 22 21:09:20.710638 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:20.710624 2580 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-20.ec2.internal\" not found" Apr 22 21:09:20.713798 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.713780 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Apr 22 21:09:20.736905 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.736886 2580 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Apr 22 21:09:20.756190 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.756171 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2qhbm" Apr 22 21:09:20.763614 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.763565 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2qhbm" Apr 22 21:09:20.781559 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.781412 2580 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Apr 22 21:09:20.810744 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:20.810717 2580 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-20.ec2.internal\" not found" Apr 22 21:09:20.852296 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:20.852271 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7702f8c6e9277cd7f5964f802e4caebb.slice/crio-493df4b5d6e36511c205be275ee4844280bb51a38fd8e791878ea8f0834fa309 WatchSource:0}: Error finding container 493df4b5d6e36511c205be275ee4844280bb51a38fd8e791878ea8f0834fa309: Status 404 returned error can't find the container with id 493df4b5d6e36511c205be275ee4844280bb51a38fd8e791878ea8f0834fa309 Apr 22 21:09:20.852755 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:20.852741 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf511a0a82c52fc6840a29f94334af07f.slice/crio-128c30282d80a50e7e4a4907b3bf23a9c6a623bd2aaa6072795c0e74f8f0bea9 WatchSource:0}: Error finding container 128c30282d80a50e7e4a4907b3bf23a9c6a623bd2aaa6072795c0e74f8f0bea9: Status 404 returned error can't find the container with id 128c30282d80a50e7e4a4907b3bf23a9c6a623bd2aaa6072795c0e74f8f0bea9 Apr 22 21:09:20.857794 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.857778 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 22 21:09:20.861018 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.860979 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal" event={"ID":"f511a0a82c52fc6840a29f94334af07f","Type":"ContainerStarted","Data":"128c30282d80a50e7e4a4907b3bf23a9c6a623bd2aaa6072795c0e74f8f0bea9"} Apr 22 21:09:20.861932 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.861907 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-20.ec2.internal" event={"ID":"7702f8c6e9277cd7f5964f802e4caebb","Type":"ContainerStarted","Data":"493df4b5d6e36511c205be275ee4844280bb51a38fd8e791878ea8f0834fa309"} Apr 22 21:09:20.903192 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.903170 2580 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Apr 22 21:09:20.915202 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.915183 2580 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-20.ec2.internal" Apr 22 21:09:20.923493 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.923479 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Apr 22 21:09:20.924399 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.924388 2580 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal" Apr 22 21:09:20.933902 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:20.933889 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Apr 22 21:09:21.694619 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.694591 2580 apiserver.go:52] "Watching apiserver" Apr 22 21:09:21.708548 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.708509 2580 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Apr 22 21:09:21.709698 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.709668 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-fx8k2","openshift-network-operator/iptables-alerter-z7k7s","openshift-ovn-kubernetes/ovnkube-node-8ngls","kube-system/konnectivity-agent-7xt7x","kube-system/kube-apiserver-proxy-ip-10-0-136-20.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw","openshift-dns/node-resolver-49dpb","openshift-image-registry/node-ca-wkk24","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal","openshift-multus/multus-additional-cni-plugins-qvlsp","openshift-network-diagnostics/network-check-target-lp7mc","openshift-cluster-node-tuning-operator/tuned-n66xm","openshift-multus/multus-zwlx7"] Apr 22 21:09:21.712465 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.712447 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-49dpb" Apr 22 21:09:21.712663 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.712606 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z7k7s" Apr 22 21:09:21.713577 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.713553 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.714794 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.714778 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7xt7x" Apr 22 21:09:21.714985 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.714966 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qrsr9\"" Apr 22 21:09:21.715381 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.715327 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Apr 22 21:09:21.715484 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.715445 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Apr 22 21:09:21.715565 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.715550 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Apr 22 21:09:21.715619 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.715567 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Apr 22 21:09:21.715962 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.715942 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" Apr 22 21:09:21.716389 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.716369 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Apr 22 21:09:21.716636 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.716621 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lcfnh\"" Apr 22 21:09:21.716846 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.716622 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hnktj\"" Apr 22 21:09:21.716976 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.716879 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Apr 22 21:09:21.717154 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.717135 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:21.717250 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:21.717226 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx8k2" podUID="843da596-cc50-4a3e-817c-4ee63a89c94a" Apr 22 21:09:21.717930 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.717910 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Apr 22 21:09:21.718030 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.718017 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Apr 22 21:09:21.718194 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.718177 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Apr 22 21:09:21.718281 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.718267 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Apr 22 21:09:21.718397 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.718380 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8dtqk\"" Apr 22 21:09:21.718449 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.718429 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Apr 22 21:09:21.718449 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.718435 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Apr 22 21:09:21.718551 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.718275 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Apr 22 21:09:21.718602 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.718594 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Apr 22 21:09:21.718902 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.718883 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Apr 22 21:09:21.719003 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.718919 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Apr 22 21:09:21.719003 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.718919 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-k884x\"" Apr 22 21:09:21.719430 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.719415 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wkk24" Apr 22 21:09:21.721466 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.720782 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qvlsp" Apr 22 21:09:21.721691 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.721671 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Apr 22 21:09:21.721962 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.721941 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Apr 22 21:09:21.722067 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.722049 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Apr 22 21:09:21.722127 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.722062 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xjz2h\"" Apr 22 21:09:21.722868 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.722832 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:09:21.722962 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:21.722931 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lp7mc" podUID="516cb834-5ee4-4971-80a9-149fd935fc80" Apr 22 21:09:21.723038 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.723018 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Apr 22 21:09:21.723340 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.723323 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Apr 22 21:09:21.724227 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.723966 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Apr 22 21:09:21.724227 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.724078 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-wtxv4\"" Apr 22 21:09:21.724723 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.724689 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Apr 22 21:09:21.725202 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.725186 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Apr 22 21:09:21.726442 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.726422 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw8wg\" (UniqueName: \"kubernetes.io/projected/5815b051-9c47-4898-9ddb-7cb51812f609-kube-api-access-dw8wg\") pod \"aws-ebs-csi-driver-node-8m2nw\" (UID: \"5815b051-9c47-4898-9ddb-7cb51812f609\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" Apr 22 21:09:21.726519 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.726475 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e4ebb4a-2ec6-41ad-ac25-a5d1d100711f-tmp-dir\") pod \"node-resolver-49dpb\" (UID: \"9e4ebb4a-2ec6-41ad-ac25-a5d1d100711f\") " pod="openshift-dns/node-resolver-49dpb" Apr 22 21:09:21.726519 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.726511 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-node-log\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.726651 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.726578 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-ovnkube-script-lib\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.726692 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.726678 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5815b051-9c47-4898-9ddb-7cb51812f609-registration-dir\") pod \"aws-ebs-csi-driver-node-8m2nw\" (UID: \"5815b051-9c47-4898-9ddb-7cb51812f609\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" Apr 22 21:09:21.726733 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.726709 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-run-systemd\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.726779 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.726765 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9e4ebb4a-2ec6-41ad-ac25-a5d1d100711f-hosts-file\") pod \"node-resolver-49dpb\" (UID: \"9e4ebb4a-2ec6-41ad-ac25-a5d1d100711f\") " pod="openshift-dns/node-resolver-49dpb" Apr 22 21:09:21.726832 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.726801 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/178e5bc4-7770-41a1-82b7-7cc3cca7ef1c-iptables-alerter-script\") pod \"iptables-alerter-z7k7s\" (UID: \"178e5bc4-7770-41a1-82b7-7cc3cca7ef1c\") " pod="openshift-network-operator/iptables-alerter-z7k7s" Apr 22 21:09:21.726909 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.726886 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-host-kubelet\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.727079 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.727048 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-systemd-units\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.727129 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.727109 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-host-run-ovn-kubernetes\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.727165 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.727155 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-env-overrides\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.727459 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.727428 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-run-openvswitch\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.727535 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.727469 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-host-cni-bin\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.727535 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.727513 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.727638 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.727556 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl4f4\" (UniqueName: \"kubernetes.io/projected/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-kube-api-access-wl4f4\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.727638 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.727587 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.727638 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.727594 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5815b051-9c47-4898-9ddb-7cb51812f609-etc-selinux\") pod \"aws-ebs-csi-driver-node-8m2nw\" (UID: \"5815b051-9c47-4898-9ddb-7cb51812f609\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" Apr 22 21:09:21.727775 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.727710 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-host-slash\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.727775 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.727719 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.727775 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.727743 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-var-lib-openvswitch\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.727950 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.727800 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-host-cni-netd\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.727950 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.727833 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/72b91440-69fd-4335-a6e6-e8f741ad2da8-konnectivity-ca\") pod \"konnectivity-agent-7xt7x\" (UID: \"72b91440-69fd-4335-a6e6-e8f741ad2da8\") " pod="kube-system/konnectivity-agent-7xt7x" Apr 22 21:09:21.728049 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.727950 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fffgd\" (UniqueName: \"kubernetes.io/projected/9e4ebb4a-2ec6-41ad-ac25-a5d1d100711f-kube-api-access-fffgd\") pod \"node-resolver-49dpb\" (UID: \"9e4ebb4a-2ec6-41ad-ac25-a5d1d100711f\") " pod="openshift-dns/node-resolver-49dpb" Apr 22 21:09:21.728049 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.727986 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c5lf\" (UniqueName: \"kubernetes.io/projected/178e5bc4-7770-41a1-82b7-7cc3cca7ef1c-kube-api-access-9c5lf\") pod \"iptables-alerter-z7k7s\" (UID: \"178e5bc4-7770-41a1-82b7-7cc3cca7ef1c\") " pod="openshift-network-operator/iptables-alerter-z7k7s" Apr 22 21:09:21.728049 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.728019 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/72b91440-69fd-4335-a6e6-e8f741ad2da8-agent-certs\") pod \"konnectivity-agent-7xt7x\" (UID: \"72b91440-69fd-4335-a6e6-e8f741ad2da8\") " pod="kube-system/konnectivity-agent-7xt7x" Apr 22 21:09:21.728184 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.728051 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5815b051-9c47-4898-9ddb-7cb51812f609-device-dir\") pod \"aws-ebs-csi-driver-node-8m2nw\" (UID: \"5815b051-9c47-4898-9ddb-7cb51812f609\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" Apr 22 21:09:21.728184 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.728104 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-ovn-node-metrics-cert\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.728184 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.728151 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/178e5bc4-7770-41a1-82b7-7cc3cca7ef1c-host-slash\") pod \"iptables-alerter-z7k7s\" (UID: \"178e5bc4-7770-41a1-82b7-7cc3cca7ef1c\") " pod="openshift-network-operator/iptables-alerter-z7k7s" Apr 22 21:09:21.728311 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.728207 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5815b051-9c47-4898-9ddb-7cb51812f609-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8m2nw\" (UID: \"5815b051-9c47-4898-9ddb-7cb51812f609\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" Apr 22 21:09:21.728311 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.728243 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5815b051-9c47-4898-9ddb-7cb51812f609-socket-dir\") pod \"aws-ebs-csi-driver-node-8m2nw\" (UID: \"5815b051-9c47-4898-9ddb-7cb51812f609\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" Apr 22 21:09:21.728381 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.728335 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5815b051-9c47-4898-9ddb-7cb51812f609-sys-fs\") pod \"aws-ebs-csi-driver-node-8m2nw\" (UID: \"5815b051-9c47-4898-9ddb-7cb51812f609\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" Apr 22 21:09:21.728437 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.728418 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-host-run-netns\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.728490 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.728474 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-etc-openvswitch\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.728532 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.728516 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-run-ovn\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.728580 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.728564 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-log-socket\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.728632 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.728619 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-ovnkube-config\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.730446 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.730424 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Apr 22 21:09:21.730598 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.730580 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Apr 22 21:09:21.730675 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.730648 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-29lzd\"" Apr 22 21:09:21.730900 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.730879 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-blwbc\"" Apr 22 21:09:21.731116 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.731092 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Apr 22 21:09:21.745606 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.745580 2580 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Apr 22 21:09:21.764149 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.764122 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-04-21 21:04:20 +0000 UTC" deadline="2027-09-16 04:41:53.199657568 +0000 UTC" Apr 22 21:09:21.764149 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.764148 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12271h32m31.435513259s" Apr 22 21:09:21.816594 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.816569 2580 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Apr 22 21:09:21.828966 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.828939 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5815b051-9c47-4898-9ddb-7cb51812f609-sys-fs\") pod \"aws-ebs-csi-driver-node-8m2nw\" (UID: \"5815b051-9c47-4898-9ddb-7cb51812f609\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" Apr 22 21:09:21.829071 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.828985 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcsdr\" (UniqueName: \"kubernetes.io/projected/843da596-cc50-4a3e-817c-4ee63a89c94a-kube-api-access-gcsdr\") pod \"network-metrics-daemon-fx8k2\" (UID: \"843da596-cc50-4a3e-817c-4ee63a89c94a\") " pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:21.829071 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.829006 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/13fe92d8-fa95-45da-8e47-5b754f77385c-host\") pod \"node-ca-wkk24\" (UID: \"13fe92d8-fa95-45da-8e47-5b754f77385c\") " pod="openshift-image-registry/node-ca-wkk24" Apr 22 21:09:21.829071 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.829027 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-etc-openvswitch\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.829071 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.829047 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5815b051-9c47-4898-9ddb-7cb51812f609-sys-fs\") pod \"aws-ebs-csi-driver-node-8m2nw\" (UID: \"5815b051-9c47-4898-9ddb-7cb51812f609\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" Apr 22 21:09:21.829071 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.829050 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-run-ovn\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.829306 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.829083 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-run-ovn\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.829306 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.829098 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-etc-kubernetes\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.829306 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.829099 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-etc-openvswitch\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.829306 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.829128 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-sys\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.829306 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.829163 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dw8wg\" (UniqueName: \"kubernetes.io/projected/5815b051-9c47-4898-9ddb-7cb51812f609-kube-api-access-dw8wg\") pod \"aws-ebs-csi-driver-node-8m2nw\" (UID: \"5815b051-9c47-4898-9ddb-7cb51812f609\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" Apr 22 21:09:21.829306 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.829245 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-os-release\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.829306 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.829299 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e4ebb4a-2ec6-41ad-ac25-a5d1d100711f-tmp-dir\") pod \"node-resolver-49dpb\" (UID: \"9e4ebb4a-2ec6-41ad-ac25-a5d1d100711f\") " pod="openshift-dns/node-resolver-49dpb" Apr 22 21:09:21.829655 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.829463 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-ovnkube-script-lib\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.829655 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.829560 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e4ebb4a-2ec6-41ad-ac25-a5d1d100711f-tmp-dir\") pod \"node-resolver-49dpb\" (UID: \"9e4ebb4a-2ec6-41ad-ac25-a5d1d100711f\") " pod="openshift-dns/node-resolver-49dpb" Apr 22 21:09:21.829655 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.829620 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/408567c2-c6f7-4edd-bea3-ff7695a4d06e-multus-daemon-config\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.829797 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.829663 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-run-systemd\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.829797 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.829697 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-run-systemd\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.829797 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.829714 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9e4ebb4a-2ec6-41ad-ac25-a5d1d100711f-hosts-file\") pod \"node-resolver-49dpb\" (UID: \"9e4ebb4a-2ec6-41ad-ac25-a5d1d100711f\") " pod="openshift-dns/node-resolver-49dpb" Apr 22 21:09:21.829797 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.829741 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/178e5bc4-7770-41a1-82b7-7cc3cca7ef1c-iptables-alerter-script\") pod \"iptables-alerter-z7k7s\" (UID: \"178e5bc4-7770-41a1-82b7-7cc3cca7ef1c\") " pod="openshift-network-operator/iptables-alerter-z7k7s" Apr 22 21:09:21.829797 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.829765 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-systemd-units\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.830106 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.830082 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-ovnkube-script-lib\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.830206 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.830166 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-systemd-units\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.830206 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.830194 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/178e5bc4-7770-41a1-82b7-7cc3cca7ef1c-iptables-alerter-script\") pod \"iptables-alerter-z7k7s\" (UID: \"178e5bc4-7770-41a1-82b7-7cc3cca7ef1c\") " pod="openshift-network-operator/iptables-alerter-z7k7s" Apr 22 21:09:21.830284 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.830206 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-env-overrides\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.830284 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.830254 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9e4ebb4a-2ec6-41ad-ac25-a5d1d100711f-hosts-file\") pod \"node-resolver-49dpb\" (UID: \"9e4ebb4a-2ec6-41ad-ac25-a5d1d100711f\") " pod="openshift-dns/node-resolver-49dpb" Apr 22 21:09:21.830284 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.830258 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/843da596-cc50-4a3e-817c-4ee63a89c94a-metrics-certs\") pod \"network-metrics-daemon-fx8k2\" (UID: \"843da596-cc50-4a3e-817c-4ee63a89c94a\") " pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:21.830426 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.830393 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-system-cni-dir\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.830477 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.830461 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-host-run-multus-certs\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.830532 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.830511 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-run-openvswitch\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.830577 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.830546 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-host-cni-bin\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.830577 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.830572 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5815b051-9c47-4898-9ddb-7cb51812f609-etc-selinux\") pod \"aws-ebs-csi-driver-node-8m2nw\" (UID: \"5815b051-9c47-4898-9ddb-7cb51812f609\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" Apr 22 21:09:21.830674 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.830597 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-cnibin\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.830674 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.830628 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-env-overrides\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.830772 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.830675 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-multus-socket-dir-parent\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.830772 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.830685 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5815b051-9c47-4898-9ddb-7cb51812f609-etc-selinux\") pod \"aws-ebs-csi-driver-node-8m2nw\" (UID: \"5815b051-9c47-4898-9ddb-7cb51812f609\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" Apr 22 21:09:21.830772 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.830703 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-etc-kubernetes\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.830772 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.830720 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-run-openvswitch\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.830772 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.830729 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/efc4b2c1-b3de-4aa6-abe3-c37e3517897f-system-cni-dir\") pod \"multus-additional-cni-plugins-qvlsp\" (UID: \"efc4b2c1-b3de-4aa6-abe3-c37e3517897f\") " pod="openshift-multus/multus-additional-cni-plugins-qvlsp" Apr 22 21:09:21.830772 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.830753 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-host-cni-bin\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.831092 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.830785 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/efc4b2c1-b3de-4aa6-abe3-c37e3517897f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qvlsp\" (UID: \"efc4b2c1-b3de-4aa6-abe3-c37e3517897f\") " pod="openshift-multus/multus-additional-cni-plugins-qvlsp" Apr 22 21:09:21.831092 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.830876 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-host-slash\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.831092 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.830922 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-var-lib-openvswitch\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.831092 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.830961 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-var-lib-openvswitch\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.831092 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.830979 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-host-slash\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.831092 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831004 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-host-cni-netd\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.831092 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831035 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-host-cni-netd\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.831092 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831048 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/72b91440-69fd-4335-a6e6-e8f741ad2da8-konnectivity-ca\") pod \"konnectivity-agent-7xt7x\" (UID: \"72b91440-69fd-4335-a6e6-e8f741ad2da8\") " pod="kube-system/konnectivity-agent-7xt7x" Apr 22 21:09:21.831092 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831090 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/efc4b2c1-b3de-4aa6-abe3-c37e3517897f-os-release\") pod \"multus-additional-cni-plugins-qvlsp\" (UID: \"efc4b2c1-b3de-4aa6-abe3-c37e3517897f\") " pod="openshift-multus/multus-additional-cni-plugins-qvlsp" Apr 22 21:09:21.831373 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831125 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-etc-modprobe-d\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.831373 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831150 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9c5lf\" (UniqueName: \"kubernetes.io/projected/178e5bc4-7770-41a1-82b7-7cc3cca7ef1c-kube-api-access-9c5lf\") pod \"iptables-alerter-z7k7s\" (UID: \"178e5bc4-7770-41a1-82b7-7cc3cca7ef1c\") " pod="openshift-network-operator/iptables-alerter-z7k7s" Apr 22 21:09:21.831373 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831174 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/13fe92d8-fa95-45da-8e47-5b754f77385c-serviceca\") pod \"node-ca-wkk24\" (UID: \"13fe92d8-fa95-45da-8e47-5b754f77385c\") " pod="openshift-image-registry/node-ca-wkk24" Apr 22 21:09:21.831373 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831295 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hffwj\" (UniqueName: \"kubernetes.io/projected/516cb834-5ee4-4971-80a9-149fd935fc80-kube-api-access-hffwj\") pod \"network-check-target-lp7mc\" (UID: \"516cb834-5ee4-4971-80a9-149fd935fc80\") " pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:09:21.831373 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831332 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-hostroot\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.831373 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831362 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/efc4b2c1-b3de-4aa6-abe3-c37e3517897f-cnibin\") pod \"multus-additional-cni-plugins-qvlsp\" (UID: \"efc4b2c1-b3de-4aa6-abe3-c37e3517897f\") " pod="openshift-multus/multus-additional-cni-plugins-qvlsp" Apr 22 21:09:21.831542 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831414 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-etc-sysconfig\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.831542 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831461 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-ovn-node-metrics-cert\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.831542 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831476 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/178e5bc4-7770-41a1-82b7-7cc3cca7ef1c-host-slash\") pod \"iptables-alerter-z7k7s\" (UID: \"178e5bc4-7770-41a1-82b7-7cc3cca7ef1c\") " pod="openshift-network-operator/iptables-alerter-z7k7s" Apr 22 21:09:21.831542 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831490 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/72b91440-69fd-4335-a6e6-e8f741ad2da8-konnectivity-ca\") pod \"konnectivity-agent-7xt7x\" (UID: \"72b91440-69fd-4335-a6e6-e8f741ad2da8\") " pod="kube-system/konnectivity-agent-7xt7x" Apr 22 21:09:21.831542 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831506 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5815b051-9c47-4898-9ddb-7cb51812f609-socket-dir\") pod \"aws-ebs-csi-driver-node-8m2nw\" (UID: \"5815b051-9c47-4898-9ddb-7cb51812f609\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" Apr 22 21:09:21.831542 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831517 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/178e5bc4-7770-41a1-82b7-7cc3cca7ef1c-host-slash\") pod \"iptables-alerter-z7k7s\" (UID: \"178e5bc4-7770-41a1-82b7-7cc3cca7ef1c\") " pod="openshift-network-operator/iptables-alerter-z7k7s" Apr 22 21:09:21.831763 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831596 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n5bf\" (UniqueName: \"kubernetes.io/projected/408567c2-c6f7-4edd-bea3-ff7695a4d06e-kube-api-access-4n5bf\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.831763 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831657 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-host-run-netns\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.831763 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831682 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5815b051-9c47-4898-9ddb-7cb51812f609-socket-dir\") pod \"aws-ebs-csi-driver-node-8m2nw\" (UID: \"5815b051-9c47-4898-9ddb-7cb51812f609\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" Apr 22 21:09:21.831763 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831697 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-log-socket\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.831763 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831727 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-host-run-netns\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.831763 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831732 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-multus-cni-dir\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.831763 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831760 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-log-socket\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.832057 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831776 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/408567c2-c6f7-4edd-bea3-ff7695a4d06e-cni-binary-copy\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.832057 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831800 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-host-var-lib-cni-bin\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.832057 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831846 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-host-var-lib-cni-multus\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.832057 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831907 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-host-var-lib-kubelet\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.832057 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831922 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Apr 22 21:09:21.832057 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831938 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/efc4b2c1-b3de-4aa6-abe3-c37e3517897f-cni-binary-copy\") pod \"multus-additional-cni-plugins-qvlsp\" (UID: \"efc4b2c1-b3de-4aa6-abe3-c37e3517897f\") " pod="openshift-multus/multus-additional-cni-plugins-qvlsp" Apr 22 21:09:21.832057 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.831986 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-ovnkube-config\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.832057 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832017 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-host-run-k8s-cni-cncf-io\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.832057 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832043 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-run\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.832401 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832069 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-lib-modules\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.832401 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832096 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-var-lib-kubelet\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.832401 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832121 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/be33f8b1-265a-4e31-9b06-742c72bf73e3-etc-tuned\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.832401 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832150 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rss7\" (UniqueName: \"kubernetes.io/projected/be33f8b1-265a-4e31-9b06-742c72bf73e3-kube-api-access-8rss7\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.832401 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832176 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-node-log\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.832401 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832228 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5815b051-9c47-4898-9ddb-7cb51812f609-registration-dir\") pod \"aws-ebs-csi-driver-node-8m2nw\" (UID: \"5815b051-9c47-4898-9ddb-7cb51812f609\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" Apr 22 21:09:21.832401 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832289 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-node-log\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.832401 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832300 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5815b051-9c47-4898-9ddb-7cb51812f609-registration-dir\") pod \"aws-ebs-csi-driver-node-8m2nw\" (UID: \"5815b051-9c47-4898-9ddb-7cb51812f609\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" Apr 22 21:09:21.832401 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832327 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6h7x\" (UniqueName: \"kubernetes.io/projected/13fe92d8-fa95-45da-8e47-5b754f77385c-kube-api-access-w6h7x\") pod \"node-ca-wkk24\" (UID: \"13fe92d8-fa95-45da-8e47-5b754f77385c\") " pod="openshift-image-registry/node-ca-wkk24" Apr 22 21:09:21.832401 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832351 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-host-run-netns\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.832401 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832387 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-multus-conf-dir\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.832401 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832406 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-etc-sysctl-d\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.832975 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832431 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-host-kubelet\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.832975 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832445 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-ovnkube-config\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.832975 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832456 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-host-run-ovn-kubernetes\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.832975 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832480 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/efc4b2c1-b3de-4aa6-abe3-c37e3517897f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qvlsp\" (UID: \"efc4b2c1-b3de-4aa6-abe3-c37e3517897f\") " pod="openshift-multus/multus-additional-cni-plugins-qvlsp" Apr 22 21:09:21.832975 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832500 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-etc-sysctl-conf\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.832975 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832513 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-host-run-ovn-kubernetes\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.832975 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832495 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-host-kubelet\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.832975 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832543 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.832975 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832614 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wl4f4\" (UniqueName: \"kubernetes.io/projected/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-kube-api-access-wl4f4\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.832975 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832578 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.832975 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832645 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/be33f8b1-265a-4e31-9b06-742c72bf73e3-tmp\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.832975 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832671 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-host\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.832975 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832697 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fffgd\" (UniqueName: \"kubernetes.io/projected/9e4ebb4a-2ec6-41ad-ac25-a5d1d100711f-kube-api-access-fffgd\") pod \"node-resolver-49dpb\" (UID: \"9e4ebb4a-2ec6-41ad-ac25-a5d1d100711f\") " pod="openshift-dns/node-resolver-49dpb" Apr 22 21:09:21.832975 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832722 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/72b91440-69fd-4335-a6e6-e8f741ad2da8-agent-certs\") pod \"konnectivity-agent-7xt7x\" (UID: \"72b91440-69fd-4335-a6e6-e8f741ad2da8\") " pod="kube-system/konnectivity-agent-7xt7x" Apr 22 21:09:21.832975 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832746 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5815b051-9c47-4898-9ddb-7cb51812f609-device-dir\") pod \"aws-ebs-csi-driver-node-8m2nw\" (UID: \"5815b051-9c47-4898-9ddb-7cb51812f609\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" Apr 22 21:09:21.832975 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832773 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/efc4b2c1-b3de-4aa6-abe3-c37e3517897f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qvlsp\" (UID: \"efc4b2c1-b3de-4aa6-abe3-c37e3517897f\") " pod="openshift-multus/multus-additional-cni-plugins-qvlsp" Apr 22 21:09:21.833699 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832800 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qbc5\" (UniqueName: \"kubernetes.io/projected/efc4b2c1-b3de-4aa6-abe3-c37e3517897f-kube-api-access-7qbc5\") pod \"multus-additional-cni-plugins-qvlsp\" (UID: \"efc4b2c1-b3de-4aa6-abe3-c37e3517897f\") " pod="openshift-multus/multus-additional-cni-plugins-qvlsp" Apr 22 21:09:21.833699 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832824 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-etc-systemd\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.833699 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832868 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5815b051-9c47-4898-9ddb-7cb51812f609-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8m2nw\" (UID: \"5815b051-9c47-4898-9ddb-7cb51812f609\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" Apr 22 21:09:21.833699 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.832959 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5815b051-9c47-4898-9ddb-7cb51812f609-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8m2nw\" (UID: \"5815b051-9c47-4898-9ddb-7cb51812f609\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" Apr 22 21:09:21.833699 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.833011 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5815b051-9c47-4898-9ddb-7cb51812f609-device-dir\") pod \"aws-ebs-csi-driver-node-8m2nw\" (UID: \"5815b051-9c47-4898-9ddb-7cb51812f609\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" Apr 22 21:09:21.836690 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.836657 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/72b91440-69fd-4335-a6e6-e8f741ad2da8-agent-certs\") pod \"konnectivity-agent-7xt7x\" (UID: \"72b91440-69fd-4335-a6e6-e8f741ad2da8\") " pod="kube-system/konnectivity-agent-7xt7x" Apr 22 21:09:21.838219 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.838194 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw8wg\" (UniqueName: \"kubernetes.io/projected/5815b051-9c47-4898-9ddb-7cb51812f609-kube-api-access-dw8wg\") pod \"aws-ebs-csi-driver-node-8m2nw\" (UID: \"5815b051-9c47-4898-9ddb-7cb51812f609\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" Apr 22 21:09:21.839559 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.839538 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c5lf\" (UniqueName: \"kubernetes.io/projected/178e5bc4-7770-41a1-82b7-7cc3cca7ef1c-kube-api-access-9c5lf\") pod \"iptables-alerter-z7k7s\" (UID: \"178e5bc4-7770-41a1-82b7-7cc3cca7ef1c\") " pod="openshift-network-operator/iptables-alerter-z7k7s" Apr 22 21:09:21.839696 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.839553 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-ovn-node-metrics-cert\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.840410 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.840390 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl4f4\" (UniqueName: \"kubernetes.io/projected/b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b-kube-api-access-wl4f4\") pod \"ovnkube-node-8ngls\" (UID: \"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:21.843659 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.843631 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fffgd\" (UniqueName: \"kubernetes.io/projected/9e4ebb4a-2ec6-41ad-ac25-a5d1d100711f-kube-api-access-fffgd\") pod \"node-resolver-49dpb\" (UID: \"9e4ebb4a-2ec6-41ad-ac25-a5d1d100711f\") " pod="openshift-dns/node-resolver-49dpb" Apr 22 21:09:21.933963 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.933935 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-system-cni-dir\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.933963 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.933968 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-host-run-multus-certs\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.934209 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.933993 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-cnibin\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.934209 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934017 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-multus-socket-dir-parent\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.934209 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934042 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-etc-kubernetes\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.934209 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934066 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/efc4b2c1-b3de-4aa6-abe3-c37e3517897f-system-cni-dir\") pod \"multus-additional-cni-plugins-qvlsp\" (UID: \"efc4b2c1-b3de-4aa6-abe3-c37e3517897f\") " pod="openshift-multus/multus-additional-cni-plugins-qvlsp" Apr 22 21:09:21.934209 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934064 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-host-run-multus-certs\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.934209 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934083 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-system-cni-dir\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.934209 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934081 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-cnibin\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.934209 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934109 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/efc4b2c1-b3de-4aa6-abe3-c37e3517897f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qvlsp\" (UID: \"efc4b2c1-b3de-4aa6-abe3-c37e3517897f\") " pod="openshift-multus/multus-additional-cni-plugins-qvlsp" Apr 22 21:09:21.934209 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934122 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-etc-kubernetes\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.934209 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934123 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-multus-socket-dir-parent\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.934209 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934139 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/efc4b2c1-b3de-4aa6-abe3-c37e3517897f-os-release\") pod \"multus-additional-cni-plugins-qvlsp\" (UID: \"efc4b2c1-b3de-4aa6-abe3-c37e3517897f\") " pod="openshift-multus/multus-additional-cni-plugins-qvlsp" Apr 22 21:09:21.934209 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934111 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/efc4b2c1-b3de-4aa6-abe3-c37e3517897f-system-cni-dir\") pod \"multus-additional-cni-plugins-qvlsp\" (UID: \"efc4b2c1-b3de-4aa6-abe3-c37e3517897f\") " pod="openshift-multus/multus-additional-cni-plugins-qvlsp" Apr 22 21:09:21.934209 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934181 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-etc-modprobe-d\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.934209 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934195 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/efc4b2c1-b3de-4aa6-abe3-c37e3517897f-os-release\") pod \"multus-additional-cni-plugins-qvlsp\" (UID: \"efc4b2c1-b3de-4aa6-abe3-c37e3517897f\") " pod="openshift-multus/multus-additional-cni-plugins-qvlsp" Apr 22 21:09:21.934209 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934210 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/13fe92d8-fa95-45da-8e47-5b754f77385c-serviceca\") pod \"node-ca-wkk24\" (UID: \"13fe92d8-fa95-45da-8e47-5b754f77385c\") " pod="openshift-image-registry/node-ca-wkk24" Apr 22 21:09:21.934875 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934238 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hffwj\" (UniqueName: \"kubernetes.io/projected/516cb834-5ee4-4971-80a9-149fd935fc80-kube-api-access-hffwj\") pod \"network-check-target-lp7mc\" (UID: \"516cb834-5ee4-4971-80a9-149fd935fc80\") " pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:09:21.934875 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934262 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-hostroot\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.934875 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934306 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-hostroot\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.934875 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934315 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-etc-modprobe-d\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.934875 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934331 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/efc4b2c1-b3de-4aa6-abe3-c37e3517897f-cnibin\") pod \"multus-additional-cni-plugins-qvlsp\" (UID: \"efc4b2c1-b3de-4aa6-abe3-c37e3517897f\") " pod="openshift-multus/multus-additional-cni-plugins-qvlsp" Apr 22 21:09:21.934875 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934355 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-etc-sysconfig\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.934875 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934380 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4n5bf\" (UniqueName: \"kubernetes.io/projected/408567c2-c6f7-4edd-bea3-ff7695a4d06e-kube-api-access-4n5bf\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.934875 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934404 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-multus-cni-dir\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.934875 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934427 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/408567c2-c6f7-4edd-bea3-ff7695a4d06e-cni-binary-copy\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.934875 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934437 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-etc-sysconfig\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.934875 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934451 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-host-var-lib-cni-bin\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.934875 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934475 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-host-var-lib-cni-multus\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.934875 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934501 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-host-var-lib-kubelet\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.934875 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934520 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-multus-cni-dir\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.934875 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934526 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/efc4b2c1-b3de-4aa6-abe3-c37e3517897f-cni-binary-copy\") pod \"multus-additional-cni-plugins-qvlsp\" (UID: \"efc4b2c1-b3de-4aa6-abe3-c37e3517897f\") " pod="openshift-multus/multus-additional-cni-plugins-qvlsp" Apr 22 21:09:21.934875 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934560 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-host-run-k8s-cni-cncf-io\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.934875 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934600 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-host-var-lib-cni-multus\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.934875 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934604 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-run\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.935866 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934641 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-run\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.935866 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934649 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/13fe92d8-fa95-45da-8e47-5b754f77385c-serviceca\") pod \"node-ca-wkk24\" (UID: \"13fe92d8-fa95-45da-8e47-5b754f77385c\") " pod="openshift-image-registry/node-ca-wkk24" Apr 22 21:09:21.935866 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934654 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-lib-modules\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.935866 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934680 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/efc4b2c1-b3de-4aa6-abe3-c37e3517897f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qvlsp\" (UID: \"efc4b2c1-b3de-4aa6-abe3-c37e3517897f\") " pod="openshift-multus/multus-additional-cni-plugins-qvlsp" Apr 22 21:09:21.935866 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934704 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-var-lib-kubelet\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.935866 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934728 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-host-var-lib-cni-bin\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.935866 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934731 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/be33f8b1-265a-4e31-9b06-742c72bf73e3-etc-tuned\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.935866 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934476 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/efc4b2c1-b3de-4aa6-abe3-c37e3517897f-cnibin\") pod \"multus-additional-cni-plugins-qvlsp\" (UID: \"efc4b2c1-b3de-4aa6-abe3-c37e3517897f\") " pod="openshift-multus/multus-additional-cni-plugins-qvlsp" Apr 22 21:09:21.935866 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934741 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-host-var-lib-kubelet\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.935866 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934704 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-host-run-k8s-cni-cncf-io\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.935866 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934757 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8rss7\" (UniqueName: \"kubernetes.io/projected/be33f8b1-265a-4e31-9b06-742c72bf73e3-kube-api-access-8rss7\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.935866 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934780 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-var-lib-kubelet\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.935866 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934800 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w6h7x\" (UniqueName: \"kubernetes.io/projected/13fe92d8-fa95-45da-8e47-5b754f77385c-kube-api-access-w6h7x\") pod \"node-ca-wkk24\" (UID: \"13fe92d8-fa95-45da-8e47-5b754f77385c\") " pod="openshift-image-registry/node-ca-wkk24" Apr 22 21:09:21.935866 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934825 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-host-run-netns\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.935866 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934870 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-multus-conf-dir\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.935866 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934894 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-etc-sysctl-d\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.935866 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934901 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-lib-modules\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.935866 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934921 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/efc4b2c1-b3de-4aa6-abe3-c37e3517897f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qvlsp\" (UID: \"efc4b2c1-b3de-4aa6-abe3-c37e3517897f\") " pod="openshift-multus/multus-additional-cni-plugins-qvlsp" Apr 22 21:09:21.936669 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934963 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-etc-sysctl-conf\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.936669 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934976 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/efc4b2c1-b3de-4aa6-abe3-c37e3517897f-cni-binary-copy\") pod \"multus-additional-cni-plugins-qvlsp\" (UID: \"efc4b2c1-b3de-4aa6-abe3-c37e3517897f\") " pod="openshift-multus/multus-additional-cni-plugins-qvlsp" Apr 22 21:09:21.936669 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.934990 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/be33f8b1-265a-4e31-9b06-742c72bf73e3-tmp\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.936669 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.935024 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-host\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.936669 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.935054 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/efc4b2c1-b3de-4aa6-abe3-c37e3517897f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qvlsp\" (UID: \"efc4b2c1-b3de-4aa6-abe3-c37e3517897f\") " pod="openshift-multus/multus-additional-cni-plugins-qvlsp" Apr 22 21:09:21.936669 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.935080 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7qbc5\" (UniqueName: \"kubernetes.io/projected/efc4b2c1-b3de-4aa6-abe3-c37e3517897f-kube-api-access-7qbc5\") pod \"multus-additional-cni-plugins-qvlsp\" (UID: \"efc4b2c1-b3de-4aa6-abe3-c37e3517897f\") " pod="openshift-multus/multus-additional-cni-plugins-qvlsp" Apr 22 21:09:21.936669 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.935106 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-etc-systemd\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.936669 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.935132 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gcsdr\" (UniqueName: \"kubernetes.io/projected/843da596-cc50-4a3e-817c-4ee63a89c94a-kube-api-access-gcsdr\") pod \"network-metrics-daemon-fx8k2\" (UID: \"843da596-cc50-4a3e-817c-4ee63a89c94a\") " pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:21.936669 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.935158 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/13fe92d8-fa95-45da-8e47-5b754f77385c-host\") pod \"node-ca-wkk24\" (UID: \"13fe92d8-fa95-45da-8e47-5b754f77385c\") " pod="openshift-image-registry/node-ca-wkk24" Apr 22 21:09:21.936669 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.935186 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-etc-kubernetes\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.936669 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.935209 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-sys\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.936669 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.935235 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-os-release\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.936669 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.935260 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/408567c2-c6f7-4edd-bea3-ff7695a4d06e-multus-daemon-config\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.936669 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.935291 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/843da596-cc50-4a3e-817c-4ee63a89c94a-metrics-certs\") pod \"network-metrics-daemon-fx8k2\" (UID: \"843da596-cc50-4a3e-817c-4ee63a89c94a\") " pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:21.936669 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.935357 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-etc-sysctl-d\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.936669 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.935369 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/13fe92d8-fa95-45da-8e47-5b754f77385c-host\") pod \"node-ca-wkk24\" (UID: \"13fe92d8-fa95-45da-8e47-5b754f77385c\") " pod="openshift-image-registry/node-ca-wkk24" Apr 22 21:09:21.936669 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.935383 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-etc-kubernetes\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.936669 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.935414 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-multus-conf-dir\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.937478 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.935421 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-host-run-netns\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.937478 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:21.935454 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 21:09:21.937478 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.935462 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-sys\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.937478 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.935501 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-host\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.937478 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.935186 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/408567c2-c6f7-4edd-bea3-ff7695a4d06e-cni-binary-copy\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.937478 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:21.935513 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/843da596-cc50-4a3e-817c-4ee63a89c94a-metrics-certs podName:843da596-cc50-4a3e-817c-4ee63a89c94a nodeName:}" failed. No retries permitted until 2026-04-22 21:09:22.435497807 +0000 UTC m=+3.161394385 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/843da596-cc50-4a3e-817c-4ee63a89c94a-metrics-certs") pod "network-metrics-daemon-fx8k2" (UID: "843da596-cc50-4a3e-817c-4ee63a89c94a") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 21:09:21.937478 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.935546 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/efc4b2c1-b3de-4aa6-abe3-c37e3517897f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qvlsp\" (UID: \"efc4b2c1-b3de-4aa6-abe3-c37e3517897f\") " pod="openshift-multus/multus-additional-cni-plugins-qvlsp" Apr 22 21:09:21.937478 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.935713 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-etc-systemd\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.937478 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.935785 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/408567c2-c6f7-4edd-bea3-ff7695a4d06e-os-release\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.937478 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.935913 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/be33f8b1-265a-4e31-9b06-742c72bf73e3-etc-sysctl-conf\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.937478 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.935992 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/408567c2-c6f7-4edd-bea3-ff7695a4d06e-multus-daemon-config\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:21.937478 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.936109 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/efc4b2c1-b3de-4aa6-abe3-c37e3517897f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qvlsp\" (UID: \"efc4b2c1-b3de-4aa6-abe3-c37e3517897f\") " pod="openshift-multus/multus-additional-cni-plugins-qvlsp" Apr 22 21:09:21.938047 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.937582 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/be33f8b1-265a-4e31-9b06-742c72bf73e3-tmp\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.938047 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.937715 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/be33f8b1-265a-4e31-9b06-742c72bf73e3-etc-tuned\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.940283 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:21.940261 2580 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 22 21:09:21.940381 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:21.940285 2580 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 22 21:09:21.940381 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:21.940302 2580 projected.go:194] Error preparing data for projected volume kube-api-access-hffwj for pod openshift-network-diagnostics/network-check-target-lp7mc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 22 21:09:21.940381 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:21.940374 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/516cb834-5ee4-4971-80a9-149fd935fc80-kube-api-access-hffwj podName:516cb834-5ee4-4971-80a9-149fd935fc80 nodeName:}" failed. No retries permitted until 2026-04-22 21:09:22.440359561 +0000 UTC m=+3.166256139 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hffwj" (UniqueName: "kubernetes.io/projected/516cb834-5ee4-4971-80a9-149fd935fc80-kube-api-access-hffwj") pod "network-check-target-lp7mc" (UID: "516cb834-5ee4-4971-80a9-149fd935fc80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 22 21:09:21.943867 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.943829 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6h7x\" (UniqueName: \"kubernetes.io/projected/13fe92d8-fa95-45da-8e47-5b754f77385c-kube-api-access-w6h7x\") pod \"node-ca-wkk24\" (UID: \"13fe92d8-fa95-45da-8e47-5b754f77385c\") " pod="openshift-image-registry/node-ca-wkk24" Apr 22 21:09:21.944081 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.944061 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rss7\" (UniqueName: \"kubernetes.io/projected/be33f8b1-265a-4e31-9b06-742c72bf73e3-kube-api-access-8rss7\") pod \"tuned-n66xm\" (UID: \"be33f8b1-265a-4e31-9b06-742c72bf73e3\") " pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:21.944322 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.944299 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qbc5\" (UniqueName: \"kubernetes.io/projected/efc4b2c1-b3de-4aa6-abe3-c37e3517897f-kube-api-access-7qbc5\") pod \"multus-additional-cni-plugins-qvlsp\" (UID: \"efc4b2c1-b3de-4aa6-abe3-c37e3517897f\") " pod="openshift-multus/multus-additional-cni-plugins-qvlsp" Apr 22 21:09:21.944610 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.944589 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcsdr\" (UniqueName: \"kubernetes.io/projected/843da596-cc50-4a3e-817c-4ee63a89c94a-kube-api-access-gcsdr\") pod \"network-metrics-daemon-fx8k2\" (UID: \"843da596-cc50-4a3e-817c-4ee63a89c94a\") " pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:21.944720 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:21.944609 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n5bf\" (UniqueName: \"kubernetes.io/projected/408567c2-c6f7-4edd-bea3-ff7695a4d06e-kube-api-access-4n5bf\") pod \"multus-zwlx7\" (UID: \"408567c2-c6f7-4edd-bea3-ff7695a4d06e\") " pod="openshift-multus/multus-zwlx7" Apr 22 21:09:22.008017 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:22.007989 2580 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Apr 22 21:09:22.025330 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:22.025304 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-49dpb" Apr 22 21:09:22.035012 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:22.034989 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z7k7s" Apr 22 21:09:22.042680 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:22.042649 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:22.048292 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:22.048275 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7xt7x" Apr 22 21:09:22.058911 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:22.058895 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" Apr 22 21:09:22.068400 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:22.068382 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wkk24" Apr 22 21:09:22.072992 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:22.072977 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qvlsp" Apr 22 21:09:22.079562 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:22.079546 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zwlx7" Apr 22 21:09:22.084171 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:22.084153 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-n66xm" Apr 22 21:09:22.437516 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:22.437480 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/843da596-cc50-4a3e-817c-4ee63a89c94a-metrics-certs\") pod \"network-metrics-daemon-fx8k2\" (UID: \"843da596-cc50-4a3e-817c-4ee63a89c94a\") " pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:22.437711 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:22.437642 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 21:09:22.437711 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:22.437696 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/843da596-cc50-4a3e-817c-4ee63a89c94a-metrics-certs podName:843da596-cc50-4a3e-817c-4ee63a89c94a nodeName:}" failed. No retries permitted until 2026-04-22 21:09:23.437680895 +0000 UTC m=+4.163577473 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/843da596-cc50-4a3e-817c-4ee63a89c94a-metrics-certs") pod "network-metrics-daemon-fx8k2" (UID: "843da596-cc50-4a3e-817c-4ee63a89c94a") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 21:09:22.519931 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:22.519895 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72b91440_69fd_4335_a6e6_e8f741ad2da8.slice/crio-10f76b311fb49dda25d58822893acff66e15cd96f6759a8bc195dc2338c7a28a WatchSource:0}: Error finding container 10f76b311fb49dda25d58822893acff66e15cd96f6759a8bc195dc2338c7a28a: Status 404 returned error can't find the container with id 10f76b311fb49dda25d58822893acff66e15cd96f6759a8bc195dc2338c7a28a Apr 22 21:09:22.522700 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:22.522678 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod408567c2_c6f7_4edd_bea3_ff7695a4d06e.slice/crio-1a541a921a1916f2c53b0853076e2ccf4c5f9b16062c1e32b4482075e75c322f WatchSource:0}: Error finding container 1a541a921a1916f2c53b0853076e2ccf4c5f9b16062c1e32b4482075e75c322f: Status 404 returned error can't find the container with id 1a541a921a1916f2c53b0853076e2ccf4c5f9b16062c1e32b4482075e75c322f Apr 22 21:09:22.524967 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:22.524940 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod178e5bc4_7770_41a1_82b7_7cc3cca7ef1c.slice/crio-b6ebe332547fb1959efc6d0eed9771510f02667eaa7a7d6d1cd6345effa0bf7b WatchSource:0}: Error finding container b6ebe332547fb1959efc6d0eed9771510f02667eaa7a7d6d1cd6345effa0bf7b: Status 404 returned error can't find the container with id b6ebe332547fb1959efc6d0eed9771510f02667eaa7a7d6d1cd6345effa0bf7b Apr 22 21:09:22.525534 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:22.525515 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb619ae54_70a8_4b9f_b5ff_4008c1fdbc9b.slice/crio-f690b4c4332d4397b1ff03b6675b0692bb5c68757f2d076f5842cb9789b3c073 WatchSource:0}: Error finding container f690b4c4332d4397b1ff03b6675b0692bb5c68757f2d076f5842cb9789b3c073: Status 404 returned error can't find the container with id f690b4c4332d4397b1ff03b6675b0692bb5c68757f2d076f5842cb9789b3c073 Apr 22 21:09:22.526631 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:22.526613 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13fe92d8_fa95_45da_8e47_5b754f77385c.slice/crio-9b7532156fd324fc3a9e16d35165c38e8a7886851d8205545bff654d9cccf5e3 WatchSource:0}: Error finding container 9b7532156fd324fc3a9e16d35165c38e8a7886851d8205545bff654d9cccf5e3: Status 404 returned error can't find the container with id 9b7532156fd324fc3a9e16d35165c38e8a7886851d8205545bff654d9cccf5e3 Apr 22 21:09:22.527946 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:22.527929 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e4ebb4a_2ec6_41ad_ac25_a5d1d100711f.slice/crio-8e2ade3e14bc597991a3908862dc354d52dc99bce82faa275017947a3cb990c7 WatchSource:0}: Error finding container 8e2ade3e14bc597991a3908862dc354d52dc99bce82faa275017947a3cb990c7: Status 404 returned error can't find the container with id 8e2ade3e14bc597991a3908862dc354d52dc99bce82faa275017947a3cb990c7 Apr 22 21:09:22.528674 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:22.528652 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5815b051_9c47_4898_9ddb_7cb51812f609.slice/crio-3fdd180adfb9c5513f358eb9b74955d01333e82f77a422ce155b135bbb7ea40a WatchSource:0}: Error finding container 3fdd180adfb9c5513f358eb9b74955d01333e82f77a422ce155b135bbb7ea40a: Status 404 returned error can't find the container with id 3fdd180adfb9c5513f358eb9b74955d01333e82f77a422ce155b135bbb7ea40a Apr 22 21:09:22.529307 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:22.529036 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefc4b2c1_b3de_4aa6_abe3_c37e3517897f.slice/crio-746e9af1226f6b52302850a15efd1cdbe2ac8833e3cdfc122499e9a6272502a4 WatchSource:0}: Error finding container 746e9af1226f6b52302850a15efd1cdbe2ac8833e3cdfc122499e9a6272502a4: Status 404 returned error can't find the container with id 746e9af1226f6b52302850a15efd1cdbe2ac8833e3cdfc122499e9a6272502a4 Apr 22 21:09:22.530554 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:09:22.530532 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe33f8b1_265a_4e31_9b06_742c72bf73e3.slice/crio-ecc45d6e0e210962702365fb31601a0d76f1758d0de9bdf67d7c54a440a744a4 WatchSource:0}: Error finding container ecc45d6e0e210962702365fb31601a0d76f1758d0de9bdf67d7c54a440a744a4: Status 404 returned error can't find the container with id ecc45d6e0e210962702365fb31601a0d76f1758d0de9bdf67d7c54a440a744a4 Apr 22 21:09:22.538621 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:22.538599 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hffwj\" (UniqueName: \"kubernetes.io/projected/516cb834-5ee4-4971-80a9-149fd935fc80-kube-api-access-hffwj\") pod \"network-check-target-lp7mc\" (UID: \"516cb834-5ee4-4971-80a9-149fd935fc80\") " pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:09:22.538755 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:22.538738 2580 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 22 21:09:22.538798 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:22.538760 2580 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 22 21:09:22.538798 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:22.538773 2580 projected.go:194] Error preparing data for projected volume kube-api-access-hffwj for pod openshift-network-diagnostics/network-check-target-lp7mc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 22 21:09:22.538884 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:22.538827 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/516cb834-5ee4-4971-80a9-149fd935fc80-kube-api-access-hffwj podName:516cb834-5ee4-4971-80a9-149fd935fc80 nodeName:}" failed. No retries permitted until 2026-04-22 21:09:23.538809312 +0000 UTC m=+4.264705890 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hffwj" (UniqueName: "kubernetes.io/projected/516cb834-5ee4-4971-80a9-149fd935fc80-kube-api-access-hffwj") pod "network-check-target-lp7mc" (UID: "516cb834-5ee4-4971-80a9-149fd935fc80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 22 21:09:22.764338 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:22.764264 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-04-21 21:04:20 +0000 UTC" deadline="2027-11-09 07:11:40.803012643 +0000 UTC" Apr 22 21:09:22.764338 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:22.764295 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13570h2m18.038720561s" Apr 22 21:09:22.874088 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:22.874046 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qvlsp" event={"ID":"efc4b2c1-b3de-4aa6-abe3-c37e3517897f","Type":"ContainerStarted","Data":"746e9af1226f6b52302850a15efd1cdbe2ac8833e3cdfc122499e9a6272502a4"} Apr 22 21:09:22.882059 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:22.881976 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-49dpb" event={"ID":"9e4ebb4a-2ec6-41ad-ac25-a5d1d100711f","Type":"ContainerStarted","Data":"8e2ade3e14bc597991a3908862dc354d52dc99bce82faa275017947a3cb990c7"} Apr 22 21:09:22.883591 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:22.883565 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wkk24" event={"ID":"13fe92d8-fa95-45da-8e47-5b754f77385c","Type":"ContainerStarted","Data":"9b7532156fd324fc3a9e16d35165c38e8a7886851d8205545bff654d9cccf5e3"} Apr 22 21:09:22.885750 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:22.885687 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" event={"ID":"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b","Type":"ContainerStarted","Data":"f690b4c4332d4397b1ff03b6675b0692bb5c68757f2d076f5842cb9789b3c073"} Apr 22 21:09:22.891306 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:22.891279 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zwlx7" event={"ID":"408567c2-c6f7-4edd-bea3-ff7695a4d06e","Type":"ContainerStarted","Data":"1a541a921a1916f2c53b0853076e2ccf4c5f9b16062c1e32b4482075e75c322f"} Apr 22 21:09:22.898247 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:22.898178 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-20.ec2.internal" event={"ID":"7702f8c6e9277cd7f5964f802e4caebb","Type":"ContainerStarted","Data":"718ea4e56cb7d17f08a658ca21a562be5d76a47683e74f35191c0f116a2555ca"} Apr 22 21:09:22.906081 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:22.906003 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-n66xm" event={"ID":"be33f8b1-265a-4e31-9b06-742c72bf73e3","Type":"ContainerStarted","Data":"ecc45d6e0e210962702365fb31601a0d76f1758d0de9bdf67d7c54a440a744a4"} Apr 22 21:09:22.910333 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:22.910306 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" event={"ID":"5815b051-9c47-4898-9ddb-7cb51812f609","Type":"ContainerStarted","Data":"3fdd180adfb9c5513f358eb9b74955d01333e82f77a422ce155b135bbb7ea40a"} Apr 22 21:09:22.921793 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:22.921755 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z7k7s" event={"ID":"178e5bc4-7770-41a1-82b7-7cc3cca7ef1c","Type":"ContainerStarted","Data":"b6ebe332547fb1959efc6d0eed9771510f02667eaa7a7d6d1cd6345effa0bf7b"} Apr 22 21:09:22.954906 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:22.954833 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7xt7x" event={"ID":"72b91440-69fd-4335-a6e6-e8f741ad2da8","Type":"ContainerStarted","Data":"10f76b311fb49dda25d58822893acff66e15cd96f6759a8bc195dc2338c7a28a"} Apr 22 21:09:23.447228 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:23.446638 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/843da596-cc50-4a3e-817c-4ee63a89c94a-metrics-certs\") pod \"network-metrics-daemon-fx8k2\" (UID: \"843da596-cc50-4a3e-817c-4ee63a89c94a\") " pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:23.447228 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:23.446795 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 21:09:23.447228 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:23.446877 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/843da596-cc50-4a3e-817c-4ee63a89c94a-metrics-certs podName:843da596-cc50-4a3e-817c-4ee63a89c94a nodeName:}" failed. No retries permitted until 2026-04-22 21:09:25.446839907 +0000 UTC m=+6.172736506 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/843da596-cc50-4a3e-817c-4ee63a89c94a-metrics-certs") pod "network-metrics-daemon-fx8k2" (UID: "843da596-cc50-4a3e-817c-4ee63a89c94a") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 21:09:23.547076 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:23.547016 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hffwj\" (UniqueName: \"kubernetes.io/projected/516cb834-5ee4-4971-80a9-149fd935fc80-kube-api-access-hffwj\") pod \"network-check-target-lp7mc\" (UID: \"516cb834-5ee4-4971-80a9-149fd935fc80\") " pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:09:23.547200 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:23.547172 2580 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 22 21:09:23.547200 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:23.547188 2580 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 22 21:09:23.547200 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:23.547200 2580 projected.go:194] Error preparing data for projected volume kube-api-access-hffwj for pod openshift-network-diagnostics/network-check-target-lp7mc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 22 21:09:23.547361 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:23.547254 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/516cb834-5ee4-4971-80a9-149fd935fc80-kube-api-access-hffwj podName:516cb834-5ee4-4971-80a9-149fd935fc80 nodeName:}" failed. No retries permitted until 2026-04-22 21:09:25.547236121 +0000 UTC m=+6.273132714 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hffwj" (UniqueName: "kubernetes.io/projected/516cb834-5ee4-4971-80a9-149fd935fc80-kube-api-access-hffwj") pod "network-check-target-lp7mc" (UID: "516cb834-5ee4-4971-80a9-149fd935fc80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 22 21:09:23.859086 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:23.859010 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:23.859511 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:23.859145 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx8k2" podUID="843da596-cc50-4a3e-817c-4ee63a89c94a" Apr 22 21:09:23.859667 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:23.859645 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:09:23.859772 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:23.859749 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lp7mc" podUID="516cb834-5ee4-4971-80a9-149fd935fc80" Apr 22 21:09:23.973476 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:23.973057 2580 generic.go:358] "Generic (PLEG): container finished" podID="f511a0a82c52fc6840a29f94334af07f" containerID="2d008b2701ba6388c037bf77a8921bb3d126801d24d6b44ea6d04c511bcc5e26" exitCode=0 Apr 22 21:09:23.973476 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:23.973165 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal" event={"ID":"f511a0a82c52fc6840a29f94334af07f","Type":"ContainerDied","Data":"2d008b2701ba6388c037bf77a8921bb3d126801d24d6b44ea6d04c511bcc5e26"} Apr 22 21:09:23.998086 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:23.998018 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-20.ec2.internal" podStartSLOduration=3.998002885 podStartE2EDuration="3.998002885s" podCreationTimestamp="2026-04-22 21:09:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-22 21:09:22.921744764 +0000 UTC m=+3.647641367" watchObservedRunningTime="2026-04-22 21:09:23.998002885 +0000 UTC m=+4.723899483" Apr 22 21:09:24.988888 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:24.988822 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal" event={"ID":"f511a0a82c52fc6840a29f94334af07f","Type":"ContainerStarted","Data":"9f3f9db0dc8263b61864fc11a80be116ec37e32ccc7dd45959e6794071fb8492"} Apr 22 21:09:25.462288 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:25.462237 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/843da596-cc50-4a3e-817c-4ee63a89c94a-metrics-certs\") pod \"network-metrics-daemon-fx8k2\" (UID: \"843da596-cc50-4a3e-817c-4ee63a89c94a\") " pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:25.462477 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:25.462397 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 21:09:25.462477 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:25.462456 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/843da596-cc50-4a3e-817c-4ee63a89c94a-metrics-certs podName:843da596-cc50-4a3e-817c-4ee63a89c94a nodeName:}" failed. No retries permitted until 2026-04-22 21:09:29.462438128 +0000 UTC m=+10.188334720 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/843da596-cc50-4a3e-817c-4ee63a89c94a-metrics-certs") pod "network-metrics-daemon-fx8k2" (UID: "843da596-cc50-4a3e-817c-4ee63a89c94a") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 21:09:25.563372 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:25.563332 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hffwj\" (UniqueName: \"kubernetes.io/projected/516cb834-5ee4-4971-80a9-149fd935fc80-kube-api-access-hffwj\") pod \"network-check-target-lp7mc\" (UID: \"516cb834-5ee4-4971-80a9-149fd935fc80\") " pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:09:25.563611 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:25.563508 2580 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 22 21:09:25.563611 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:25.563528 2580 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 22 21:09:25.563611 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:25.563542 2580 projected.go:194] Error preparing data for projected volume kube-api-access-hffwj for pod openshift-network-diagnostics/network-check-target-lp7mc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 22 21:09:25.563611 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:25.563599 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/516cb834-5ee4-4971-80a9-149fd935fc80-kube-api-access-hffwj podName:516cb834-5ee4-4971-80a9-149fd935fc80 nodeName:}" failed. No retries permitted until 2026-04-22 21:09:29.563581158 +0000 UTC m=+10.289477741 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hffwj" (UniqueName: "kubernetes.io/projected/516cb834-5ee4-4971-80a9-149fd935fc80-kube-api-access-hffwj") pod "network-check-target-lp7mc" (UID: "516cb834-5ee4-4971-80a9-149fd935fc80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 22 21:09:25.859081 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:25.859001 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:25.859233 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:25.859153 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx8k2" podUID="843da596-cc50-4a3e-817c-4ee63a89c94a" Apr 22 21:09:25.859589 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:25.859568 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:09:25.859689 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:25.859669 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lp7mc" podUID="516cb834-5ee4-4971-80a9-149fd935fc80" Apr 22 21:09:27.858089 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:27.858055 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:27.858598 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:27.858265 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx8k2" podUID="843da596-cc50-4a3e-817c-4ee63a89c94a" Apr 22 21:09:27.858598 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:27.858293 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:09:27.858598 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:27.858409 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lp7mc" podUID="516cb834-5ee4-4971-80a9-149fd935fc80" Apr 22 21:09:29.496912 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:29.496867 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/843da596-cc50-4a3e-817c-4ee63a89c94a-metrics-certs\") pod \"network-metrics-daemon-fx8k2\" (UID: \"843da596-cc50-4a3e-817c-4ee63a89c94a\") " pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:29.497430 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:29.497050 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 21:09:29.497430 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:29.497118 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/843da596-cc50-4a3e-817c-4ee63a89c94a-metrics-certs podName:843da596-cc50-4a3e-817c-4ee63a89c94a nodeName:}" failed. No retries permitted until 2026-04-22 21:09:37.497095374 +0000 UTC m=+18.222991970 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/843da596-cc50-4a3e-817c-4ee63a89c94a-metrics-certs") pod "network-metrics-daemon-fx8k2" (UID: "843da596-cc50-4a3e-817c-4ee63a89c94a") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 21:09:29.597741 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:29.597672 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hffwj\" (UniqueName: \"kubernetes.io/projected/516cb834-5ee4-4971-80a9-149fd935fc80-kube-api-access-hffwj\") pod \"network-check-target-lp7mc\" (UID: \"516cb834-5ee4-4971-80a9-149fd935fc80\") " pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:09:29.597933 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:29.597871 2580 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 22 21:09:29.597933 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:29.597909 2580 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 22 21:09:29.597933 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:29.597922 2580 projected.go:194] Error preparing data for projected volume kube-api-access-hffwj for pod openshift-network-diagnostics/network-check-target-lp7mc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 22 21:09:29.598109 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:29.597985 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/516cb834-5ee4-4971-80a9-149fd935fc80-kube-api-access-hffwj podName:516cb834-5ee4-4971-80a9-149fd935fc80 nodeName:}" failed. No retries permitted until 2026-04-22 21:09:37.597965446 +0000 UTC m=+18.323862028 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hffwj" (UniqueName: "kubernetes.io/projected/516cb834-5ee4-4971-80a9-149fd935fc80-kube-api-access-hffwj") pod "network-check-target-lp7mc" (UID: "516cb834-5ee4-4971-80a9-149fd935fc80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 22 21:09:29.860216 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:29.859923 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:29.860371 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:29.860284 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx8k2" podUID="843da596-cc50-4a3e-817c-4ee63a89c94a" Apr 22 21:09:29.860493 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:29.860434 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:09:29.860563 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:29.860537 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lp7mc" podUID="516cb834-5ee4-4971-80a9-149fd935fc80" Apr 22 21:09:31.858726 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:31.858697 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:31.859218 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:31.858808 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx8k2" podUID="843da596-cc50-4a3e-817c-4ee63a89c94a" Apr 22 21:09:31.859218 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:31.859116 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:09:31.859218 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:31.859186 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lp7mc" podUID="516cb834-5ee4-4971-80a9-149fd935fc80" Apr 22 21:09:33.858713 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:33.858682 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:33.859215 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:33.858692 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:09:33.859215 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:33.858833 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx8k2" podUID="843da596-cc50-4a3e-817c-4ee63a89c94a" Apr 22 21:09:33.859215 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:33.858911 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lp7mc" podUID="516cb834-5ee4-4971-80a9-149fd935fc80" Apr 22 21:09:35.858077 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:35.858024 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:35.858077 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:35.858043 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:09:35.864596 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:35.863708 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx8k2" podUID="843da596-cc50-4a3e-817c-4ee63a89c94a" Apr 22 21:09:35.864596 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:35.863885 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lp7mc" podUID="516cb834-5ee4-4971-80a9-149fd935fc80" Apr 22 21:09:37.557984 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:37.557945 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/843da596-cc50-4a3e-817c-4ee63a89c94a-metrics-certs\") pod \"network-metrics-daemon-fx8k2\" (UID: \"843da596-cc50-4a3e-817c-4ee63a89c94a\") " pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:37.558428 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:37.558117 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 21:09:37.558428 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:37.558196 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/843da596-cc50-4a3e-817c-4ee63a89c94a-metrics-certs podName:843da596-cc50-4a3e-817c-4ee63a89c94a nodeName:}" failed. No retries permitted until 2026-04-22 21:09:53.558174212 +0000 UTC m=+34.284070790 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/843da596-cc50-4a3e-817c-4ee63a89c94a-metrics-certs") pod "network-metrics-daemon-fx8k2" (UID: "843da596-cc50-4a3e-817c-4ee63a89c94a") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 21:09:37.658412 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:37.658381 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hffwj\" (UniqueName: \"kubernetes.io/projected/516cb834-5ee4-4971-80a9-149fd935fc80-kube-api-access-hffwj\") pod \"network-check-target-lp7mc\" (UID: \"516cb834-5ee4-4971-80a9-149fd935fc80\") " pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:09:37.658584 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:37.658542 2580 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 22 21:09:37.658584 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:37.658563 2580 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 22 21:09:37.658584 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:37.658573 2580 projected.go:194] Error preparing data for projected volume kube-api-access-hffwj for pod openshift-network-diagnostics/network-check-target-lp7mc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 22 21:09:37.658697 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:37.658632 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/516cb834-5ee4-4971-80a9-149fd935fc80-kube-api-access-hffwj podName:516cb834-5ee4-4971-80a9-149fd935fc80 nodeName:}" failed. No retries permitted until 2026-04-22 21:09:53.65861721 +0000 UTC m=+34.384513806 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hffwj" (UniqueName: "kubernetes.io/projected/516cb834-5ee4-4971-80a9-149fd935fc80-kube-api-access-hffwj") pod "network-check-target-lp7mc" (UID: "516cb834-5ee4-4971-80a9-149fd935fc80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 22 21:09:37.858472 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:37.858385 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:37.858472 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:37.858441 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:09:37.858654 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:37.858524 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx8k2" podUID="843da596-cc50-4a3e-817c-4ee63a89c94a" Apr 22 21:09:37.858909 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:37.858882 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lp7mc" podUID="516cb834-5ee4-4971-80a9-149fd935fc80" Apr 22 21:09:39.859646 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:39.859322 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:39.859646 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:39.859365 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:09:39.859646 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:39.859445 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx8k2" podUID="843da596-cc50-4a3e-817c-4ee63a89c94a" Apr 22 21:09:39.859646 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:39.859579 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lp7mc" podUID="516cb834-5ee4-4971-80a9-149fd935fc80" Apr 22 21:09:40.015618 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:40.015584 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7xt7x" event={"ID":"72b91440-69fd-4335-a6e6-e8f741ad2da8","Type":"ContainerStarted","Data":"490d5d91a2b783b6790b0678fbe5ab4dca6cb3ce329976983b10dc2d1518afea"} Apr 22 21:09:40.017527 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:40.016880 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-49dpb" event={"ID":"9e4ebb4a-2ec6-41ad-ac25-a5d1d100711f","Type":"ContainerStarted","Data":"bc3c798c31d50be28c3f276d647c420ee5c4b1f5422cd62d7f2b0fa81f82acaf"} Apr 22 21:09:40.019817 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:40.019783 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wkk24" event={"ID":"13fe92d8-fa95-45da-8e47-5b754f77385c","Type":"ContainerStarted","Data":"37f509de8333575ab41f46b5118dafffd797aaac963f694e9d4812c5905acce3"} Apr 22 21:09:40.029428 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:40.029003 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7xt7x" podStartSLOduration=4.139070698 podStartE2EDuration="21.028987061s" podCreationTimestamp="2026-04-22 21:09:19 +0000 UTC" firstStartedPulling="2026-04-22 21:09:22.522191004 +0000 UTC m=+3.248087581" lastFinishedPulling="2026-04-22 21:09:39.412107367 +0000 UTC m=+20.138003944" observedRunningTime="2026-04-22 21:09:40.028412481 +0000 UTC m=+20.754309082" watchObservedRunningTime="2026-04-22 21:09:40.028987061 +0000 UTC m=+20.754883663" Apr 22 21:09:40.029801 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:40.029766 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal" podStartSLOduration=20.029751132 podStartE2EDuration="20.029751132s" podCreationTimestamp="2026-04-22 21:09:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-22 21:09:25.014383067 +0000 UTC m=+5.740279668" watchObservedRunningTime="2026-04-22 21:09:40.029751132 +0000 UTC m=+20.755647734" Apr 22 21:09:40.042757 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:40.042709 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wkk24" podStartSLOduration=11.951760151 podStartE2EDuration="21.042692727s" podCreationTimestamp="2026-04-22 21:09:19 +0000 UTC" firstStartedPulling="2026-04-22 21:09:22.528885628 +0000 UTC m=+3.254782208" lastFinishedPulling="2026-04-22 21:09:31.619818206 +0000 UTC m=+12.345714784" observedRunningTime="2026-04-22 21:09:40.041638356 +0000 UTC m=+20.767534957" watchObservedRunningTime="2026-04-22 21:09:40.042692727 +0000 UTC m=+20.768589328" Apr 22 21:09:40.055150 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:40.054658 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-49dpb" podStartSLOduration=4.173349177 podStartE2EDuration="21.054642922s" podCreationTimestamp="2026-04-22 21:09:19 +0000 UTC" firstStartedPulling="2026-04-22 21:09:22.5308112 +0000 UTC m=+3.256707792" lastFinishedPulling="2026-04-22 21:09:39.412104944 +0000 UTC m=+20.138001537" observedRunningTime="2026-04-22 21:09:40.05448129 +0000 UTC m=+20.780377886" watchObservedRunningTime="2026-04-22 21:09:40.054642922 +0000 UTC m=+20.780539524" Apr 22 21:09:41.023254 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:41.023021 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zwlx7" event={"ID":"408567c2-c6f7-4edd-bea3-ff7695a4d06e","Type":"ContainerStarted","Data":"cdb7650d380fafffbf99df759d8b1a1b78536017f2bf14d157627013b9fb9341"} Apr 22 21:09:41.024322 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:41.024298 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-n66xm" event={"ID":"be33f8b1-265a-4e31-9b06-742c72bf73e3","Type":"ContainerStarted","Data":"17b919ec9a23e013e69c5cb9374d06f333a3e5615d760d6e7ac3b986b397d04b"} Apr 22 21:09:41.025408 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:41.025388 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" event={"ID":"5815b051-9c47-4898-9ddb-7cb51812f609","Type":"ContainerStarted","Data":"d77564c15659b55a30ec81df6d0a7f289496087cc47ee163257e281311405381"} Apr 22 21:09:41.026662 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:41.026639 2580 generic.go:358] "Generic (PLEG): container finished" podID="efc4b2c1-b3de-4aa6-abe3-c37e3517897f" containerID="234ea90332fbcba56ea67d9938c504cd71a0e41042b6ee291061a991126e20fa" exitCode=0 Apr 22 21:09:41.026749 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:41.026721 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qvlsp" event={"ID":"efc4b2c1-b3de-4aa6-abe3-c37e3517897f","Type":"ContainerDied","Data":"234ea90332fbcba56ea67d9938c504cd71a0e41042b6ee291061a991126e20fa"} Apr 22 21:09:41.029164 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:41.029148 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ngls_b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b/ovn-acl-logging/0.log" Apr 22 21:09:41.029457 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:41.029437 2580 generic.go:358] "Generic (PLEG): container finished" podID="b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b" containerID="53ad46a793900c7259a36c7b5c1fdd99840b0c48ac83a140f3e0c766c617dffb" exitCode=1 Apr 22 21:09:41.029572 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:41.029530 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" event={"ID":"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b","Type":"ContainerStarted","Data":"ae99c3727e9f6c3a4e7c02b2b0ace9c5f2969d0f192ef8ae99e522e2d07bd329"} Apr 22 21:09:41.029572 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:41.029556 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" event={"ID":"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b","Type":"ContainerStarted","Data":"6d0fb5258e689cbfe20a8e8b13caf44ad63ac98bc998958bd59def76e0288a4b"} Apr 22 21:09:41.029650 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:41.029573 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" event={"ID":"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b","Type":"ContainerStarted","Data":"4c74edb7d5faca638c8c6db0333c4457ef337bdd45c107436f1e6ea8e9a34b26"} Apr 22 21:09:41.029650 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:41.029598 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" event={"ID":"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b","Type":"ContainerStarted","Data":"b599560f35c7966ee1bb1a5264cac9f276a70401d3d2e91453ebf07148403acd"} Apr 22 21:09:41.029650 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:41.029615 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" event={"ID":"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b","Type":"ContainerDied","Data":"53ad46a793900c7259a36c7b5c1fdd99840b0c48ac83a140f3e0c766c617dffb"} Apr 22 21:09:41.029650 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:41.029630 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" event={"ID":"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b","Type":"ContainerStarted","Data":"7e21a072c49684f419db9ddd7d2747bbda21655ae7c879016a2c9d733d0644d5"} Apr 22 21:09:41.060635 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:41.060574 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-zwlx7" podStartSLOduration=4.793435936 podStartE2EDuration="22.060564334s" podCreationTimestamp="2026-04-22 21:09:19 +0000 UTC" firstStartedPulling="2026-04-22 21:09:22.524593117 +0000 UTC m=+3.250489695" lastFinishedPulling="2026-04-22 21:09:39.791721502 +0000 UTC m=+20.517618093" observedRunningTime="2026-04-22 21:09:41.040613918 +0000 UTC m=+21.766510517" watchObservedRunningTime="2026-04-22 21:09:41.060564334 +0000 UTC m=+21.786460934" Apr 22 21:09:41.074995 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:41.074965 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-n66xm" podStartSLOduration=4.832729487 podStartE2EDuration="22.074956105s" podCreationTimestamp="2026-04-22 21:09:19 +0000 UTC" firstStartedPulling="2026-04-22 21:09:22.532693163 +0000 UTC m=+3.258589742" lastFinishedPulling="2026-04-22 21:09:39.77491977 +0000 UTC m=+20.500816360" observedRunningTime="2026-04-22 21:09:41.074837288 +0000 UTC m=+21.800733887" watchObservedRunningTime="2026-04-22 21:09:41.074956105 +0000 UTC m=+21.800852705" Apr 22 21:09:41.082999 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:41.082982 2580 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7xt7x" Apr 22 21:09:41.083610 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:41.083589 2580 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7xt7x" Apr 22 21:09:41.246215 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:41.246189 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Apr 22 21:09:41.794759 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:41.794644 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-04-22T21:09:41.24621002Z","UUID":"df0690cf-9341-474e-8471-4a870108a426","Handler":null,"Name":"","Endpoint":""} Apr 22 21:09:41.796765 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:41.796743 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Apr 22 21:09:41.796765 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:41.796772 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Apr 22 21:09:41.858726 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:41.858694 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:41.858927 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:41.858829 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx8k2" podUID="843da596-cc50-4a3e-817c-4ee63a89c94a" Apr 22 21:09:41.859038 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:41.858902 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:09:41.859108 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:41.859082 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lp7mc" podUID="516cb834-5ee4-4971-80a9-149fd935fc80" Apr 22 21:09:42.033790 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:42.033752 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" event={"ID":"5815b051-9c47-4898-9ddb-7cb51812f609","Type":"ContainerStarted","Data":"5d672ca16cdd04c5476dde0e6d2f4ad56d7c2e60eaab4a22fd05e8e9f2cad107"} Apr 22 21:09:42.036592 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:42.036561 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z7k7s" event={"ID":"178e5bc4-7770-41a1-82b7-7cc3cca7ef1c","Type":"ContainerStarted","Data":"0c06e80a074dcf1ef6b90fd3965a9402704974c10bd408777d4151205549f047"} Apr 22 21:09:42.051967 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:42.051885 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-z7k7s" podStartSLOduration=5.806064631 podStartE2EDuration="23.051872311s" podCreationTimestamp="2026-04-22 21:09:19 +0000 UTC" firstStartedPulling="2026-04-22 21:09:22.526819981 +0000 UTC m=+3.252716572" lastFinishedPulling="2026-04-22 21:09:39.77262766 +0000 UTC m=+20.498524252" observedRunningTime="2026-04-22 21:09:42.051407481 +0000 UTC m=+22.777304083" watchObservedRunningTime="2026-04-22 21:09:42.051872311 +0000 UTC m=+22.777768916" Apr 22 21:09:43.041454 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:43.041122 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" event={"ID":"5815b051-9c47-4898-9ddb-7cb51812f609","Type":"ContainerStarted","Data":"4981b8148ccce7c8aeb45a4e295fa278f1037fe0194c8c27bf37c75e475e2f6f"} Apr 22 21:09:43.044072 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:43.044039 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ngls_b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b/ovn-acl-logging/0.log" Apr 22 21:09:43.044479 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:43.044442 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" event={"ID":"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b","Type":"ContainerStarted","Data":"fec461a92f83bbd644fe579f9146e846a303256fea0b423355e30364d3395581"} Apr 22 21:09:43.044555 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:43.044480 2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Apr 22 21:09:43.057406 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:43.057361 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8m2nw" podStartSLOduration=4.065533852 podStartE2EDuration="24.057349039s" podCreationTimestamp="2026-04-22 21:09:19 +0000 UTC" firstStartedPulling="2026-04-22 21:09:22.530792668 +0000 UTC m=+3.256689246" lastFinishedPulling="2026-04-22 21:09:42.522607837 +0000 UTC m=+23.248504433" observedRunningTime="2026-04-22 21:09:43.056917801 +0000 UTC m=+23.782814402" watchObservedRunningTime="2026-04-22 21:09:43.057349039 +0000 UTC m=+23.783245647" Apr 22 21:09:43.858901 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:43.858867 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:09:43.858901 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:43.858894 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:43.859125 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:43.859004 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lp7mc" podUID="516cb834-5ee4-4971-80a9-149fd935fc80" Apr 22 21:09:43.859173 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:43.859137 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx8k2" podUID="843da596-cc50-4a3e-817c-4ee63a89c94a" Apr 22 21:09:45.858653 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:45.858624 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:45.858653 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:45.858644 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:09:45.859137 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:45.858721 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx8k2" podUID="843da596-cc50-4a3e-817c-4ee63a89c94a" Apr 22 21:09:45.859137 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:45.858885 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lp7mc" podUID="516cb834-5ee4-4971-80a9-149fd935fc80" Apr 22 21:09:46.051927 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:46.051894 2580 generic.go:358] "Generic (PLEG): container finished" podID="efc4b2c1-b3de-4aa6-abe3-c37e3517897f" containerID="ca6ab9b9214cbeb755fc897e796d7b74e1d80daad3b33426a447d72cddce4584" exitCode=0 Apr 22 21:09:46.052083 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:46.051982 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qvlsp" event={"ID":"efc4b2c1-b3de-4aa6-abe3-c37e3517897f","Type":"ContainerDied","Data":"ca6ab9b9214cbeb755fc897e796d7b74e1d80daad3b33426a447d72cddce4584"} Apr 22 21:09:46.055213 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:46.055195 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ngls_b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b/ovn-acl-logging/0.log" Apr 22 21:09:46.055560 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:46.055540 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" event={"ID":"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b","Type":"ContainerStarted","Data":"ce9161598f9a88166821551eca743b05296b31b6761c869911014b7d493961a6"} Apr 22 21:09:46.055878 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:46.055844 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:46.055939 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:46.055887 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:46.056036 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:46.055996 2580 scope.go:117] "RemoveContainer" containerID="53ad46a793900c7259a36c7b5c1fdd99840b0c48ac83a140f3e0c766c617dffb" Apr 22 21:09:46.075630 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:46.075612 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:46.968638 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:46.968360 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lp7mc"] Apr 22 21:09:46.969212 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:46.968673 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:09:46.969212 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:46.968758 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lp7mc" podUID="516cb834-5ee4-4971-80a9-149fd935fc80" Apr 22 21:09:46.971136 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:46.971112 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fx8k2"] Apr 22 21:09:46.971259 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:46.971227 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:46.971348 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:46.971330 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx8k2" podUID="843da596-cc50-4a3e-817c-4ee63a89c94a" Apr 22 21:09:47.060110 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:47.060084 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ngls_b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b/ovn-acl-logging/0.log" Apr 22 21:09:47.060459 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:47.060438 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" event={"ID":"b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b","Type":"ContainerStarted","Data":"339c2e2cce644236f83f984191e83165b6efc6b96d0fa7fa287efb447cee876f"} Apr 22 21:09:47.060649 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:47.060632 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:47.062486 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:47.062460 2580 generic.go:358] "Generic (PLEG): container finished" podID="efc4b2c1-b3de-4aa6-abe3-c37e3517897f" containerID="6b5c2a22cb28a86793bcf0ecf59b6b1b0c4595416c68ec475e69c47c7cff1a64" exitCode=0 Apr 22 21:09:47.062598 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:47.062496 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qvlsp" event={"ID":"efc4b2c1-b3de-4aa6-abe3-c37e3517897f","Type":"ContainerDied","Data":"6b5c2a22cb28a86793bcf0ecf59b6b1b0c4595416c68ec475e69c47c7cff1a64"} Apr 22 21:09:47.076138 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:47.076117 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:09:47.087050 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:47.087016 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" podStartSLOduration=10.775445011 podStartE2EDuration="28.087004945s" podCreationTimestamp="2026-04-22 21:09:19 +0000 UTC" firstStartedPulling="2026-04-22 21:09:22.527456156 +0000 UTC m=+3.253352734" lastFinishedPulling="2026-04-22 21:09:39.839016089 +0000 UTC m=+20.564912668" observedRunningTime="2026-04-22 21:09:47.085291214 +0000 UTC m=+27.811187814" watchObservedRunningTime="2026-04-22 21:09:47.087004945 +0000 UTC m=+27.812901571" Apr 22 21:09:48.065981 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:48.065899 2580 generic.go:358] "Generic (PLEG): container finished" podID="efc4b2c1-b3de-4aa6-abe3-c37e3517897f" containerID="ea4eb5917623fd926cc5d74d2286ebe00d1f574d84378dbced73bbacdc9603ed" exitCode=0 Apr 22 21:09:48.066376 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:48.065973 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qvlsp" event={"ID":"efc4b2c1-b3de-4aa6-abe3-c37e3517897f","Type":"ContainerDied","Data":"ea4eb5917623fd926cc5d74d2286ebe00d1f574d84378dbced73bbacdc9603ed"} Apr 22 21:09:48.858175 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:48.858145 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:09:48.858175 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:48.858176 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:48.858408 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:48.858249 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lp7mc" podUID="516cb834-5ee4-4971-80a9-149fd935fc80" Apr 22 21:09:48.858408 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:48.858320 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx8k2" podUID="843da596-cc50-4a3e-817c-4ee63a89c94a" Apr 22 21:09:50.594949 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:50.594913 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7xt7x" Apr 22 21:09:50.595473 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:50.595065 2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Apr 22 21:09:50.595576 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:50.595554 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7xt7x" Apr 22 21:09:50.858298 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:50.858224 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:09:50.858461 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:50.858235 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:50.858461 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:50.858330 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lp7mc" podUID="516cb834-5ee4-4971-80a9-149fd935fc80" Apr 22 21:09:50.858461 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:50.858440 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx8k2" podUID="843da596-cc50-4a3e-817c-4ee63a89c94a" Apr 22 21:09:52.858696 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:52.858465 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:09:52.859195 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:52.858468 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:52.859195 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:52.858788 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lp7mc" podUID="516cb834-5ee4-4971-80a9-149fd935fc80" Apr 22 21:09:52.859195 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:52.858908 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fx8k2" podUID="843da596-cc50-4a3e-817c-4ee63a89c94a" Apr 22 21:09:53.116553 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.116473 2580 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeReady" Apr 22 21:09:53.116703 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.116618 2580 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Apr 22 21:09:53.161066 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.159443 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9klhr"] Apr 22 21:09:53.190680 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.190653 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rp9w5"] Apr 22 21:09:53.190829 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.190801 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9klhr" Apr 22 21:09:53.193475 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.193439 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Apr 22 21:09:53.193601 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.193502 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8d48v\"" Apr 22 21:09:53.193766 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.193745 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Apr 22 21:09:53.215534 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.215509 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9klhr"] Apr 22 21:09:53.215534 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.215530 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rp9w5"] Apr 22 21:09:53.215693 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.215653 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rp9w5" Apr 22 21:09:53.218296 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.218274 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tq52g\"" Apr 22 21:09:53.218566 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.218422 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Apr 22 21:09:53.218566 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.218465 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Apr 22 21:09:53.218566 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.218479 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Apr 22 21:09:53.276225 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.276194 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-config-volume\") pod \"dns-default-9klhr\" (UID: \"e92ccf2d-8b03-40ec-81d7-e0752b8f6c78\") " pod="openshift-dns/dns-default-9klhr" Apr 22 21:09:53.276364 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.276242 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l462z\" (UniqueName: \"kubernetes.io/projected/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-kube-api-access-l462z\") pod \"dns-default-9klhr\" (UID: \"e92ccf2d-8b03-40ec-81d7-e0752b8f6c78\") " pod="openshift-dns/dns-default-9klhr" Apr 22 21:09:53.276364 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.276275 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-tmp-dir\") pod \"dns-default-9klhr\" (UID: \"e92ccf2d-8b03-40ec-81d7-e0752b8f6c78\") " pod="openshift-dns/dns-default-9klhr" Apr 22 21:09:53.276465 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.276365 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-metrics-tls\") pod \"dns-default-9klhr\" (UID: \"e92ccf2d-8b03-40ec-81d7-e0752b8f6c78\") " pod="openshift-dns/dns-default-9klhr" Apr 22 21:09:53.377515 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.377433 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-metrics-tls\") pod \"dns-default-9klhr\" (UID: \"e92ccf2d-8b03-40ec-81d7-e0752b8f6c78\") " pod="openshift-dns/dns-default-9klhr" Apr 22 21:09:53.377515 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.377503 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfl68\" (UniqueName: \"kubernetes.io/projected/9ef207b1-3445-413b-8855-1e0c977efcf2-kube-api-access-nfl68\") pod \"ingress-canary-rp9w5\" (UID: \"9ef207b1-3445-413b-8855-1e0c977efcf2\") " pod="openshift-ingress-canary/ingress-canary-rp9w5" Apr 22 21:09:53.377733 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.377535 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-config-volume\") pod \"dns-default-9klhr\" (UID: \"e92ccf2d-8b03-40ec-81d7-e0752b8f6c78\") " pod="openshift-dns/dns-default-9klhr" Apr 22 21:09:53.377733 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.377558 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l462z\" (UniqueName: \"kubernetes.io/projected/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-kube-api-access-l462z\") pod \"dns-default-9klhr\" (UID: \"e92ccf2d-8b03-40ec-81d7-e0752b8f6c78\") " pod="openshift-dns/dns-default-9klhr" Apr 22 21:09:53.377733 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.377594 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ef207b1-3445-413b-8855-1e0c977efcf2-cert\") pod \"ingress-canary-rp9w5\" (UID: \"9ef207b1-3445-413b-8855-1e0c977efcf2\") " pod="openshift-ingress-canary/ingress-canary-rp9w5" Apr 22 21:09:53.377733 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:53.377599 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 22 21:09:53.377733 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.377622 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-tmp-dir\") pod \"dns-default-9klhr\" (UID: \"e92ccf2d-8b03-40ec-81d7-e0752b8f6c78\") " pod="openshift-dns/dns-default-9klhr" Apr 22 21:09:53.377733 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:53.377681 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-metrics-tls podName:e92ccf2d-8b03-40ec-81d7-e0752b8f6c78 nodeName:}" failed. No retries permitted until 2026-04-22 21:09:53.87765848 +0000 UTC m=+34.603555058 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-metrics-tls") pod "dns-default-9klhr" (UID: "e92ccf2d-8b03-40ec-81d7-e0752b8f6c78") : secret "dns-default-metrics-tls" not found Apr 22 21:09:53.378069 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.378015 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-tmp-dir\") pod \"dns-default-9klhr\" (UID: \"e92ccf2d-8b03-40ec-81d7-e0752b8f6c78\") " pod="openshift-dns/dns-default-9klhr" Apr 22 21:09:53.378452 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.378427 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-config-volume\") pod \"dns-default-9klhr\" (UID: \"e92ccf2d-8b03-40ec-81d7-e0752b8f6c78\") " pod="openshift-dns/dns-default-9klhr" Apr 22 21:09:53.392284 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.392253 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l462z\" (UniqueName: \"kubernetes.io/projected/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-kube-api-access-l462z\") pod \"dns-default-9klhr\" (UID: \"e92ccf2d-8b03-40ec-81d7-e0752b8f6c78\") " pod="openshift-dns/dns-default-9klhr" Apr 22 21:09:53.478823 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.478793 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nfl68\" (UniqueName: \"kubernetes.io/projected/9ef207b1-3445-413b-8855-1e0c977efcf2-kube-api-access-nfl68\") pod \"ingress-canary-rp9w5\" (UID: \"9ef207b1-3445-413b-8855-1e0c977efcf2\") " pod="openshift-ingress-canary/ingress-canary-rp9w5" Apr 22 21:09:53.479048 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.478833 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ef207b1-3445-413b-8855-1e0c977efcf2-cert\") pod \"ingress-canary-rp9w5\" (UID: \"9ef207b1-3445-413b-8855-1e0c977efcf2\") " pod="openshift-ingress-canary/ingress-canary-rp9w5" Apr 22 21:09:53.479048 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:53.478943 2580 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 22 21:09:53.479048 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:53.479014 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ef207b1-3445-413b-8855-1e0c977efcf2-cert podName:9ef207b1-3445-413b-8855-1e0c977efcf2 nodeName:}" failed. No retries permitted until 2026-04-22 21:09:53.978994023 +0000 UTC m=+34.704890602 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9ef207b1-3445-413b-8855-1e0c977efcf2-cert") pod "ingress-canary-rp9w5" (UID: "9ef207b1-3445-413b-8855-1e0c977efcf2") : secret "canary-serving-cert" not found Apr 22 21:09:53.488444 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.488422 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfl68\" (UniqueName: \"kubernetes.io/projected/9ef207b1-3445-413b-8855-1e0c977efcf2-kube-api-access-nfl68\") pod \"ingress-canary-rp9w5\" (UID: \"9ef207b1-3445-413b-8855-1e0c977efcf2\") " pod="openshift-ingress-canary/ingress-canary-rp9w5" Apr 22 21:09:53.579693 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.579646 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/843da596-cc50-4a3e-817c-4ee63a89c94a-metrics-certs\") pod \"network-metrics-daemon-fx8k2\" (UID: \"843da596-cc50-4a3e-817c-4ee63a89c94a\") " pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:53.579839 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:53.579805 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 21:09:53.579910 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:53.579881 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/843da596-cc50-4a3e-817c-4ee63a89c94a-metrics-certs podName:843da596-cc50-4a3e-817c-4ee63a89c94a nodeName:}" failed. No retries permitted until 2026-04-22 21:10:25.57986612 +0000 UTC m=+66.305762710 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/843da596-cc50-4a3e-817c-4ee63a89c94a-metrics-certs") pod "network-metrics-daemon-fx8k2" (UID: "843da596-cc50-4a3e-817c-4ee63a89c94a") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 22 21:09:53.680764 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.680736 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hffwj\" (UniqueName: \"kubernetes.io/projected/516cb834-5ee4-4971-80a9-149fd935fc80-kube-api-access-hffwj\") pod \"network-check-target-lp7mc\" (UID: \"516cb834-5ee4-4971-80a9-149fd935fc80\") " pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:09:53.680929 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:53.680898 2580 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 22 21:09:53.680929 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:53.680917 2580 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 22 21:09:53.680929 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:53.680927 2580 projected.go:194] Error preparing data for projected volume kube-api-access-hffwj for pod openshift-network-diagnostics/network-check-target-lp7mc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 22 21:09:53.681029 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:53.680969 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/516cb834-5ee4-4971-80a9-149fd935fc80-kube-api-access-hffwj podName:516cb834-5ee4-4971-80a9-149fd935fc80 nodeName:}" failed. No retries permitted until 2026-04-22 21:10:25.680957169 +0000 UTC m=+66.406853747 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hffwj" (UniqueName: "kubernetes.io/projected/516cb834-5ee4-4971-80a9-149fd935fc80-kube-api-access-hffwj") pod "network-check-target-lp7mc" (UID: "516cb834-5ee4-4971-80a9-149fd935fc80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 22 21:09:53.882964 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.882935 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-metrics-tls\") pod \"dns-default-9klhr\" (UID: \"e92ccf2d-8b03-40ec-81d7-e0752b8f6c78\") " pod="openshift-dns/dns-default-9klhr" Apr 22 21:09:53.883392 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:53.883080 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 22 21:09:53.883392 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:53.883140 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-metrics-tls podName:e92ccf2d-8b03-40ec-81d7-e0752b8f6c78 nodeName:}" failed. No retries permitted until 2026-04-22 21:09:54.88312089 +0000 UTC m=+35.609017467 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-metrics-tls") pod "dns-default-9klhr" (UID: "e92ccf2d-8b03-40ec-81d7-e0752b8f6c78") : secret "dns-default-metrics-tls" not found Apr 22 21:09:53.984169 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:53.984138 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ef207b1-3445-413b-8855-1e0c977efcf2-cert\") pod \"ingress-canary-rp9w5\" (UID: \"9ef207b1-3445-413b-8855-1e0c977efcf2\") " pod="openshift-ingress-canary/ingress-canary-rp9w5" Apr 22 21:09:53.984296 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:53.984278 2580 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 22 21:09:53.984367 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:53.984346 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ef207b1-3445-413b-8855-1e0c977efcf2-cert podName:9ef207b1-3445-413b-8855-1e0c977efcf2 nodeName:}" failed. No retries permitted until 2026-04-22 21:09:54.984326283 +0000 UTC m=+35.710222863 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9ef207b1-3445-413b-8855-1e0c977efcf2-cert") pod "ingress-canary-rp9w5" (UID: "9ef207b1-3445-413b-8855-1e0c977efcf2") : secret "canary-serving-cert" not found Apr 22 21:09:54.079587 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:54.079558 2580 generic.go:358] "Generic (PLEG): container finished" podID="efc4b2c1-b3de-4aa6-abe3-c37e3517897f" containerID="55cedb193731ade291c6b065805f5f85d21af348c0cf960711a72278f0f32e99" exitCode=0 Apr 22 21:09:54.079685 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:54.079607 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qvlsp" event={"ID":"efc4b2c1-b3de-4aa6-abe3-c37e3517897f","Type":"ContainerDied","Data":"55cedb193731ade291c6b065805f5f85d21af348c0cf960711a72278f0f32e99"} Apr 22 21:09:54.858285 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:54.858251 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:09:54.858451 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:54.858251 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:09:54.861411 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:54.861383 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Apr 22 21:09:54.861547 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:54.861439 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gqtcf\"" Apr 22 21:09:54.861547 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:54.861445 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Apr 22 21:09:54.862837 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:54.862820 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Apr 22 21:09:54.862947 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:54.862902 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bw4nx\"" Apr 22 21:09:54.889953 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:54.889932 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-metrics-tls\") pod \"dns-default-9klhr\" (UID: \"e92ccf2d-8b03-40ec-81d7-e0752b8f6c78\") " pod="openshift-dns/dns-default-9klhr" Apr 22 21:09:54.890233 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:54.890061 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 22 21:09:54.890233 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:54.890109 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-metrics-tls podName:e92ccf2d-8b03-40ec-81d7-e0752b8f6c78 nodeName:}" failed. No retries permitted until 2026-04-22 21:09:56.890095789 +0000 UTC m=+37.615992367 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-metrics-tls") pod "dns-default-9klhr" (UID: "e92ccf2d-8b03-40ec-81d7-e0752b8f6c78") : secret "dns-default-metrics-tls" not found Apr 22 21:09:54.991090 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:54.991064 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ef207b1-3445-413b-8855-1e0c977efcf2-cert\") pod \"ingress-canary-rp9w5\" (UID: \"9ef207b1-3445-413b-8855-1e0c977efcf2\") " pod="openshift-ingress-canary/ingress-canary-rp9w5" Apr 22 21:09:54.991232 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:54.991200 2580 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 22 21:09:54.991275 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:54.991256 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ef207b1-3445-413b-8855-1e0c977efcf2-cert podName:9ef207b1-3445-413b-8855-1e0c977efcf2 nodeName:}" failed. No retries permitted until 2026-04-22 21:09:56.991243538 +0000 UTC m=+37.717140116 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9ef207b1-3445-413b-8855-1e0c977efcf2-cert") pod "ingress-canary-rp9w5" (UID: "9ef207b1-3445-413b-8855-1e0c977efcf2") : secret "canary-serving-cert" not found Apr 22 21:09:55.084599 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:55.084568 2580 generic.go:358] "Generic (PLEG): container finished" podID="efc4b2c1-b3de-4aa6-abe3-c37e3517897f" containerID="925217d299221c4b14baf3c5e3a335696faaf0b3c7b63ed5ccae3a577eb0706a" exitCode=0 Apr 22 21:09:55.084744 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:55.084617 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qvlsp" event={"ID":"efc4b2c1-b3de-4aa6-abe3-c37e3517897f","Type":"ContainerDied","Data":"925217d299221c4b14baf3c5e3a335696faaf0b3c7b63ed5ccae3a577eb0706a"} Apr 22 21:09:56.089119 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:56.089084 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qvlsp" event={"ID":"efc4b2c1-b3de-4aa6-abe3-c37e3517897f","Type":"ContainerStarted","Data":"c72fe7697a51039d56a5eae4f93e248bc2f8fc2ea3983bea7095d099d1ce4dae"} Apr 22 21:09:56.111347 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:56.111306 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qvlsp" podStartSLOduration=5.914479339 podStartE2EDuration="37.11128959s" podCreationTimestamp="2026-04-22 21:09:19 +0000 UTC" firstStartedPulling="2026-04-22 21:09:22.532549444 +0000 UTC m=+3.258446022" lastFinishedPulling="2026-04-22 21:09:53.729359693 +0000 UTC m=+34.455256273" observedRunningTime="2026-04-22 21:09:56.111046622 +0000 UTC m=+36.836943223" watchObservedRunningTime="2026-04-22 21:09:56.11128959 +0000 UTC m=+36.837186190" Apr 22 21:09:56.904531 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:56.904499 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-metrics-tls\") pod \"dns-default-9klhr\" (UID: \"e92ccf2d-8b03-40ec-81d7-e0752b8f6c78\") " pod="openshift-dns/dns-default-9klhr" Apr 22 21:09:56.904674 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:56.904643 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 22 21:09:56.904714 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:56.904707 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-metrics-tls podName:e92ccf2d-8b03-40ec-81d7-e0752b8f6c78 nodeName:}" failed. No retries permitted until 2026-04-22 21:10:00.904688119 +0000 UTC m=+41.630584698 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-metrics-tls") pod "dns-default-9klhr" (UID: "e92ccf2d-8b03-40ec-81d7-e0752b8f6c78") : secret "dns-default-metrics-tls" not found Apr 22 21:09:57.005562 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:09:57.005530 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ef207b1-3445-413b-8855-1e0c977efcf2-cert\") pod \"ingress-canary-rp9w5\" (UID: \"9ef207b1-3445-413b-8855-1e0c977efcf2\") " pod="openshift-ingress-canary/ingress-canary-rp9w5" Apr 22 21:09:57.005706 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:57.005649 2580 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 22 21:09:57.005706 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:09:57.005698 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ef207b1-3445-413b-8855-1e0c977efcf2-cert podName:9ef207b1-3445-413b-8855-1e0c977efcf2 nodeName:}" failed. No retries permitted until 2026-04-22 21:10:01.005685347 +0000 UTC m=+41.731581924 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9ef207b1-3445-413b-8855-1e0c977efcf2-cert") pod "ingress-canary-rp9w5" (UID: "9ef207b1-3445-413b-8855-1e0c977efcf2") : secret "canary-serving-cert" not found Apr 22 21:10:00.929073 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:00.929031 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-metrics-tls\") pod \"dns-default-9klhr\" (UID: \"e92ccf2d-8b03-40ec-81d7-e0752b8f6c78\") " pod="openshift-dns/dns-default-9klhr" Apr 22 21:10:00.929512 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:10:00.929186 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 22 21:10:00.929512 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:10:00.929280 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-metrics-tls podName:e92ccf2d-8b03-40ec-81d7-e0752b8f6c78 nodeName:}" failed. No retries permitted until 2026-04-22 21:10:08.929264934 +0000 UTC m=+49.655161512 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-metrics-tls") pod "dns-default-9klhr" (UID: "e92ccf2d-8b03-40ec-81d7-e0752b8f6c78") : secret "dns-default-metrics-tls" not found Apr 22 21:10:01.030150 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:01.030115 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ef207b1-3445-413b-8855-1e0c977efcf2-cert\") pod \"ingress-canary-rp9w5\" (UID: \"9ef207b1-3445-413b-8855-1e0c977efcf2\") " pod="openshift-ingress-canary/ingress-canary-rp9w5" Apr 22 21:10:01.030292 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:10:01.030273 2580 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 22 21:10:01.030351 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:10:01.030343 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ef207b1-3445-413b-8855-1e0c977efcf2-cert podName:9ef207b1-3445-413b-8855-1e0c977efcf2 nodeName:}" failed. No retries permitted until 2026-04-22 21:10:09.030323745 +0000 UTC m=+49.756220325 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9ef207b1-3445-413b-8855-1e0c977efcf2-cert") pod "ingress-canary-rp9w5" (UID: "9ef207b1-3445-413b-8855-1e0c977efcf2") : secret "canary-serving-cert" not found Apr 22 21:10:08.987056 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:08.987018 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-metrics-tls\") pod \"dns-default-9klhr\" (UID: \"e92ccf2d-8b03-40ec-81d7-e0752b8f6c78\") " pod="openshift-dns/dns-default-9klhr" Apr 22 21:10:08.987437 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:10:08.987136 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 22 21:10:08.987437 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:10:08.987185 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-metrics-tls podName:e92ccf2d-8b03-40ec-81d7-e0752b8f6c78 nodeName:}" failed. No retries permitted until 2026-04-22 21:10:24.987172668 +0000 UTC m=+65.713069246 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-metrics-tls") pod "dns-default-9klhr" (UID: "e92ccf2d-8b03-40ec-81d7-e0752b8f6c78") : secret "dns-default-metrics-tls" not found Apr 22 21:10:09.087887 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:09.087840 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ef207b1-3445-413b-8855-1e0c977efcf2-cert\") pod \"ingress-canary-rp9w5\" (UID: \"9ef207b1-3445-413b-8855-1e0c977efcf2\") " pod="openshift-ingress-canary/ingress-canary-rp9w5" Apr 22 21:10:09.088010 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:10:09.087996 2580 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 22 21:10:09.088070 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:10:09.088060 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ef207b1-3445-413b-8855-1e0c977efcf2-cert podName:9ef207b1-3445-413b-8855-1e0c977efcf2 nodeName:}" failed. No retries permitted until 2026-04-22 21:10:25.08804395 +0000 UTC m=+65.813940532 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9ef207b1-3445-413b-8855-1e0c977efcf2-cert") pod "ingress-canary-rp9w5" (UID: "9ef207b1-3445-413b-8855-1e0c977efcf2") : secret "canary-serving-cert" not found Apr 22 21:10:19.079218 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:19.079190 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8ngls" Apr 22 21:10:25.000044 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:25.000004 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-metrics-tls\") pod \"dns-default-9klhr\" (UID: \"e92ccf2d-8b03-40ec-81d7-e0752b8f6c78\") " pod="openshift-dns/dns-default-9klhr" Apr 22 21:10:25.000420 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:10:25.000147 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 22 21:10:25.000420 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:10:25.000216 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-metrics-tls podName:e92ccf2d-8b03-40ec-81d7-e0752b8f6c78 nodeName:}" failed. No retries permitted until 2026-04-22 21:10:57.000195297 +0000 UTC m=+97.726091889 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-metrics-tls") pod "dns-default-9klhr" (UID: "e92ccf2d-8b03-40ec-81d7-e0752b8f6c78") : secret "dns-default-metrics-tls" not found Apr 22 21:10:25.101154 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:25.101121 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ef207b1-3445-413b-8855-1e0c977efcf2-cert\") pod \"ingress-canary-rp9w5\" (UID: \"9ef207b1-3445-413b-8855-1e0c977efcf2\") " pod="openshift-ingress-canary/ingress-canary-rp9w5" Apr 22 21:10:25.101299 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:10:25.101280 2580 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 22 21:10:25.101361 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:10:25.101352 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ef207b1-3445-413b-8855-1e0c977efcf2-cert podName:9ef207b1-3445-413b-8855-1e0c977efcf2 nodeName:}" failed. No retries permitted until 2026-04-22 21:10:57.101337437 +0000 UTC m=+97.827234016 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9ef207b1-3445-413b-8855-1e0c977efcf2-cert") pod "ingress-canary-rp9w5" (UID: "9ef207b1-3445-413b-8855-1e0c977efcf2") : secret "canary-serving-cert" not found Apr 22 21:10:25.603910 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:25.603846 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/843da596-cc50-4a3e-817c-4ee63a89c94a-metrics-certs\") pod \"network-metrics-daemon-fx8k2\" (UID: \"843da596-cc50-4a3e-817c-4ee63a89c94a\") " pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:10:25.606953 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:25.606937 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Apr 22 21:10:25.614055 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:10:25.614029 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Apr 22 21:10:25.614150 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:10:25.614128 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/843da596-cc50-4a3e-817c-4ee63a89c94a-metrics-certs podName:843da596-cc50-4a3e-817c-4ee63a89c94a nodeName:}" failed. No retries permitted until 2026-04-22 21:11:29.614106808 +0000 UTC m=+130.340003386 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/843da596-cc50-4a3e-817c-4ee63a89c94a-metrics-certs") pod "network-metrics-daemon-fx8k2" (UID: "843da596-cc50-4a3e-817c-4ee63a89c94a") : secret "metrics-daemon-secret" not found Apr 22 21:10:25.704771 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:25.704742 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hffwj\" (UniqueName: \"kubernetes.io/projected/516cb834-5ee4-4971-80a9-149fd935fc80-kube-api-access-hffwj\") pod \"network-check-target-lp7mc\" (UID: \"516cb834-5ee4-4971-80a9-149fd935fc80\") " pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:10:25.707772 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:25.707757 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Apr 22 21:10:25.717155 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:25.717138 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Apr 22 21:10:25.728619 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:25.728594 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hffwj\" (UniqueName: \"kubernetes.io/projected/516cb834-5ee4-4971-80a9-149fd935fc80-kube-api-access-hffwj\") pod \"network-check-target-lp7mc\" (UID: \"516cb834-5ee4-4971-80a9-149fd935fc80\") " pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:10:25.769943 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:25.769925 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gqtcf\"" Apr 22 21:10:25.778084 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:25.778063 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:10:25.904311 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:25.904286 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lp7mc"] Apr 22 21:10:25.907341 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:10:25.907305 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod516cb834_5ee4_4971_80a9_149fd935fc80.slice/crio-f4e3ce82cfa5b10d52a4021ab9511f8be9b5490ffe908328a0c3530dda1de0b2 WatchSource:0}: Error finding container f4e3ce82cfa5b10d52a4021ab9511f8be9b5490ffe908328a0c3530dda1de0b2: Status 404 returned error can't find the container with id f4e3ce82cfa5b10d52a4021ab9511f8be9b5490ffe908328a0c3530dda1de0b2 Apr 22 21:10:26.145095 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:26.145061 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lp7mc" event={"ID":"516cb834-5ee4-4971-80a9-149fd935fc80","Type":"ContainerStarted","Data":"f4e3ce82cfa5b10d52a4021ab9511f8be9b5490ffe908328a0c3530dda1de0b2"} Apr 22 21:10:29.152519 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:29.152481 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lp7mc" event={"ID":"516cb834-5ee4-4971-80a9-149fd935fc80","Type":"ContainerStarted","Data":"55fe87621509d3cd00fbc332424433a4bb96e614f1dd79164e675c7f9d36ca3a"} Apr 22 21:10:29.152910 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:29.152695 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:10:29.168876 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:29.168803 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lp7mc" podStartSLOduration=67.650330047 podStartE2EDuration="1m10.168787311s" podCreationTimestamp="2026-04-22 21:09:19 +0000 UTC" firstStartedPulling="2026-04-22 21:10:25.909167075 +0000 UTC m=+66.635063654" lastFinishedPulling="2026-04-22 21:10:28.42762434 +0000 UTC m=+69.153520918" observedRunningTime="2026-04-22 21:10:29.168490734 +0000 UTC m=+69.894387335" watchObservedRunningTime="2026-04-22 21:10:29.168787311 +0000 UTC m=+69.894683915" Apr 22 21:10:57.010832 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:57.010784 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-metrics-tls\") pod \"dns-default-9klhr\" (UID: \"e92ccf2d-8b03-40ec-81d7-e0752b8f6c78\") " pod="openshift-dns/dns-default-9klhr" Apr 22 21:10:57.011362 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:10:57.010972 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 22 21:10:57.011362 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:10:57.011068 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-metrics-tls podName:e92ccf2d-8b03-40ec-81d7-e0752b8f6c78 nodeName:}" failed. No retries permitted until 2026-04-22 21:12:01.011047302 +0000 UTC m=+161.736943882 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-metrics-tls") pod "dns-default-9klhr" (UID: "e92ccf2d-8b03-40ec-81d7-e0752b8f6c78") : secret "dns-default-metrics-tls" not found Apr 22 21:10:57.111370 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:57.111341 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ef207b1-3445-413b-8855-1e0c977efcf2-cert\") pod \"ingress-canary-rp9w5\" (UID: \"9ef207b1-3445-413b-8855-1e0c977efcf2\") " pod="openshift-ingress-canary/ingress-canary-rp9w5" Apr 22 21:10:57.111548 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:10:57.111470 2580 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 22 21:10:57.111548 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:10:57.111532 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ef207b1-3445-413b-8855-1e0c977efcf2-cert podName:9ef207b1-3445-413b-8855-1e0c977efcf2 nodeName:}" failed. No retries permitted until 2026-04-22 21:12:01.111514521 +0000 UTC m=+161.837411099 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9ef207b1-3445-413b-8855-1e0c977efcf2-cert") pod "ingress-canary-rp9w5" (UID: "9ef207b1-3445-413b-8855-1e0c977efcf2") : secret "canary-serving-cert" not found Apr 22 21:10:59.378183 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.378151 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-8594858796-49pkv"] Apr 22 21:10:59.380961 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.380945 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:10:59.383492 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.383470 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Apr 22 21:10:59.383492 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.383490 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Apr 22 21:10:59.383642 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.383622 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Apr 22 21:10:59.383757 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.383741 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Apr 22 21:10:59.383834 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.383820 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Apr 22 21:10:59.384949 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.384933 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Apr 22 21:10:59.385042 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.384967 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-bqnqt\"" Apr 22 21:10:59.390403 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.390366 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-8594858796-49pkv"] Apr 22 21:10:59.427864 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.425461 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qkn7\" (UniqueName: \"kubernetes.io/projected/39daacd1-808b-4be7-95be-bccd8a0817e7-kube-api-access-7qkn7\") pod \"router-default-8594858796-49pkv\" (UID: \"39daacd1-808b-4be7-95be-bccd8a0817e7\") " pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:10:59.427864 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.425585 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/39daacd1-808b-4be7-95be-bccd8a0817e7-default-certificate\") pod \"router-default-8594858796-49pkv\" (UID: \"39daacd1-808b-4be7-95be-bccd8a0817e7\") " pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:10:59.427864 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.425641 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39daacd1-808b-4be7-95be-bccd8a0817e7-metrics-certs\") pod \"router-default-8594858796-49pkv\" (UID: \"39daacd1-808b-4be7-95be-bccd8a0817e7\") " pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:10:59.427864 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.425677 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39daacd1-808b-4be7-95be-bccd8a0817e7-service-ca-bundle\") pod \"router-default-8594858796-49pkv\" (UID: \"39daacd1-808b-4be7-95be-bccd8a0817e7\") " pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:10:59.427864 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.425708 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/39daacd1-808b-4be7-95be-bccd8a0817e7-stats-auth\") pod \"router-default-8594858796-49pkv\" (UID: \"39daacd1-808b-4be7-95be-bccd8a0817e7\") " pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:10:59.482537 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.482504 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-585dfdc468-gwzzw"] Apr 22 21:10:59.485092 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.485077 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-585dfdc468-gwzzw" Apr 22 21:10:59.487684 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.487664 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Apr 22 21:10:59.487826 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.487804 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-f8l4j\"" Apr 22 21:10:59.487963 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.487830 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Apr 22 21:10:59.487963 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.487838 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Apr 22 21:10:59.487963 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.487917 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Apr 22 21:10:59.493169 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.493146 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Apr 22 21:10:59.493520 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.493501 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-585dfdc468-gwzzw"] Apr 22 21:10:59.526928 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.526905 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d912a63-fac1-4f77-b588-e42de7a51799-trusted-ca-bundle\") pod \"insights-operator-585dfdc468-gwzzw\" (UID: \"7d912a63-fac1-4f77-b588-e42de7a51799\") " pod="openshift-insights/insights-operator-585dfdc468-gwzzw" Apr 22 21:10:59.527036 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.526947 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d912a63-fac1-4f77-b588-e42de7a51799-service-ca-bundle\") pod \"insights-operator-585dfdc468-gwzzw\" (UID: \"7d912a63-fac1-4f77-b588-e42de7a51799\") " pod="openshift-insights/insights-operator-585dfdc468-gwzzw" Apr 22 21:10:59.527036 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.526966 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7qkn7\" (UniqueName: \"kubernetes.io/projected/39daacd1-808b-4be7-95be-bccd8a0817e7-kube-api-access-7qkn7\") pod \"router-default-8594858796-49pkv\" (UID: \"39daacd1-808b-4be7-95be-bccd8a0817e7\") " pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:10:59.527036 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.526983 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csjqc\" (UniqueName: \"kubernetes.io/projected/7d912a63-fac1-4f77-b588-e42de7a51799-kube-api-access-csjqc\") pod \"insights-operator-585dfdc468-gwzzw\" (UID: \"7d912a63-fac1-4f77-b588-e42de7a51799\") " pod="openshift-insights/insights-operator-585dfdc468-gwzzw" Apr 22 21:10:59.527190 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.527055 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/39daacd1-808b-4be7-95be-bccd8a0817e7-default-certificate\") pod \"router-default-8594858796-49pkv\" (UID: \"39daacd1-808b-4be7-95be-bccd8a0817e7\") " pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:10:59.527190 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.527098 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39daacd1-808b-4be7-95be-bccd8a0817e7-metrics-certs\") pod \"router-default-8594858796-49pkv\" (UID: \"39daacd1-808b-4be7-95be-bccd8a0817e7\") " pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:10:59.527190 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.527127 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7d912a63-fac1-4f77-b588-e42de7a51799-tmp\") pod \"insights-operator-585dfdc468-gwzzw\" (UID: \"7d912a63-fac1-4f77-b588-e42de7a51799\") " pod="openshift-insights/insights-operator-585dfdc468-gwzzw" Apr 22 21:10:59.527190 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.527147 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7d912a63-fac1-4f77-b588-e42de7a51799-snapshots\") pod \"insights-operator-585dfdc468-gwzzw\" (UID: \"7d912a63-fac1-4f77-b588-e42de7a51799\") " pod="openshift-insights/insights-operator-585dfdc468-gwzzw" Apr 22 21:10:59.527190 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.527164 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7d912a63-fac1-4f77-b588-e42de7a51799-serving-cert\") pod \"insights-operator-585dfdc468-gwzzw\" (UID: \"7d912a63-fac1-4f77-b588-e42de7a51799\") " pod="openshift-insights/insights-operator-585dfdc468-gwzzw" Apr 22 21:10:59.527421 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.527193 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39daacd1-808b-4be7-95be-bccd8a0817e7-service-ca-bundle\") pod \"router-default-8594858796-49pkv\" (UID: \"39daacd1-808b-4be7-95be-bccd8a0817e7\") " pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:10:59.527421 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.527218 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/39daacd1-808b-4be7-95be-bccd8a0817e7-stats-auth\") pod \"router-default-8594858796-49pkv\" (UID: \"39daacd1-808b-4be7-95be-bccd8a0817e7\") " pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:10:59.527421 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:10:59.527254 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Apr 22 21:10:59.527421 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:10:59.527290 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/39daacd1-808b-4be7-95be-bccd8a0817e7-service-ca-bundle podName:39daacd1-808b-4be7-95be-bccd8a0817e7 nodeName:}" failed. No retries permitted until 2026-04-22 21:11:00.027275341 +0000 UTC m=+100.753171952 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/39daacd1-808b-4be7-95be-bccd8a0817e7-service-ca-bundle") pod "router-default-8594858796-49pkv" (UID: "39daacd1-808b-4be7-95be-bccd8a0817e7") : configmap references non-existent config key: service-ca.crt Apr 22 21:10:59.527421 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:10:59.527317 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39daacd1-808b-4be7-95be-bccd8a0817e7-metrics-certs podName:39daacd1-808b-4be7-95be-bccd8a0817e7 nodeName:}" failed. No retries permitted until 2026-04-22 21:11:00.027301605 +0000 UTC m=+100.753198203 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/39daacd1-808b-4be7-95be-bccd8a0817e7-metrics-certs") pod "router-default-8594858796-49pkv" (UID: "39daacd1-808b-4be7-95be-bccd8a0817e7") : secret "router-metrics-certs-default" not found Apr 22 21:10:59.529541 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.529513 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/39daacd1-808b-4be7-95be-bccd8a0817e7-stats-auth\") pod \"router-default-8594858796-49pkv\" (UID: \"39daacd1-808b-4be7-95be-bccd8a0817e7\") " pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:10:59.529658 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.529639 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/39daacd1-808b-4be7-95be-bccd8a0817e7-default-certificate\") pod \"router-default-8594858796-49pkv\" (UID: \"39daacd1-808b-4be7-95be-bccd8a0817e7\") " pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:10:59.534807 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.534787 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qkn7\" (UniqueName: \"kubernetes.io/projected/39daacd1-808b-4be7-95be-bccd8a0817e7-kube-api-access-7qkn7\") pod \"router-default-8594858796-49pkv\" (UID: \"39daacd1-808b-4be7-95be-bccd8a0817e7\") " pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:10:59.627929 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.627866 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7d912a63-fac1-4f77-b588-e42de7a51799-tmp\") pod \"insights-operator-585dfdc468-gwzzw\" (UID: \"7d912a63-fac1-4f77-b588-e42de7a51799\") " pod="openshift-insights/insights-operator-585dfdc468-gwzzw" Apr 22 21:10:59.627929 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.627931 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7d912a63-fac1-4f77-b588-e42de7a51799-snapshots\") pod \"insights-operator-585dfdc468-gwzzw\" (UID: \"7d912a63-fac1-4f77-b588-e42de7a51799\") " pod="openshift-insights/insights-operator-585dfdc468-gwzzw" Apr 22 21:10:59.628112 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.627962 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7d912a63-fac1-4f77-b588-e42de7a51799-serving-cert\") pod \"insights-operator-585dfdc468-gwzzw\" (UID: \"7d912a63-fac1-4f77-b588-e42de7a51799\") " pod="openshift-insights/insights-operator-585dfdc468-gwzzw" Apr 22 21:10:59.628112 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.627990 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d912a63-fac1-4f77-b588-e42de7a51799-trusted-ca-bundle\") pod \"insights-operator-585dfdc468-gwzzw\" (UID: \"7d912a63-fac1-4f77-b588-e42de7a51799\") " pod="openshift-insights/insights-operator-585dfdc468-gwzzw" Apr 22 21:10:59.628112 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.628026 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d912a63-fac1-4f77-b588-e42de7a51799-service-ca-bundle\") pod \"insights-operator-585dfdc468-gwzzw\" (UID: \"7d912a63-fac1-4f77-b588-e42de7a51799\") " pod="openshift-insights/insights-operator-585dfdc468-gwzzw" Apr 22 21:10:59.628112 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.628055 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-csjqc\" (UniqueName: \"kubernetes.io/projected/7d912a63-fac1-4f77-b588-e42de7a51799-kube-api-access-csjqc\") pod \"insights-operator-585dfdc468-gwzzw\" (UID: \"7d912a63-fac1-4f77-b588-e42de7a51799\") " pod="openshift-insights/insights-operator-585dfdc468-gwzzw" Apr 22 21:10:59.628374 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.628330 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7d912a63-fac1-4f77-b588-e42de7a51799-tmp\") pod \"insights-operator-585dfdc468-gwzzw\" (UID: \"7d912a63-fac1-4f77-b588-e42de7a51799\") " pod="openshift-insights/insights-operator-585dfdc468-gwzzw" Apr 22 21:10:59.628551 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.628531 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d912a63-fac1-4f77-b588-e42de7a51799-service-ca-bundle\") pod \"insights-operator-585dfdc468-gwzzw\" (UID: \"7d912a63-fac1-4f77-b588-e42de7a51799\") " pod="openshift-insights/insights-operator-585dfdc468-gwzzw" Apr 22 21:10:59.628613 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.628602 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7d912a63-fac1-4f77-b588-e42de7a51799-snapshots\") pod \"insights-operator-585dfdc468-gwzzw\" (UID: \"7d912a63-fac1-4f77-b588-e42de7a51799\") " pod="openshift-insights/insights-operator-585dfdc468-gwzzw" Apr 22 21:10:59.628964 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.628947 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d912a63-fac1-4f77-b588-e42de7a51799-trusted-ca-bundle\") pod \"insights-operator-585dfdc468-gwzzw\" (UID: \"7d912a63-fac1-4f77-b588-e42de7a51799\") " pod="openshift-insights/insights-operator-585dfdc468-gwzzw" Apr 22 21:10:59.630485 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.630470 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7d912a63-fac1-4f77-b588-e42de7a51799-serving-cert\") pod \"insights-operator-585dfdc468-gwzzw\" (UID: \"7d912a63-fac1-4f77-b588-e42de7a51799\") " pod="openshift-insights/insights-operator-585dfdc468-gwzzw" Apr 22 21:10:59.635474 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.635456 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-csjqc\" (UniqueName: \"kubernetes.io/projected/7d912a63-fac1-4f77-b588-e42de7a51799-kube-api-access-csjqc\") pod \"insights-operator-585dfdc468-gwzzw\" (UID: \"7d912a63-fac1-4f77-b588-e42de7a51799\") " pod="openshift-insights/insights-operator-585dfdc468-gwzzw" Apr 22 21:10:59.794333 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.794287 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-585dfdc468-gwzzw" Apr 22 21:10:59.907178 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:10:59.907148 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-585dfdc468-gwzzw"] Apr 22 21:10:59.909833 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:10:59.909807 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d912a63_fac1_4f77_b588_e42de7a51799.slice/crio-4f27ffb2a58514b060630e582cc7e21d4ef0b2a22b4e0c8422fc8924918e370a WatchSource:0}: Error finding container 4f27ffb2a58514b060630e582cc7e21d4ef0b2a22b4e0c8422fc8924918e370a: Status 404 returned error can't find the container with id 4f27ffb2a58514b060630e582cc7e21d4ef0b2a22b4e0c8422fc8924918e370a Apr 22 21:11:00.031796 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:00.031767 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39daacd1-808b-4be7-95be-bccd8a0817e7-service-ca-bundle\") pod \"router-default-8594858796-49pkv\" (UID: \"39daacd1-808b-4be7-95be-bccd8a0817e7\") " pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:11:00.031943 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:00.031880 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39daacd1-808b-4be7-95be-bccd8a0817e7-metrics-certs\") pod \"router-default-8594858796-49pkv\" (UID: \"39daacd1-808b-4be7-95be-bccd8a0817e7\") " pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:11:00.031943 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:00.031926 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/39daacd1-808b-4be7-95be-bccd8a0817e7-service-ca-bundle podName:39daacd1-808b-4be7-95be-bccd8a0817e7 nodeName:}" failed. No retries permitted until 2026-04-22 21:11:01.03191044 +0000 UTC m=+101.757807041 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/39daacd1-808b-4be7-95be-bccd8a0817e7-service-ca-bundle") pod "router-default-8594858796-49pkv" (UID: "39daacd1-808b-4be7-95be-bccd8a0817e7") : configmap references non-existent config key: service-ca.crt Apr 22 21:11:00.032026 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:00.031980 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Apr 22 21:11:00.032060 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:00.032028 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39daacd1-808b-4be7-95be-bccd8a0817e7-metrics-certs podName:39daacd1-808b-4be7-95be-bccd8a0817e7 nodeName:}" failed. No retries permitted until 2026-04-22 21:11:01.032012567 +0000 UTC m=+101.757909146 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/39daacd1-808b-4be7-95be-bccd8a0817e7-metrics-certs") pod "router-default-8594858796-49pkv" (UID: "39daacd1-808b-4be7-95be-bccd8a0817e7") : secret "router-metrics-certs-default" not found Apr 22 21:11:00.155919 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:00.155895 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lp7mc" Apr 22 21:11:00.210817 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:00.210735 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-585dfdc468-gwzzw" event={"ID":"7d912a63-fac1-4f77-b588-e42de7a51799","Type":"ContainerStarted","Data":"4f27ffb2a58514b060630e582cc7e21d4ef0b2a22b4e0c8422fc8924918e370a"} Apr 22 21:11:01.041955 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:01.041923 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39daacd1-808b-4be7-95be-bccd8a0817e7-service-ca-bundle\") pod \"router-default-8594858796-49pkv\" (UID: \"39daacd1-808b-4be7-95be-bccd8a0817e7\") " pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:11:01.042320 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:01.041999 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39daacd1-808b-4be7-95be-bccd8a0817e7-metrics-certs\") pod \"router-default-8594858796-49pkv\" (UID: \"39daacd1-808b-4be7-95be-bccd8a0817e7\") " pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:11:01.042320 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:01.042079 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/39daacd1-808b-4be7-95be-bccd8a0817e7-service-ca-bundle podName:39daacd1-808b-4be7-95be-bccd8a0817e7 nodeName:}" failed. No retries permitted until 2026-04-22 21:11:03.042063024 +0000 UTC m=+103.767959622 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/39daacd1-808b-4be7-95be-bccd8a0817e7-service-ca-bundle") pod "router-default-8594858796-49pkv" (UID: "39daacd1-808b-4be7-95be-bccd8a0817e7") : configmap references non-existent config key: service-ca.crt Apr 22 21:11:01.042320 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:01.042082 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Apr 22 21:11:01.042320 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:01.042118 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39daacd1-808b-4be7-95be-bccd8a0817e7-metrics-certs podName:39daacd1-808b-4be7-95be-bccd8a0817e7 nodeName:}" failed. No retries permitted until 2026-04-22 21:11:03.04210982 +0000 UTC m=+103.768006398 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/39daacd1-808b-4be7-95be-bccd8a0817e7-metrics-certs") pod "router-default-8594858796-49pkv" (UID: "39daacd1-808b-4be7-95be-bccd8a0817e7") : secret "router-metrics-certs-default" not found Apr 22 21:11:02.215995 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:02.215952 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-585dfdc468-gwzzw" event={"ID":"7d912a63-fac1-4f77-b588-e42de7a51799","Type":"ContainerStarted","Data":"2eb6d14b0e8ea85e575a7e4e5f66946a060e010ed445f49c4067e80b83ae5dba"} Apr 22 21:11:02.232893 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:02.232830 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-operator-585dfdc468-gwzzw" podStartSLOduration=1.332692059 podStartE2EDuration="3.232818006s" podCreationTimestamp="2026-04-22 21:10:59 +0000 UTC" firstStartedPulling="2026-04-22 21:10:59.911523333 +0000 UTC m=+100.637419913" lastFinishedPulling="2026-04-22 21:11:01.811649283 +0000 UTC m=+102.537545860" observedRunningTime="2026-04-22 21:11:02.232644116 +0000 UTC m=+102.958540734" watchObservedRunningTime="2026-04-22 21:11:02.232818006 +0000 UTC m=+102.958714606" Apr 22 21:11:03.058351 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:03.058323 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39daacd1-808b-4be7-95be-bccd8a0817e7-metrics-certs\") pod \"router-default-8594858796-49pkv\" (UID: \"39daacd1-808b-4be7-95be-bccd8a0817e7\") " pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:11:03.058512 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:03.058361 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39daacd1-808b-4be7-95be-bccd8a0817e7-service-ca-bundle\") pod \"router-default-8594858796-49pkv\" (UID: \"39daacd1-808b-4be7-95be-bccd8a0817e7\") " pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:11:03.058512 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:03.058501 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Apr 22 21:11:03.058580 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:03.058573 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39daacd1-808b-4be7-95be-bccd8a0817e7-metrics-certs podName:39daacd1-808b-4be7-95be-bccd8a0817e7 nodeName:}" failed. No retries permitted until 2026-04-22 21:11:07.058551516 +0000 UTC m=+107.784448095 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/39daacd1-808b-4be7-95be-bccd8a0817e7-metrics-certs") pod "router-default-8594858796-49pkv" (UID: "39daacd1-808b-4be7-95be-bccd8a0817e7") : secret "router-metrics-certs-default" not found Apr 22 21:11:03.058620 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:03.058587 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/39daacd1-808b-4be7-95be-bccd8a0817e7-service-ca-bundle podName:39daacd1-808b-4be7-95be-bccd8a0817e7 nodeName:}" failed. No retries permitted until 2026-04-22 21:11:07.058581084 +0000 UTC m=+107.784477663 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/39daacd1-808b-4be7-95be-bccd8a0817e7-service-ca-bundle") pod "router-default-8594858796-49pkv" (UID: "39daacd1-808b-4be7-95be-bccd8a0817e7") : configmap references non-existent config key: service-ca.crt Apr 22 21:11:04.932722 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:04.932695 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-49dpb_9e4ebb4a-2ec6-41ad-ac25-a5d1d100711f/dns-node-resolver/0.log" Apr 22 21:11:05.933296 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:05.933275 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wkk24_13fe92d8-fa95-45da-8e47-5b754f77385c/node-ca/0.log" Apr 22 21:11:07.089505 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:07.089476 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39daacd1-808b-4be7-95be-bccd8a0817e7-metrics-certs\") pod \"router-default-8594858796-49pkv\" (UID: \"39daacd1-808b-4be7-95be-bccd8a0817e7\") " pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:11:07.089870 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:07.089514 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39daacd1-808b-4be7-95be-bccd8a0817e7-service-ca-bundle\") pod \"router-default-8594858796-49pkv\" (UID: \"39daacd1-808b-4be7-95be-bccd8a0817e7\") " pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:11:07.089870 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:07.089623 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Apr 22 21:11:07.089870 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:07.089649 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/39daacd1-808b-4be7-95be-bccd8a0817e7-service-ca-bundle podName:39daacd1-808b-4be7-95be-bccd8a0817e7 nodeName:}" failed. No retries permitted until 2026-04-22 21:11:15.089635624 +0000 UTC m=+115.815532201 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/39daacd1-808b-4be7-95be-bccd8a0817e7-service-ca-bundle") pod "router-default-8594858796-49pkv" (UID: "39daacd1-808b-4be7-95be-bccd8a0817e7") : configmap references non-existent config key: service-ca.crt Apr 22 21:11:07.089870 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:07.089685 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39daacd1-808b-4be7-95be-bccd8a0817e7-metrics-certs podName:39daacd1-808b-4be7-95be-bccd8a0817e7 nodeName:}" failed. No retries permitted until 2026-04-22 21:11:15.089667381 +0000 UTC m=+115.815563978 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/39daacd1-808b-4be7-95be-bccd8a0817e7-metrics-certs") pod "router-default-8594858796-49pkv" (UID: "39daacd1-808b-4be7-95be-bccd8a0817e7") : secret "router-metrics-certs-default" not found Apr 22 21:11:09.406701 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.406669 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-jhsqb"] Apr 22 21:11:09.408525 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.408510 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-jhsqb" Apr 22 21:11:09.411270 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.411240 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-wv66h\"" Apr 22 21:11:09.411270 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.411261 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Apr 22 21:11:09.411401 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.411358 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Apr 22 21:11:09.415947 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.415925 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-jhsqb"] Apr 22 21:11:09.508629 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.508596 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-d6fc45fc5-xf75v"] Apr 22 21:11:09.510388 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.510372 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-xf75v" Apr 22 21:11:09.512135 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.512109 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkgw5\" (UniqueName: \"kubernetes.io/projected/aa9a495b-78dc-4501-9952-5b4900126489-kube-api-access-bkgw5\") pod \"volume-data-source-validator-7c6cbb6c87-jhsqb\" (UID: \"aa9a495b-78dc-4501-9952-5b4900126489\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-jhsqb" Apr 22 21:11:09.513054 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.513028 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Apr 22 21:11:09.513183 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.513075 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Apr 22 21:11:09.513183 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.513133 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Apr 22 21:11:09.513183 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.513169 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-fzj6q\"" Apr 22 21:11:09.513406 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.513390 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Apr 22 21:11:09.519460 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.519439 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-d6fc45fc5-xf75v"] Apr 22 21:11:09.611911 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.611884 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-j45pb"] Apr 22 21:11:09.613317 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.613296 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bkgw5\" (UniqueName: \"kubernetes.io/projected/aa9a495b-78dc-4501-9952-5b4900126489-kube-api-access-bkgw5\") pod \"volume-data-source-validator-7c6cbb6c87-jhsqb\" (UID: \"aa9a495b-78dc-4501-9952-5b4900126489\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-jhsqb" Apr 22 21:11:09.613442 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.613365 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh87f\" (UniqueName: \"kubernetes.io/projected/a292639d-6620-4540-8ee0-322ee00ff2f7-kube-api-access-vh87f\") pod \"service-ca-operator-d6fc45fc5-xf75v\" (UID: \"a292639d-6620-4540-8ee0-322ee00ff2f7\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-xf75v" Apr 22 21:11:09.613442 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.613432 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a292639d-6620-4540-8ee0-322ee00ff2f7-config\") pod \"service-ca-operator-d6fc45fc5-xf75v\" (UID: \"a292639d-6620-4540-8ee0-322ee00ff2f7\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-xf75v" Apr 22 21:11:09.613536 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.613485 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a292639d-6620-4540-8ee0-322ee00ff2f7-serving-cert\") pod \"service-ca-operator-d6fc45fc5-xf75v\" (UID: \"a292639d-6620-4540-8ee0-322ee00ff2f7\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-xf75v" Apr 22 21:11:09.613536 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.613496 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-j45pb" Apr 22 21:11:09.616093 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.616063 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Apr 22 21:11:09.616093 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.616101 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Apr 22 21:11:09.616350 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.616332 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-2qzw9\"" Apr 22 21:11:09.616505 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.616489 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Apr 22 21:11:09.616621 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.616606 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Apr 22 21:11:09.622989 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.622967 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-j45pb"] Apr 22 21:11:09.629566 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.629545 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkgw5\" (UniqueName: \"kubernetes.io/projected/aa9a495b-78dc-4501-9952-5b4900126489-kube-api-access-bkgw5\") pod \"volume-data-source-validator-7c6cbb6c87-jhsqb\" (UID: \"aa9a495b-78dc-4501-9952-5b4900126489\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-jhsqb" Apr 22 21:11:09.714199 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.714113 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdjtd\" (UniqueName: \"kubernetes.io/projected/dcdefc0e-dace-496e-bccf-de663633bec9-kube-api-access-jdjtd\") pod \"kube-storage-version-migrator-operator-6769c5d45-j45pb\" (UID: \"dcdefc0e-dace-496e-bccf-de663633bec9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-j45pb" Apr 22 21:11:09.714199 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.714181 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dcdefc0e-dace-496e-bccf-de663633bec9-serving-cert\") pod \"kube-storage-version-migrator-operator-6769c5d45-j45pb\" (UID: \"dcdefc0e-dace-496e-bccf-de663633bec9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-j45pb" Apr 22 21:11:09.714382 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.714239 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vh87f\" (UniqueName: \"kubernetes.io/projected/a292639d-6620-4540-8ee0-322ee00ff2f7-kube-api-access-vh87f\") pod \"service-ca-operator-d6fc45fc5-xf75v\" (UID: \"a292639d-6620-4540-8ee0-322ee00ff2f7\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-xf75v" Apr 22 21:11:09.714382 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.714299 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a292639d-6620-4540-8ee0-322ee00ff2f7-config\") pod \"service-ca-operator-d6fc45fc5-xf75v\" (UID: \"a292639d-6620-4540-8ee0-322ee00ff2f7\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-xf75v" Apr 22 21:11:09.714382 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.714330 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dcdefc0e-dace-496e-bccf-de663633bec9-config\") pod \"kube-storage-version-migrator-operator-6769c5d45-j45pb\" (UID: \"dcdefc0e-dace-496e-bccf-de663633bec9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-j45pb" Apr 22 21:11:09.714382 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.714358 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a292639d-6620-4540-8ee0-322ee00ff2f7-serving-cert\") pod \"service-ca-operator-d6fc45fc5-xf75v\" (UID: \"a292639d-6620-4540-8ee0-322ee00ff2f7\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-xf75v" Apr 22 21:11:09.715558 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.715535 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a292639d-6620-4540-8ee0-322ee00ff2f7-config\") pod \"service-ca-operator-d6fc45fc5-xf75v\" (UID: \"a292639d-6620-4540-8ee0-322ee00ff2f7\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-xf75v" Apr 22 21:11:09.716798 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.716777 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a292639d-6620-4540-8ee0-322ee00ff2f7-serving-cert\") pod \"service-ca-operator-d6fc45fc5-xf75v\" (UID: \"a292639d-6620-4540-8ee0-322ee00ff2f7\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-xf75v" Apr 22 21:11:09.717765 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.717745 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-jhsqb" Apr 22 21:11:09.722720 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.722703 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh87f\" (UniqueName: \"kubernetes.io/projected/a292639d-6620-4540-8ee0-322ee00ff2f7-kube-api-access-vh87f\") pod \"service-ca-operator-d6fc45fc5-xf75v\" (UID: \"a292639d-6620-4540-8ee0-322ee00ff2f7\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-xf75v" Apr 22 21:11:09.815372 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.815344 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dcdefc0e-dace-496e-bccf-de663633bec9-config\") pod \"kube-storage-version-migrator-operator-6769c5d45-j45pb\" (UID: \"dcdefc0e-dace-496e-bccf-de663633bec9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-j45pb" Apr 22 21:11:09.815506 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.815404 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jdjtd\" (UniqueName: \"kubernetes.io/projected/dcdefc0e-dace-496e-bccf-de663633bec9-kube-api-access-jdjtd\") pod \"kube-storage-version-migrator-operator-6769c5d45-j45pb\" (UID: \"dcdefc0e-dace-496e-bccf-de663633bec9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-j45pb" Apr 22 21:11:09.815506 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.815444 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dcdefc0e-dace-496e-bccf-de663633bec9-serving-cert\") pod \"kube-storage-version-migrator-operator-6769c5d45-j45pb\" (UID: \"dcdefc0e-dace-496e-bccf-de663633bec9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-j45pb" Apr 22 21:11:09.815908 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.815883 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dcdefc0e-dace-496e-bccf-de663633bec9-config\") pod \"kube-storage-version-migrator-operator-6769c5d45-j45pb\" (UID: \"dcdefc0e-dace-496e-bccf-de663633bec9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-j45pb" Apr 22 21:11:09.818100 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.818082 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dcdefc0e-dace-496e-bccf-de663633bec9-serving-cert\") pod \"kube-storage-version-migrator-operator-6769c5d45-j45pb\" (UID: \"dcdefc0e-dace-496e-bccf-de663633bec9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-j45pb" Apr 22 21:11:09.819098 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.819074 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-xf75v" Apr 22 21:11:09.824384 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.824359 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdjtd\" (UniqueName: \"kubernetes.io/projected/dcdefc0e-dace-496e-bccf-de663633bec9-kube-api-access-jdjtd\") pod \"kube-storage-version-migrator-operator-6769c5d45-j45pb\" (UID: \"dcdefc0e-dace-496e-bccf-de663633bec9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-j45pb" Apr 22 21:11:09.832442 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.832420 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-jhsqb"] Apr 22 21:11:09.836159 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:11:09.836135 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa9a495b_78dc_4501_9952_5b4900126489.slice/crio-f292c7012ed9858d865db883013d6ecd9ae76a9824bc35556df20481dcae7fc4 WatchSource:0}: Error finding container f292c7012ed9858d865db883013d6ecd9ae76a9824bc35556df20481dcae7fc4: Status 404 returned error can't find the container with id f292c7012ed9858d865db883013d6ecd9ae76a9824bc35556df20481dcae7fc4 Apr 22 21:11:09.922269 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.922243 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-j45pb" Apr 22 21:11:09.932954 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:09.932929 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-d6fc45fc5-xf75v"] Apr 22 21:11:09.936207 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:11:09.936178 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda292639d_6620_4540_8ee0_322ee00ff2f7.slice/crio-00a5c4765bca922af9506d43abdcfdf4a1f84a426ee262a4e5077c12629a6ba0 WatchSource:0}: Error finding container 00a5c4765bca922af9506d43abdcfdf4a1f84a426ee262a4e5077c12629a6ba0: Status 404 returned error can't find the container with id 00a5c4765bca922af9506d43abdcfdf4a1f84a426ee262a4e5077c12629a6ba0 Apr 22 21:11:10.035778 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:10.035750 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-j45pb"] Apr 22 21:11:10.038823 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:11:10.038798 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcdefc0e_dace_496e_bccf_de663633bec9.slice/crio-90de03bd27fcb4489bbe35d9a0308e3fd8f2452b829007e569f1c438136e53cb WatchSource:0}: Error finding container 90de03bd27fcb4489bbe35d9a0308e3fd8f2452b829007e569f1c438136e53cb: Status 404 returned error can't find the container with id 90de03bd27fcb4489bbe35d9a0308e3fd8f2452b829007e569f1c438136e53cb Apr 22 21:11:10.232702 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:10.232605 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-j45pb" event={"ID":"dcdefc0e-dace-496e-bccf-de663633bec9","Type":"ContainerStarted","Data":"90de03bd27fcb4489bbe35d9a0308e3fd8f2452b829007e569f1c438136e53cb"} Apr 22 21:11:10.233584 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:10.233560 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-jhsqb" event={"ID":"aa9a495b-78dc-4501-9952-5b4900126489","Type":"ContainerStarted","Data":"f292c7012ed9858d865db883013d6ecd9ae76a9824bc35556df20481dcae7fc4"} Apr 22 21:11:10.234505 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:10.234485 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-xf75v" event={"ID":"a292639d-6620-4540-8ee0-322ee00ff2f7","Type":"ContainerStarted","Data":"00a5c4765bca922af9506d43abdcfdf4a1f84a426ee262a4e5077c12629a6ba0"} Apr 22 21:11:12.240829 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:12.240791 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-jhsqb" event={"ID":"aa9a495b-78dc-4501-9952-5b4900126489","Type":"ContainerStarted","Data":"5a73babddcc4c5691292b632171c053a742f7ca5fd051cd1d7a0cf003ebc0bbc"} Apr 22 21:11:12.258323 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:12.257040 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-jhsqb" podStartSLOduration=1.867991358 podStartE2EDuration="3.257021348s" podCreationTimestamp="2026-04-22 21:11:09 +0000 UTC" firstStartedPulling="2026-04-22 21:11:09.837937654 +0000 UTC m=+110.563834250" lastFinishedPulling="2026-04-22 21:11:11.226967659 +0000 UTC m=+111.952864240" observedRunningTime="2026-04-22 21:11:12.255978312 +0000 UTC m=+112.981874933" watchObservedRunningTime="2026-04-22 21:11:12.257021348 +0000 UTC m=+112.982917953" Apr 22 21:11:13.245595 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:13.245551 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-xf75v" event={"ID":"a292639d-6620-4540-8ee0-322ee00ff2f7","Type":"ContainerStarted","Data":"f71e721c0db825e39e456b87ce58c70d790362802bc2977b7ae33011eab2a0b4"} Apr 22 21:11:13.246983 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:13.246955 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-j45pb" event={"ID":"dcdefc0e-dace-496e-bccf-de663633bec9","Type":"ContainerStarted","Data":"6f5e2bdd58762749e3e7025c40cd5cde976256abcf4d9039bd1e489378bc286f"} Apr 22 21:11:13.260054 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:13.260015 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-xf75v" podStartSLOduration=1.894305531 podStartE2EDuration="4.260004682s" podCreationTimestamp="2026-04-22 21:11:09 +0000 UTC" firstStartedPulling="2026-04-22 21:11:09.938056421 +0000 UTC m=+110.663953003" lastFinishedPulling="2026-04-22 21:11:12.303755574 +0000 UTC m=+113.029652154" observedRunningTime="2026-04-22 21:11:13.259315587 +0000 UTC m=+113.985212191" watchObservedRunningTime="2026-04-22 21:11:13.260004682 +0000 UTC m=+113.985901282" Apr 22 21:11:15.161438 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:15.161403 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39daacd1-808b-4be7-95be-bccd8a0817e7-metrics-certs\") pod \"router-default-8594858796-49pkv\" (UID: \"39daacd1-808b-4be7-95be-bccd8a0817e7\") " pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:11:15.161438 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:15.161444 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39daacd1-808b-4be7-95be-bccd8a0817e7-service-ca-bundle\") pod \"router-default-8594858796-49pkv\" (UID: \"39daacd1-808b-4be7-95be-bccd8a0817e7\") " pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:11:15.161821 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:15.161562 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Apr 22 21:11:15.161821 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:15.161571 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/39daacd1-808b-4be7-95be-bccd8a0817e7-service-ca-bundle podName:39daacd1-808b-4be7-95be-bccd8a0817e7 nodeName:}" failed. No retries permitted until 2026-04-22 21:11:31.161557859 +0000 UTC m=+131.887454436 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/39daacd1-808b-4be7-95be-bccd8a0817e7-service-ca-bundle") pod "router-default-8594858796-49pkv" (UID: "39daacd1-808b-4be7-95be-bccd8a0817e7") : configmap references non-existent config key: service-ca.crt Apr 22 21:11:15.161821 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:15.161626 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39daacd1-808b-4be7-95be-bccd8a0817e7-metrics-certs podName:39daacd1-808b-4be7-95be-bccd8a0817e7 nodeName:}" failed. No retries permitted until 2026-04-22 21:11:31.161612065 +0000 UTC m=+131.887508642 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/39daacd1-808b-4be7-95be-bccd8a0817e7-metrics-certs") pod "router-default-8594858796-49pkv" (UID: "39daacd1-808b-4be7-95be-bccd8a0817e7") : secret "router-metrics-certs-default" not found Apr 22 21:11:15.999646 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:15.999596 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-j45pb" podStartSLOduration=4.735367028 podStartE2EDuration="6.999580893s" podCreationTimestamp="2026-04-22 21:11:09 +0000 UTC" firstStartedPulling="2026-04-22 21:11:10.040606645 +0000 UTC m=+110.766503222" lastFinishedPulling="2026-04-22 21:11:12.304820509 +0000 UTC m=+113.030717087" observedRunningTime="2026-04-22 21:11:13.274601178 +0000 UTC m=+114.000497806" watchObservedRunningTime="2026-04-22 21:11:15.999580893 +0000 UTC m=+116.725477533" Apr 22 21:11:15.999890 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:15.999875 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-865cb79987-2rt46"] Apr 22 21:11:16.001872 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.001829 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-865cb79987-2rt46" Apr 22 21:11:16.004570 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.004547 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Apr 22 21:11:16.005756 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.005738 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-6f5g8\"" Apr 22 21:11:16.005897 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.005738 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Apr 22 21:11:16.005897 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.005739 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Apr 22 21:11:16.005897 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.005797 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Apr 22 21:11:16.008664 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.008645 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-865cb79987-2rt46"] Apr 22 21:11:16.070152 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.070121 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b4e8a913-cc63-4959-88da-4dbe43062247-signing-key\") pod \"service-ca-865cb79987-2rt46\" (UID: \"b4e8a913-cc63-4959-88da-4dbe43062247\") " pod="openshift-service-ca/service-ca-865cb79987-2rt46" Apr 22 21:11:16.070313 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.070287 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdw2k\" (UniqueName: \"kubernetes.io/projected/b4e8a913-cc63-4959-88da-4dbe43062247-kube-api-access-jdw2k\") pod \"service-ca-865cb79987-2rt46\" (UID: \"b4e8a913-cc63-4959-88da-4dbe43062247\") " pod="openshift-service-ca/service-ca-865cb79987-2rt46" Apr 22 21:11:16.070348 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.070336 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b4e8a913-cc63-4959-88da-4dbe43062247-signing-cabundle\") pod \"service-ca-865cb79987-2rt46\" (UID: \"b4e8a913-cc63-4959-88da-4dbe43062247\") " pod="openshift-service-ca/service-ca-865cb79987-2rt46" Apr 22 21:11:16.171460 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.171421 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b4e8a913-cc63-4959-88da-4dbe43062247-signing-cabundle\") pod \"service-ca-865cb79987-2rt46\" (UID: \"b4e8a913-cc63-4959-88da-4dbe43062247\") " pod="openshift-service-ca/service-ca-865cb79987-2rt46" Apr 22 21:11:16.171894 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.171476 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b4e8a913-cc63-4959-88da-4dbe43062247-signing-key\") pod \"service-ca-865cb79987-2rt46\" (UID: \"b4e8a913-cc63-4959-88da-4dbe43062247\") " pod="openshift-service-ca/service-ca-865cb79987-2rt46" Apr 22 21:11:16.171894 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.171534 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jdw2k\" (UniqueName: \"kubernetes.io/projected/b4e8a913-cc63-4959-88da-4dbe43062247-kube-api-access-jdw2k\") pod \"service-ca-865cb79987-2rt46\" (UID: \"b4e8a913-cc63-4959-88da-4dbe43062247\") " pod="openshift-service-ca/service-ca-865cb79987-2rt46" Apr 22 21:11:16.172080 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.172061 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b4e8a913-cc63-4959-88da-4dbe43062247-signing-cabundle\") pod \"service-ca-865cb79987-2rt46\" (UID: \"b4e8a913-cc63-4959-88da-4dbe43062247\") " pod="openshift-service-ca/service-ca-865cb79987-2rt46" Apr 22 21:11:16.173940 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.173918 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b4e8a913-cc63-4959-88da-4dbe43062247-signing-key\") pod \"service-ca-865cb79987-2rt46\" (UID: \"b4e8a913-cc63-4959-88da-4dbe43062247\") " pod="openshift-service-ca/service-ca-865cb79987-2rt46" Apr 22 21:11:16.179925 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.179902 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdw2k\" (UniqueName: \"kubernetes.io/projected/b4e8a913-cc63-4959-88da-4dbe43062247-kube-api-access-jdw2k\") pod \"service-ca-865cb79987-2rt46\" (UID: \"b4e8a913-cc63-4959-88da-4dbe43062247\") " pod="openshift-service-ca/service-ca-865cb79987-2rt46" Apr 22 21:11:16.311399 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.311319 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-865cb79987-2rt46" Apr 22 21:11:16.369742 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.369176 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5d2jm"] Apr 22 21:11:16.372271 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.372244 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5d2jm" Apr 22 21:11:16.375194 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.375169 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Apr 22 21:11:16.375387 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.375347 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rwlrc\"" Apr 22 21:11:16.375538 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.375407 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Apr 22 21:11:16.380743 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.380724 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5d2jm"] Apr 22 21:11:16.430378 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.430352 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-865cb79987-2rt46"] Apr 22 21:11:16.432960 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:11:16.432933 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4e8a913_cc63_4959_88da_4dbe43062247.slice/crio-6b7ab91b8eabf65e487e9802c5a029458b1f0ac39dfe09c061815c0faaa600e8 WatchSource:0}: Error finding container 6b7ab91b8eabf65e487e9802c5a029458b1f0ac39dfe09c061815c0faaa600e8: Status 404 returned error can't find the container with id 6b7ab91b8eabf65e487e9802c5a029458b1f0ac39dfe09c061815c0faaa600e8 Apr 22 21:11:16.473710 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.473689 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f1048e6f-786c-4048-a184-b4e8454b570d-crio-socket\") pod \"insights-runtime-extractor-5d2jm\" (UID: \"f1048e6f-786c-4048-a184-b4e8454b570d\") " pod="openshift-insights/insights-runtime-extractor-5d2jm" Apr 22 21:11:16.473826 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.473717 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1048e6f-786c-4048-a184-b4e8454b570d-data-volume\") pod \"insights-runtime-extractor-5d2jm\" (UID: \"f1048e6f-786c-4048-a184-b4e8454b570d\") " pod="openshift-insights/insights-runtime-extractor-5d2jm" Apr 22 21:11:16.473826 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.473736 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6njfs\" (UniqueName: \"kubernetes.io/projected/f1048e6f-786c-4048-a184-b4e8454b570d-kube-api-access-6njfs\") pod \"insights-runtime-extractor-5d2jm\" (UID: \"f1048e6f-786c-4048-a184-b4e8454b570d\") " pod="openshift-insights/insights-runtime-extractor-5d2jm" Apr 22 21:11:16.473826 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.473755 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1048e6f-786c-4048-a184-b4e8454b570d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5d2jm\" (UID: \"f1048e6f-786c-4048-a184-b4e8454b570d\") " pod="openshift-insights/insights-runtime-extractor-5d2jm" Apr 22 21:11:16.473955 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.473875 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1048e6f-786c-4048-a184-b4e8454b570d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5d2jm\" (UID: \"f1048e6f-786c-4048-a184-b4e8454b570d\") " pod="openshift-insights/insights-runtime-extractor-5d2jm" Apr 22 21:11:16.575141 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.575061 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f1048e6f-786c-4048-a184-b4e8454b570d-crio-socket\") pod \"insights-runtime-extractor-5d2jm\" (UID: \"f1048e6f-786c-4048-a184-b4e8454b570d\") " pod="openshift-insights/insights-runtime-extractor-5d2jm" Apr 22 21:11:16.575141 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.575099 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1048e6f-786c-4048-a184-b4e8454b570d-data-volume\") pod \"insights-runtime-extractor-5d2jm\" (UID: \"f1048e6f-786c-4048-a184-b4e8454b570d\") " pod="openshift-insights/insights-runtime-extractor-5d2jm" Apr 22 21:11:16.575141 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.575127 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6njfs\" (UniqueName: \"kubernetes.io/projected/f1048e6f-786c-4048-a184-b4e8454b570d-kube-api-access-6njfs\") pod \"insights-runtime-extractor-5d2jm\" (UID: \"f1048e6f-786c-4048-a184-b4e8454b570d\") " pod="openshift-insights/insights-runtime-extractor-5d2jm" Apr 22 21:11:16.575402 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.575150 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1048e6f-786c-4048-a184-b4e8454b570d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5d2jm\" (UID: \"f1048e6f-786c-4048-a184-b4e8454b570d\") " pod="openshift-insights/insights-runtime-extractor-5d2jm" Apr 22 21:11:16.575402 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.575161 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f1048e6f-786c-4048-a184-b4e8454b570d-crio-socket\") pod \"insights-runtime-extractor-5d2jm\" (UID: \"f1048e6f-786c-4048-a184-b4e8454b570d\") " pod="openshift-insights/insights-runtime-extractor-5d2jm" Apr 22 21:11:16.575402 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.575235 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1048e6f-786c-4048-a184-b4e8454b570d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5d2jm\" (UID: \"f1048e6f-786c-4048-a184-b4e8454b570d\") " pod="openshift-insights/insights-runtime-extractor-5d2jm" Apr 22 21:11:16.575402 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:16.575277 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Apr 22 21:11:16.575402 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:16.575342 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1048e6f-786c-4048-a184-b4e8454b570d-insights-runtime-extractor-tls podName:f1048e6f-786c-4048-a184-b4e8454b570d nodeName:}" failed. No retries permitted until 2026-04-22 21:11:17.075325643 +0000 UTC m=+117.801222222 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f1048e6f-786c-4048-a184-b4e8454b570d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5d2jm" (UID: "f1048e6f-786c-4048-a184-b4e8454b570d") : secret "insights-runtime-extractor-tls" not found Apr 22 21:11:16.575935 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.575917 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1048e6f-786c-4048-a184-b4e8454b570d-data-volume\") pod \"insights-runtime-extractor-5d2jm\" (UID: \"f1048e6f-786c-4048-a184-b4e8454b570d\") " pod="openshift-insights/insights-runtime-extractor-5d2jm" Apr 22 21:11:16.576152 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.576136 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1048e6f-786c-4048-a184-b4e8454b570d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5d2jm\" (UID: \"f1048e6f-786c-4048-a184-b4e8454b570d\") " pod="openshift-insights/insights-runtime-extractor-5d2jm" Apr 22 21:11:16.593479 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:16.593458 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6njfs\" (UniqueName: \"kubernetes.io/projected/f1048e6f-786c-4048-a184-b4e8454b570d-kube-api-access-6njfs\") pod \"insights-runtime-extractor-5d2jm\" (UID: \"f1048e6f-786c-4048-a184-b4e8454b570d\") " pod="openshift-insights/insights-runtime-extractor-5d2jm" Apr 22 21:11:17.079014 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:17.078972 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1048e6f-786c-4048-a184-b4e8454b570d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5d2jm\" (UID: \"f1048e6f-786c-4048-a184-b4e8454b570d\") " pod="openshift-insights/insights-runtime-extractor-5d2jm" Apr 22 21:11:17.079255 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:17.079180 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Apr 22 21:11:17.079321 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:17.079279 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1048e6f-786c-4048-a184-b4e8454b570d-insights-runtime-extractor-tls podName:f1048e6f-786c-4048-a184-b4e8454b570d nodeName:}" failed. No retries permitted until 2026-04-22 21:11:18.079251044 +0000 UTC m=+118.805147624 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f1048e6f-786c-4048-a184-b4e8454b570d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5d2jm" (UID: "f1048e6f-786c-4048-a184-b4e8454b570d") : secret "insights-runtime-extractor-tls" not found Apr 22 21:11:17.257745 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:17.257707 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-865cb79987-2rt46" event={"ID":"b4e8a913-cc63-4959-88da-4dbe43062247","Type":"ContainerStarted","Data":"f235fbe28af07fb08af5d0398a789090f56aa7cab373aeb1ac138a82a5c3899c"} Apr 22 21:11:17.257745 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:17.257747 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-865cb79987-2rt46" event={"ID":"b4e8a913-cc63-4959-88da-4dbe43062247","Type":"ContainerStarted","Data":"6b7ab91b8eabf65e487e9802c5a029458b1f0ac39dfe09c061815c0faaa600e8"} Apr 22 21:11:17.273034 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:17.272993 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-865cb79987-2rt46" podStartSLOduration=2.272980336 podStartE2EDuration="2.272980336s" podCreationTimestamp="2026-04-22 21:11:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-22 21:11:17.272389444 +0000 UTC m=+117.998286046" watchObservedRunningTime="2026-04-22 21:11:17.272980336 +0000 UTC m=+117.998876936" Apr 22 21:11:18.088514 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:18.088485 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1048e6f-786c-4048-a184-b4e8454b570d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5d2jm\" (UID: \"f1048e6f-786c-4048-a184-b4e8454b570d\") " pod="openshift-insights/insights-runtime-extractor-5d2jm" Apr 22 21:11:18.088677 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:18.088594 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Apr 22 21:11:18.088677 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:18.088667 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1048e6f-786c-4048-a184-b4e8454b570d-insights-runtime-extractor-tls podName:f1048e6f-786c-4048-a184-b4e8454b570d nodeName:}" failed. No retries permitted until 2026-04-22 21:11:20.088652308 +0000 UTC m=+120.814548886 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f1048e6f-786c-4048-a184-b4e8454b570d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5d2jm" (UID: "f1048e6f-786c-4048-a184-b4e8454b570d") : secret "insights-runtime-extractor-tls" not found Apr 22 21:11:20.104815 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:20.104771 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1048e6f-786c-4048-a184-b4e8454b570d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5d2jm\" (UID: \"f1048e6f-786c-4048-a184-b4e8454b570d\") " pod="openshift-insights/insights-runtime-extractor-5d2jm" Apr 22 21:11:20.105189 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:20.104948 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Apr 22 21:11:20.105189 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:20.105032 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1048e6f-786c-4048-a184-b4e8454b570d-insights-runtime-extractor-tls podName:f1048e6f-786c-4048-a184-b4e8454b570d nodeName:}" failed. No retries permitted until 2026-04-22 21:11:24.105009128 +0000 UTC m=+124.830905710 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f1048e6f-786c-4048-a184-b4e8454b570d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5d2jm" (UID: "f1048e6f-786c-4048-a184-b4e8454b570d") : secret "insights-runtime-extractor-tls" not found Apr 22 21:11:24.139553 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:24.139504 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1048e6f-786c-4048-a184-b4e8454b570d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5d2jm\" (UID: \"f1048e6f-786c-4048-a184-b4e8454b570d\") " pod="openshift-insights/insights-runtime-extractor-5d2jm" Apr 22 21:11:24.141906 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:24.141884 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1048e6f-786c-4048-a184-b4e8454b570d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5d2jm\" (UID: \"f1048e6f-786c-4048-a184-b4e8454b570d\") " pod="openshift-insights/insights-runtime-extractor-5d2jm" Apr 22 21:11:24.185626 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:24.185598 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rwlrc\"" Apr 22 21:11:24.193882 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:24.193843 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5d2jm" Apr 22 21:11:24.314422 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:24.314392 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5d2jm"] Apr 22 21:11:24.317902 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:11:24.317839 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1048e6f_786c_4048_a184_b4e8454b570d.slice/crio-bfc82ecc73e6d20de9947acaa26de633f28aadd979a54ded007c0876752b0658 WatchSource:0}: Error finding container bfc82ecc73e6d20de9947acaa26de633f28aadd979a54ded007c0876752b0658: Status 404 returned error can't find the container with id bfc82ecc73e6d20de9947acaa26de633f28aadd979a54ded007c0876752b0658 Apr 22 21:11:25.284030 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:25.283955 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5d2jm" event={"ID":"f1048e6f-786c-4048-a184-b4e8454b570d","Type":"ContainerStarted","Data":"82e0ed7084b89a5fedf29ddb3f28f5b1dff3cb009fa82716655254fb63350380"} Apr 22 21:11:25.284030 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:25.283991 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5d2jm" event={"ID":"f1048e6f-786c-4048-a184-b4e8454b570d","Type":"ContainerStarted","Data":"fc89689246691979c42f019cc1a7b46b8e2a6d4b03e8519ee683ed36d57076de"} Apr 22 21:11:25.284030 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:25.284001 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5d2jm" event={"ID":"f1048e6f-786c-4048-a184-b4e8454b570d","Type":"ContainerStarted","Data":"bfc82ecc73e6d20de9947acaa26de633f28aadd979a54ded007c0876752b0658"} Apr 22 21:11:27.290231 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:27.290193 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5d2jm" event={"ID":"f1048e6f-786c-4048-a184-b4e8454b570d","Type":"ContainerStarted","Data":"331b1931d00158252408a99920a0c0d8802a0b6b2e1cc9e21f30ea9ac27f129b"} Apr 22 21:11:27.307775 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:27.307702 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5d2jm" podStartSLOduration=9.351195689 podStartE2EDuration="11.307690634s" podCreationTimestamp="2026-04-22 21:11:16 +0000 UTC" firstStartedPulling="2026-04-22 21:11:24.367288538 +0000 UTC m=+125.093185119" lastFinishedPulling="2026-04-22 21:11:26.323783468 +0000 UTC m=+127.049680064" observedRunningTime="2026-04-22 21:11:27.307073624 +0000 UTC m=+128.032970243" watchObservedRunningTime="2026-04-22 21:11:27.307690634 +0000 UTC m=+128.033587235" Apr 22 21:11:29.687453 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:29.687412 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/843da596-cc50-4a3e-817c-4ee63a89c94a-metrics-certs\") pod \"network-metrics-daemon-fx8k2\" (UID: \"843da596-cc50-4a3e-817c-4ee63a89c94a\") " pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:11:29.689921 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:29.689899 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/843da596-cc50-4a3e-817c-4ee63a89c94a-metrics-certs\") pod \"network-metrics-daemon-fx8k2\" (UID: \"843da596-cc50-4a3e-817c-4ee63a89c94a\") " pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:11:29.975630 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:29.975545 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bw4nx\"" Apr 22 21:11:29.982705 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:29.982685 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fx8k2" Apr 22 21:11:30.098482 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:30.098454 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fx8k2"] Apr 22 21:11:30.101313 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:11:30.101284 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod843da596_cc50_4a3e_817c_4ee63a89c94a.slice/crio-c78bb89dc6fb70d4c64cca73b59a706b9586c10fb9b52c2f1b8995136d5be93e WatchSource:0}: Error finding container c78bb89dc6fb70d4c64cca73b59a706b9586c10fb9b52c2f1b8995136d5be93e: Status 404 returned error can't find the container with id c78bb89dc6fb70d4c64cca73b59a706b9586c10fb9b52c2f1b8995136d5be93e Apr 22 21:11:30.298337 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:30.298242 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fx8k2" event={"ID":"843da596-cc50-4a3e-817c-4ee63a89c94a","Type":"ContainerStarted","Data":"c78bb89dc6fb70d4c64cca73b59a706b9586c10fb9b52c2f1b8995136d5be93e"} Apr 22 21:11:31.199536 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:31.199511 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39daacd1-808b-4be7-95be-bccd8a0817e7-service-ca-bundle\") pod \"router-default-8594858796-49pkv\" (UID: \"39daacd1-808b-4be7-95be-bccd8a0817e7\") " pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:11:31.199928 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:31.199584 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39daacd1-808b-4be7-95be-bccd8a0817e7-metrics-certs\") pod \"router-default-8594858796-49pkv\" (UID: \"39daacd1-808b-4be7-95be-bccd8a0817e7\") " pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:11:31.200049 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:31.200032 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39daacd1-808b-4be7-95be-bccd8a0817e7-service-ca-bundle\") pod \"router-default-8594858796-49pkv\" (UID: \"39daacd1-808b-4be7-95be-bccd8a0817e7\") " pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:11:31.202483 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:31.202429 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39daacd1-808b-4be7-95be-bccd8a0817e7-metrics-certs\") pod \"router-default-8594858796-49pkv\" (UID: \"39daacd1-808b-4be7-95be-bccd8a0817e7\") " pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:11:31.303531 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:31.303447 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fx8k2" event={"ID":"843da596-cc50-4a3e-817c-4ee63a89c94a","Type":"ContainerStarted","Data":"6e584f91a5dc8edf6cb1beab9a21c54c05904a89bc73d6f32e6837d20e7e18fe"} Apr 22 21:11:31.303531 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:31.303485 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fx8k2" event={"ID":"843da596-cc50-4a3e-817c-4ee63a89c94a","Type":"ContainerStarted","Data":"252f7e626b2f4b60793a816a40a88e22122c3eca7df840ba5dcfd8ebb0e0aff4"} Apr 22 21:11:31.318112 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:31.318068 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fx8k2" podStartSLOduration=131.435323781 podStartE2EDuration="2m12.31805508s" podCreationTimestamp="2026-04-22 21:09:19 +0000 UTC" firstStartedPulling="2026-04-22 21:11:30.103219098 +0000 UTC m=+130.829115676" lastFinishedPulling="2026-04-22 21:11:30.985950377 +0000 UTC m=+131.711846975" observedRunningTime="2026-04-22 21:11:31.317656229 +0000 UTC m=+132.043552830" watchObservedRunningTime="2026-04-22 21:11:31.31805508 +0000 UTC m=+132.043951697" Apr 22 21:11:31.492982 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:31.492951 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-bqnqt\"" Apr 22 21:11:31.501149 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:31.501112 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:11:31.617518 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:31.617485 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-8594858796-49pkv"] Apr 22 21:11:31.620253 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:11:31.620223 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39daacd1_808b_4be7_95be_bccd8a0817e7.slice/crio-b021073af5fd65b8de1fc09d9ead33825ca7d569002bfb7a120f466fbd6b52bc WatchSource:0}: Error finding container b021073af5fd65b8de1fc09d9ead33825ca7d569002bfb7a120f466fbd6b52bc: Status 404 returned error can't find the container with id b021073af5fd65b8de1fc09d9ead33825ca7d569002bfb7a120f466fbd6b52bc Apr 22 21:11:32.307143 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:32.307106 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-8594858796-49pkv" event={"ID":"39daacd1-808b-4be7-95be-bccd8a0817e7","Type":"ContainerStarted","Data":"0a3cb0d041e5d6ccd4e9a8c39037df001bfaca290e473b19623908d8f9a8b532"} Apr 22 21:11:32.307143 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:32.307151 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-8594858796-49pkv" event={"ID":"39daacd1-808b-4be7-95be-bccd8a0817e7","Type":"ContainerStarted","Data":"b021073af5fd65b8de1fc09d9ead33825ca7d569002bfb7a120f466fbd6b52bc"} Apr 22 21:11:32.324349 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:32.324298 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/router-default-8594858796-49pkv" podStartSLOduration=33.324283109 podStartE2EDuration="33.324283109s" podCreationTimestamp="2026-04-22 21:10:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-22 21:11:32.32333357 +0000 UTC m=+133.049230169" watchObservedRunningTime="2026-04-22 21:11:32.324283109 +0000 UTC m=+133.050179708" Apr 22 21:11:32.501871 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:32.501828 2580 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:11:32.504409 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:32.504387 2580 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:11:33.309433 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:33.309405 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:11:33.310754 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:33.310738 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-8594858796-49pkv" Apr 22 21:11:35.733988 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.733962 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-gsv7g"] Apr 22 21:11:35.736260 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.736239 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-gsv7g" Apr 22 21:11:35.739036 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.739014 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-dx789\"" Apr 22 21:11:35.740390 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.740364 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Apr 22 21:11:35.741733 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.741711 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7db9c69bcc-wxqq6"] Apr 22 21:11:35.743818 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.743731 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-gsv7g"] Apr 22 21:11:35.743818 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.743799 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:11:35.746734 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.746717 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-j9t5d\"" Apr 22 21:11:35.746831 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.746738 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Apr 22 21:11:35.746831 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.746764 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Apr 22 21:11:35.746831 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.746804 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Apr 22 21:11:35.752263 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.752244 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Apr 22 21:11:35.756906 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.756886 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7db9c69bcc-wxqq6"] Apr 22 21:11:35.831506 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.831478 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0f2eedec-34a3-43cc-8586-ec247b1fd8bc-installation-pull-secrets\") pod \"image-registry-7db9c69bcc-wxqq6\" (UID: \"0f2eedec-34a3-43cc-8586-ec247b1fd8bc\") " pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:11:35.831506 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.831509 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbjmg\" (UniqueName: \"kubernetes.io/projected/0f2eedec-34a3-43cc-8586-ec247b1fd8bc-kube-api-access-jbjmg\") pod \"image-registry-7db9c69bcc-wxqq6\" (UID: \"0f2eedec-34a3-43cc-8586-ec247b1fd8bc\") " pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:11:35.831731 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.831533 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f2eedec-34a3-43cc-8586-ec247b1fd8bc-registry-tls\") pod \"image-registry-7db9c69bcc-wxqq6\" (UID: \"0f2eedec-34a3-43cc-8586-ec247b1fd8bc\") " pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:11:35.831731 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.831548 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0f2eedec-34a3-43cc-8586-ec247b1fd8bc-registry-certificates\") pod \"image-registry-7db9c69bcc-wxqq6\" (UID: \"0f2eedec-34a3-43cc-8586-ec247b1fd8bc\") " pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:11:35.831731 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.831627 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0f2eedec-34a3-43cc-8586-ec247b1fd8bc-trusted-ca\") pod \"image-registry-7db9c69bcc-wxqq6\" (UID: \"0f2eedec-34a3-43cc-8586-ec247b1fd8bc\") " pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:11:35.831731 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.831678 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a23ef821-2192-402d-91ae-c95fc61fad45-tls-certificates\") pod \"prometheus-operator-admission-webhook-57cf98b594-gsv7g\" (UID: \"a23ef821-2192-402d-91ae-c95fc61fad45\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-gsv7g" Apr 22 21:11:35.831731 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.831706 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0f2eedec-34a3-43cc-8586-ec247b1fd8bc-ca-trust-extracted\") pod \"image-registry-7db9c69bcc-wxqq6\" (UID: \"0f2eedec-34a3-43cc-8586-ec247b1fd8bc\") " pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:11:35.831731 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.831727 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f2eedec-34a3-43cc-8586-ec247b1fd8bc-bound-sa-token\") pod \"image-registry-7db9c69bcc-wxqq6\" (UID: \"0f2eedec-34a3-43cc-8586-ec247b1fd8bc\") " pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:11:35.831984 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.831759 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0f2eedec-34a3-43cc-8586-ec247b1fd8bc-image-registry-private-configuration\") pod \"image-registry-7db9c69bcc-wxqq6\" (UID: \"0f2eedec-34a3-43cc-8586-ec247b1fd8bc\") " pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:11:35.932804 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.932719 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0f2eedec-34a3-43cc-8586-ec247b1fd8bc-trusted-ca\") pod \"image-registry-7db9c69bcc-wxqq6\" (UID: \"0f2eedec-34a3-43cc-8586-ec247b1fd8bc\") " pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:11:35.932804 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.932769 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a23ef821-2192-402d-91ae-c95fc61fad45-tls-certificates\") pod \"prometheus-operator-admission-webhook-57cf98b594-gsv7g\" (UID: \"a23ef821-2192-402d-91ae-c95fc61fad45\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-gsv7g" Apr 22 21:11:35.932804 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.932791 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0f2eedec-34a3-43cc-8586-ec247b1fd8bc-ca-trust-extracted\") pod \"image-registry-7db9c69bcc-wxqq6\" (UID: \"0f2eedec-34a3-43cc-8586-ec247b1fd8bc\") " pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:11:35.933080 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.932811 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f2eedec-34a3-43cc-8586-ec247b1fd8bc-bound-sa-token\") pod \"image-registry-7db9c69bcc-wxqq6\" (UID: \"0f2eedec-34a3-43cc-8586-ec247b1fd8bc\") " pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:11:35.933080 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.932839 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0f2eedec-34a3-43cc-8586-ec247b1fd8bc-image-registry-private-configuration\") pod \"image-registry-7db9c69bcc-wxqq6\" (UID: \"0f2eedec-34a3-43cc-8586-ec247b1fd8bc\") " pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:11:35.933080 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.932879 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0f2eedec-34a3-43cc-8586-ec247b1fd8bc-installation-pull-secrets\") pod \"image-registry-7db9c69bcc-wxqq6\" (UID: \"0f2eedec-34a3-43cc-8586-ec247b1fd8bc\") " pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:11:35.933080 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.932895 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jbjmg\" (UniqueName: \"kubernetes.io/projected/0f2eedec-34a3-43cc-8586-ec247b1fd8bc-kube-api-access-jbjmg\") pod \"image-registry-7db9c69bcc-wxqq6\" (UID: \"0f2eedec-34a3-43cc-8586-ec247b1fd8bc\") " pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:11:35.933080 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.933019 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f2eedec-34a3-43cc-8586-ec247b1fd8bc-registry-tls\") pod \"image-registry-7db9c69bcc-wxqq6\" (UID: \"0f2eedec-34a3-43cc-8586-ec247b1fd8bc\") " pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:11:35.933080 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.933054 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0f2eedec-34a3-43cc-8586-ec247b1fd8bc-registry-certificates\") pod \"image-registry-7db9c69bcc-wxqq6\" (UID: \"0f2eedec-34a3-43cc-8586-ec247b1fd8bc\") " pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:11:35.933373 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.933285 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0f2eedec-34a3-43cc-8586-ec247b1fd8bc-ca-trust-extracted\") pod \"image-registry-7db9c69bcc-wxqq6\" (UID: \"0f2eedec-34a3-43cc-8586-ec247b1fd8bc\") " pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:11:35.933987 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.933963 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0f2eedec-34a3-43cc-8586-ec247b1fd8bc-trusted-ca\") pod \"image-registry-7db9c69bcc-wxqq6\" (UID: \"0f2eedec-34a3-43cc-8586-ec247b1fd8bc\") " pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:11:35.934103 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.934054 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0f2eedec-34a3-43cc-8586-ec247b1fd8bc-registry-certificates\") pod \"image-registry-7db9c69bcc-wxqq6\" (UID: \"0f2eedec-34a3-43cc-8586-ec247b1fd8bc\") " pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:11:35.935785 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.935760 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f2eedec-34a3-43cc-8586-ec247b1fd8bc-registry-tls\") pod \"image-registry-7db9c69bcc-wxqq6\" (UID: \"0f2eedec-34a3-43cc-8586-ec247b1fd8bc\") " pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:11:35.935909 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.935875 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a23ef821-2192-402d-91ae-c95fc61fad45-tls-certificates\") pod \"prometheus-operator-admission-webhook-57cf98b594-gsv7g\" (UID: \"a23ef821-2192-402d-91ae-c95fc61fad45\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-gsv7g" Apr 22 21:11:35.936120 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.936104 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0f2eedec-34a3-43cc-8586-ec247b1fd8bc-installation-pull-secrets\") pod \"image-registry-7db9c69bcc-wxqq6\" (UID: \"0f2eedec-34a3-43cc-8586-ec247b1fd8bc\") " pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:11:35.936120 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.936110 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0f2eedec-34a3-43cc-8586-ec247b1fd8bc-image-registry-private-configuration\") pod \"image-registry-7db9c69bcc-wxqq6\" (UID: \"0f2eedec-34a3-43cc-8586-ec247b1fd8bc\") " pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:11:35.941461 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.941438 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f2eedec-34a3-43cc-8586-ec247b1fd8bc-bound-sa-token\") pod \"image-registry-7db9c69bcc-wxqq6\" (UID: \"0f2eedec-34a3-43cc-8586-ec247b1fd8bc\") " pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:11:35.941833 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:35.941801 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbjmg\" (UniqueName: \"kubernetes.io/projected/0f2eedec-34a3-43cc-8586-ec247b1fd8bc-kube-api-access-jbjmg\") pod \"image-registry-7db9c69bcc-wxqq6\" (UID: \"0f2eedec-34a3-43cc-8586-ec247b1fd8bc\") " pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:11:36.047914 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:36.047871 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-gsv7g" Apr 22 21:11:36.054627 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:36.054597 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:11:36.193263 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:36.193196 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-gsv7g"] Apr 22 21:11:36.196870 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:11:36.196831 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda23ef821_2192_402d_91ae_c95fc61fad45.slice/crio-6dac0495932b4a255bd44fbf1edb8097fa012a72d23959b2ea0900575fe328bc WatchSource:0}: Error finding container 6dac0495932b4a255bd44fbf1edb8097fa012a72d23959b2ea0900575fe328bc: Status 404 returned error can't find the container with id 6dac0495932b4a255bd44fbf1edb8097fa012a72d23959b2ea0900575fe328bc Apr 22 21:11:36.213988 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:36.213965 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7db9c69bcc-wxqq6"] Apr 22 21:11:36.216496 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:11:36.216470 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f2eedec_34a3_43cc_8586_ec247b1fd8bc.slice/crio-fda933daf7ebb02169f511e3cfbce38049193b05de6d7fa79965413d1853af3c WatchSource:0}: Error finding container fda933daf7ebb02169f511e3cfbce38049193b05de6d7fa79965413d1853af3c: Status 404 returned error can't find the container with id fda933daf7ebb02169f511e3cfbce38049193b05de6d7fa79965413d1853af3c Apr 22 21:11:36.318059 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:36.318022 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" event={"ID":"0f2eedec-34a3-43cc-8586-ec247b1fd8bc","Type":"ContainerStarted","Data":"b4605d33d5fbd415cecabe9ad6165d81ae61c27fb4045b4ce18eae4e93ff3c55"} Apr 22 21:11:36.318059 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:36.318061 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" event={"ID":"0f2eedec-34a3-43cc-8586-ec247b1fd8bc","Type":"ContainerStarted","Data":"fda933daf7ebb02169f511e3cfbce38049193b05de6d7fa79965413d1853af3c"} Apr 22 21:11:36.318291 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:36.318151 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:11:36.319209 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:36.319188 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-gsv7g" event={"ID":"a23ef821-2192-402d-91ae-c95fc61fad45","Type":"ContainerStarted","Data":"6dac0495932b4a255bd44fbf1edb8097fa012a72d23959b2ea0900575fe328bc"} Apr 22 21:11:36.337100 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:36.337059 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" podStartSLOduration=1.337047531 podStartE2EDuration="1.337047531s" podCreationTimestamp="2026-04-22 21:11:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-22 21:11:36.335647977 +0000 UTC m=+137.061544567" watchObservedRunningTime="2026-04-22 21:11:36.337047531 +0000 UTC m=+137.062944133" Apr 22 21:11:37.323464 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:37.323430 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-gsv7g" event={"ID":"a23ef821-2192-402d-91ae-c95fc61fad45","Type":"ContainerStarted","Data":"0e87e83825dc1fe1d1864ae79227d5b5dd1a6d16d75c0864eaf57e91b24c341f"} Apr 22 21:11:37.338423 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:37.338380 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-gsv7g" podStartSLOduration=1.379137109 podStartE2EDuration="2.338366578s" podCreationTimestamp="2026-04-22 21:11:35 +0000 UTC" firstStartedPulling="2026-04-22 21:11:36.198719602 +0000 UTC m=+136.924616182" lastFinishedPulling="2026-04-22 21:11:37.157949068 +0000 UTC m=+137.883845651" observedRunningTime="2026-04-22 21:11:37.337281756 +0000 UTC m=+138.063178360" watchObservedRunningTime="2026-04-22 21:11:37.338366578 +0000 UTC m=+138.064263178" Apr 22 21:11:38.326115 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:38.326077 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-gsv7g" Apr 22 21:11:38.330586 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:38.330564 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-gsv7g" Apr 22 21:11:42.977998 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:42.977963 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rcwjz"] Apr 22 21:11:42.980923 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:42.980900 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:42.983572 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:42.983551 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Apr 22 21:11:42.984094 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:42.984074 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Apr 22 21:11:42.984206 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:42.984089 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qctg5\"" Apr 22 21:11:42.984206 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:42.984141 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Apr 22 21:11:42.985322 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:42.985225 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Apr 22 21:11:42.985322 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:42.985257 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Apr 22 21:11:42.985322 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:42.985246 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Apr 22 21:11:43.087061 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.087024 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-sys\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.087229 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.087069 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-node-exporter-accelerators-collector-config\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.087229 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.087097 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-node-exporter-wtmp\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.087229 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.087194 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-node-exporter-tls\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.087407 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.087230 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-root\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.087407 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.087249 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-node-exporter-textfile\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.087407 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.087270 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7w97\" (UniqueName: \"kubernetes.io/projected/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-kube-api-access-m7w97\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.087407 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.087302 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.087407 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.087319 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-metrics-client-ca\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.188001 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.187964 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-node-exporter-wtmp\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.188187 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.188028 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-node-exporter-tls\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.188187 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.188064 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-root\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.188187 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.188087 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-node-exporter-textfile\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.188187 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.188114 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m7w97\" (UniqueName: \"kubernetes.io/projected/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-kube-api-access-m7w97\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.188187 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.188146 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-node-exporter-wtmp\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.188187 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.188159 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-root\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.188476 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.188183 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.188476 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:43.188194 2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Apr 22 21:11:43.188476 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.188222 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-metrics-client-ca\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.188476 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:43.188258 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-node-exporter-tls podName:bcd6d2ac-f291-46f0-b1d1-f15f26c41c79 nodeName:}" failed. No retries permitted until 2026-04-22 21:11:43.688236908 +0000 UTC m=+144.414133485 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-node-exporter-tls") pod "node-exporter-rcwjz" (UID: "bcd6d2ac-f291-46f0-b1d1-f15f26c41c79") : secret "node-exporter-tls" not found Apr 22 21:11:43.188476 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.188319 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-sys\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.188476 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.188366 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-node-exporter-accelerators-collector-config\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.188476 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.188409 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-sys\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.188476 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.188445 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-node-exporter-textfile\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.188884 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.188813 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-metrics-client-ca\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.188939 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.188901 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-node-exporter-accelerators-collector-config\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.190656 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.190627 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.197334 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.197300 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7w97\" (UniqueName: \"kubernetes.io/projected/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-kube-api-access-m7w97\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.691602 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.691563 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-node-exporter-tls\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.694099 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.694080 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bcd6d2ac-f291-46f0-b1d1-f15f26c41c79-node-exporter-tls\") pod \"node-exporter-rcwjz\" (UID: \"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79\") " pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.890808 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:43.890777 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rcwjz" Apr 22 21:11:43.900312 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:11:43.900274 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcd6d2ac_f291_46f0_b1d1_f15f26c41c79.slice/crio-c1571736a6362733876f7246af83fa3b657825d31383e47c1dbdd22a79d5dc1d WatchSource:0}: Error finding container c1571736a6362733876f7246af83fa3b657825d31383e47c1dbdd22a79d5dc1d: Status 404 returned error can't find the container with id c1571736a6362733876f7246af83fa3b657825d31383e47c1dbdd22a79d5dc1d Apr 22 21:11:44.341620 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:44.341576 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rcwjz" event={"ID":"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79","Type":"ContainerStarted","Data":"c1571736a6362733876f7246af83fa3b657825d31383e47c1dbdd22a79d5dc1d"} Apr 22 21:11:45.345462 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:45.345428 2580 generic.go:358] "Generic (PLEG): container finished" podID="bcd6d2ac-f291-46f0-b1d1-f15f26c41c79" containerID="581a102d7975eec1f2e26499e543830fb5fff588787252d32b9f2f321cce6622" exitCode=0 Apr 22 21:11:45.345797 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:45.345479 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rcwjz" event={"ID":"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79","Type":"ContainerDied","Data":"581a102d7975eec1f2e26499e543830fb5fff588787252d32b9f2f321cce6622"} Apr 22 21:11:46.349952 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:46.349913 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rcwjz" event={"ID":"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79","Type":"ContainerStarted","Data":"66ff0825e4b6cbbd157af2bb3ace7a9ebae786f53a8b3ea965b2e777d436d0f2"} Apr 22 21:11:46.349952 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:46.349951 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rcwjz" event={"ID":"bcd6d2ac-f291-46f0-b1d1-f15f26c41c79","Type":"ContainerStarted","Data":"101f49da04abe0e064fe37ac4955a1f8b180712feafc779127a2847a1146c8fb"} Apr 22 21:11:46.374300 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:46.374249 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rcwjz" podStartSLOduration=3.268855529 podStartE2EDuration="4.374232517s" podCreationTimestamp="2026-04-22 21:11:42 +0000 UTC" firstStartedPulling="2026-04-22 21:11:43.902143889 +0000 UTC m=+144.628040467" lastFinishedPulling="2026-04-22 21:11:45.007520876 +0000 UTC m=+145.733417455" observedRunningTime="2026-04-22 21:11:46.372706988 +0000 UTC m=+147.098603600" watchObservedRunningTime="2026-04-22 21:11:46.374232517 +0000 UTC m=+147.100129118" Apr 22 21:11:47.734262 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:47.734226 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7dccd58f55-4p2k6"] Apr 22 21:11:47.736601 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:47.736582 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-4p2k6" Apr 22 21:11:47.739318 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:47.739283 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-tqpfc\"" Apr 22 21:11:47.739318 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:47.739303 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Apr 22 21:11:47.743367 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:47.743345 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7dccd58f55-4p2k6"] Apr 22 21:11:47.825259 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:47.825225 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/10bb0162-71a7-451d-bc0c-578af94b5b8a-monitoring-plugin-cert\") pod \"monitoring-plugin-7dccd58f55-4p2k6\" (UID: \"10bb0162-71a7-451d-bc0c-578af94b5b8a\") " pod="openshift-monitoring/monitoring-plugin-7dccd58f55-4p2k6" Apr 22 21:11:47.926211 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:47.926174 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/10bb0162-71a7-451d-bc0c-578af94b5b8a-monitoring-plugin-cert\") pod \"monitoring-plugin-7dccd58f55-4p2k6\" (UID: \"10bb0162-71a7-451d-bc0c-578af94b5b8a\") " pod="openshift-monitoring/monitoring-plugin-7dccd58f55-4p2k6" Apr 22 21:11:47.926374 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:47.926353 2580 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Apr 22 21:11:47.926448 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:47.926437 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10bb0162-71a7-451d-bc0c-578af94b5b8a-monitoring-plugin-cert podName:10bb0162-71a7-451d-bc0c-578af94b5b8a nodeName:}" failed. No retries permitted until 2026-04-22 21:11:48.426414545 +0000 UTC m=+149.152311131 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/10bb0162-71a7-451d-bc0c-578af94b5b8a-monitoring-plugin-cert") pod "monitoring-plugin-7dccd58f55-4p2k6" (UID: "10bb0162-71a7-451d-bc0c-578af94b5b8a") : secret "monitoring-plugin-cert" not found Apr 22 21:11:48.430080 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:48.430044 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/10bb0162-71a7-451d-bc0c-578af94b5b8a-monitoring-plugin-cert\") pod \"monitoring-plugin-7dccd58f55-4p2k6\" (UID: \"10bb0162-71a7-451d-bc0c-578af94b5b8a\") " pod="openshift-monitoring/monitoring-plugin-7dccd58f55-4p2k6" Apr 22 21:11:48.432578 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:48.432554 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/10bb0162-71a7-451d-bc0c-578af94b5b8a-monitoring-plugin-cert\") pod \"monitoring-plugin-7dccd58f55-4p2k6\" (UID: \"10bb0162-71a7-451d-bc0c-578af94b5b8a\") " pod="openshift-monitoring/monitoring-plugin-7dccd58f55-4p2k6" Apr 22 21:11:48.646905 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:48.646866 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-4p2k6" Apr 22 21:11:48.762469 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:48.762374 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7dccd58f55-4p2k6"] Apr 22 21:11:48.764871 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:11:48.764827 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10bb0162_71a7_451d_bc0c_578af94b5b8a.slice/crio-dfdaf192ca3971d8c27a886a0cdd4696ec7334ccfec1d7848812c59a476f55de WatchSource:0}: Error finding container dfdaf192ca3971d8c27a886a0cdd4696ec7334ccfec1d7848812c59a476f55de: Status 404 returned error can't find the container with id dfdaf192ca3971d8c27a886a0cdd4696ec7334ccfec1d7848812c59a476f55de Apr 22 21:11:49.191058 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.191026 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 22 21:11:49.194139 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.194123 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.197000 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.196972 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Apr 22 21:11:49.197570 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.197545 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Apr 22 21:11:49.197666 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.197574 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Apr 22 21:11:49.197666 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.197636 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Apr 22 21:11:49.197910 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.197886 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Apr 22 21:11:49.201771 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.199937 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4s76p81ptr7as\"" Apr 22 21:11:49.201771 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.200160 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Apr 22 21:11:49.201771 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.200330 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Apr 22 21:11:49.201771 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.200549 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Apr 22 21:11:49.201771 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.201685 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-b6b78\"" Apr 22 21:11:49.202288 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.202146 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Apr 22 21:11:49.202288 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.202182 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Apr 22 21:11:49.204363 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.204334 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Apr 22 21:11:49.206393 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.206374 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Apr 22 21:11:49.206976 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.206956 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 22 21:11:49.207193 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.207177 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Apr 22 21:11:49.237460 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.237431 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.237578 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.237472 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.237578 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.237499 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.237578 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.237517 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.237578 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.237548 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.237578 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.237571 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-config-out\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.237840 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.237632 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.237840 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.237714 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.237840 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.237771 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-config\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.237840 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.237797 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.237840 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.237833 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.238083 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.237917 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.238083 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.237948 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.238083 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.237996 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r25ln\" (UniqueName: \"kubernetes.io/projected/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-kube-api-access-r25ln\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.238083 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.238046 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.238083 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.238073 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.238264 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.238101 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.238264 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.238123 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-web-config\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.339359 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.339317 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.339529 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.339374 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.339529 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.339403 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.339529 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.339457 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.339529 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.339482 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-config-out\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.339529 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.339507 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.339784 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.339534 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.339784 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.339576 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-config\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.339784 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.339598 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.339784 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.339622 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.339784 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.339667 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.339784 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.339696 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.339784 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.339731 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r25ln\" (UniqueName: \"kubernetes.io/projected/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-kube-api-access-r25ln\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.340643 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.340611 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.343370 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.343338 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.343476 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.343434 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.343543 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.343488 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.343543 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.343528 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.343648 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.343553 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-web-config\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.343648 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.343583 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.343648 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.343591 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.343802 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.343785 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.343970 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.343947 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.344146 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.344107 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.344979 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.344531 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.344979 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.344780 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-config-out\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.345163 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.345023 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.345386 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.345341 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-config\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.346388 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.346358 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.346550 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.346505 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.346638 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.346549 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.346828 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.346810 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.347248 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.347204 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-web-config\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.348313 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.348294 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.348402 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.348376 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.348563 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.348539 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r25ln\" (UniqueName: \"kubernetes.io/projected/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-kube-api-access-r25ln\") pod \"prometheus-k8s-0\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.359156 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.359129 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-4p2k6" event={"ID":"10bb0162-71a7-451d-bc0c-578af94b5b8a","Type":"ContainerStarted","Data":"dfdaf192ca3971d8c27a886a0cdd4696ec7334ccfec1d7848812c59a476f55de"} Apr 22 21:11:49.508753 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.508672 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:11:49.651119 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:49.651082 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 22 21:11:49.931891 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:11:49.931839 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod503b81b1_1119_4a8d_b1bd_fa6cf6b9ecfc.slice/crio-b9f2f94029f90547afc1ffa1c4808b05f3eb87f78f446afac39cb88c0b8d73da WatchSource:0}: Error finding container b9f2f94029f90547afc1ffa1c4808b05f3eb87f78f446afac39cb88c0b8d73da: Status 404 returned error can't find the container with id b9f2f94029f90547afc1ffa1c4808b05f3eb87f78f446afac39cb88c0b8d73da Apr 22 21:11:50.364420 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:50.364324 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-4p2k6" event={"ID":"10bb0162-71a7-451d-bc0c-578af94b5b8a","Type":"ContainerStarted","Data":"f6bd0ad764aa2c45135cc81636ca030ceaab823e4d00f722a3e46286bf5285fd"} Apr 22 21:11:50.364676 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:50.364579 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-4p2k6" Apr 22 21:11:50.365587 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:50.365565 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc","Type":"ContainerStarted","Data":"b9f2f94029f90547afc1ffa1c4808b05f3eb87f78f446afac39cb88c0b8d73da"} Apr 22 21:11:50.370520 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:50.370466 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-4p2k6" Apr 22 21:11:50.379237 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:50.379168 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-4p2k6" podStartSLOduration=2.189748127 podStartE2EDuration="3.379156829s" podCreationTimestamp="2026-04-22 21:11:47 +0000 UTC" firstStartedPulling="2026-04-22 21:11:48.766668228 +0000 UTC m=+149.492564805" lastFinishedPulling="2026-04-22 21:11:49.956076929 +0000 UTC m=+150.681973507" observedRunningTime="2026-04-22 21:11:50.377909136 +0000 UTC m=+151.103805738" watchObservedRunningTime="2026-04-22 21:11:50.379156829 +0000 UTC m=+151.105053457" Apr 22 21:11:51.369217 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:51.369184 2580 generic.go:358] "Generic (PLEG): container finished" podID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerID="0758042218fd8496ab7a7715f31a4dfe9b7b7d47fb65820ca5b293b9da3c394f" exitCode=0 Apr 22 21:11:51.369635 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:51.369271 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc","Type":"ContainerDied","Data":"0758042218fd8496ab7a7715f31a4dfe9b7b7d47fb65820ca5b293b9da3c394f"} Apr 22 21:11:54.379379 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:54.379288 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc","Type":"ContainerStarted","Data":"129ec3dca79759e71f616e16ca7a3bb4d7aed852e286a93bef699183619ce294"} Apr 22 21:11:54.379379 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:54.379329 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc","Type":"ContainerStarted","Data":"437903f98557692143da4e30c0d6e538c5015694bf8b8d65ac321b4c57ed3201"} Apr 22 21:11:56.059498 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:56.059369 2580 patch_prober.go:28] interesting pod/image-registry-7db9c69bcc-wxqq6 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Apr 22 21:11:56.059498 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:56.059452 2580 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" podUID="0f2eedec-34a3-43cc-8586-ec247b1fd8bc" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Apr 22 21:11:56.202537 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:56.202495 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-9klhr" podUID="e92ccf2d-8b03-40ec-81d7-e0752b8f6c78" Apr 22 21:11:56.230564 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:11:56.227268 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-rp9w5" podUID="9ef207b1-3445-413b-8855-1e0c977efcf2" Apr 22 21:11:56.387652 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:56.387609 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc","Type":"ContainerStarted","Data":"c41910aafdbe2ffd00897b22d06bd01c7d0ab29534f9d42c4e28598d0dfa7ba4"} Apr 22 21:11:56.387652 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:56.387653 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9klhr" Apr 22 21:11:56.387844 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:56.387660 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc","Type":"ContainerStarted","Data":"2b9abfd2c0207e480996e62b704393dae3a5dcd273ad14cf1f093abb36451747"} Apr 22 21:11:56.387844 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:56.387675 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc","Type":"ContainerStarted","Data":"6566a988615a9096fbb04663e61cdc64bcfc829859946fa8845baa4bde61c243"} Apr 22 21:11:56.387844 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:56.387689 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc","Type":"ContainerStarted","Data":"786758fdf24d3398ecb70b961c531b435d50172dce4d85179f4b477ee99b6f2b"} Apr 22 21:11:56.417438 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:56.417389 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.609841108 podStartE2EDuration="7.417377988s" podCreationTimestamp="2026-04-22 21:11:49 +0000 UTC" firstStartedPulling="2026-04-22 21:11:49.934262194 +0000 UTC m=+150.660158772" lastFinishedPulling="2026-04-22 21:11:55.741799074 +0000 UTC m=+156.467695652" observedRunningTime="2026-04-22 21:11:56.417272115 +0000 UTC m=+157.143168741" watchObservedRunningTime="2026-04-22 21:11:56.417377988 +0000 UTC m=+157.143274588" Apr 22 21:11:57.327375 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:57.327342 2580 patch_prober.go:28] interesting pod/image-registry-7db9c69bcc-wxqq6 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Apr 22 21:11:57.327750 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:57.327396 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" podUID="0f2eedec-34a3-43cc-8586-ec247b1fd8bc" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Apr 22 21:11:59.509642 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:11:59.509596 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:12:01.047645 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:01.047607 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-metrics-tls\") pod \"dns-default-9klhr\" (UID: \"e92ccf2d-8b03-40ec-81d7-e0752b8f6c78\") " pod="openshift-dns/dns-default-9klhr" Apr 22 21:12:01.050158 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:01.050130 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e92ccf2d-8b03-40ec-81d7-e0752b8f6c78-metrics-tls\") pod \"dns-default-9klhr\" (UID: \"e92ccf2d-8b03-40ec-81d7-e0752b8f6c78\") " pod="openshift-dns/dns-default-9klhr" Apr 22 21:12:01.148593 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:01.148560 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ef207b1-3445-413b-8855-1e0c977efcf2-cert\") pod \"ingress-canary-rp9w5\" (UID: \"9ef207b1-3445-413b-8855-1e0c977efcf2\") " pod="openshift-ingress-canary/ingress-canary-rp9w5" Apr 22 21:12:01.151058 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:01.151036 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ef207b1-3445-413b-8855-1e0c977efcf2-cert\") pod \"ingress-canary-rp9w5\" (UID: \"9ef207b1-3445-413b-8855-1e0c977efcf2\") " pod="openshift-ingress-canary/ingress-canary-rp9w5" Apr 22 21:12:01.190719 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:01.190697 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8d48v\"" Apr 22 21:12:01.198992 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:01.198975 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9klhr" Apr 22 21:12:01.315099 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:01.315033 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9klhr"] Apr 22 21:12:01.318555 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:12:01.318529 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode92ccf2d_8b03_40ec_81d7_e0752b8f6c78.slice/crio-5b322581a3dfdebc1ba1593419aaf2618949c7fd73e7c070dd357a4724f5c7bc WatchSource:0}: Error finding container 5b322581a3dfdebc1ba1593419aaf2618949c7fd73e7c070dd357a4724f5c7bc: Status 404 returned error can't find the container with id 5b322581a3dfdebc1ba1593419aaf2618949c7fd73e7c070dd357a4724f5c7bc Apr 22 21:12:01.400965 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:01.400934 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9klhr" event={"ID":"e92ccf2d-8b03-40ec-81d7-e0752b8f6c78","Type":"ContainerStarted","Data":"5b322581a3dfdebc1ba1593419aaf2618949c7fd73e7c070dd357a4724f5c7bc"} Apr 22 21:12:03.407664 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:03.407628 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9klhr" event={"ID":"e92ccf2d-8b03-40ec-81d7-e0752b8f6c78","Type":"ContainerStarted","Data":"0d661e45e047f79e211d39aa278661bff759cd6f7692fe6956257251cebd17c3"} Apr 22 21:12:03.407664 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:03.407668 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9klhr" event={"ID":"e92ccf2d-8b03-40ec-81d7-e0752b8f6c78","Type":"ContainerStarted","Data":"14bddc94989d1f2f110f46634098f78d5719842a7e99892d30d342672377db83"} Apr 22 21:12:03.408163 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:03.407808 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9klhr" Apr 22 21:12:03.423140 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:03.423094 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-9klhr" podStartSLOduration=129.22286328 podStartE2EDuration="2m10.423082126s" podCreationTimestamp="2026-04-22 21:09:53 +0000 UTC" firstStartedPulling="2026-04-22 21:12:01.320680363 +0000 UTC m=+162.046576941" lastFinishedPulling="2026-04-22 21:12:02.520899208 +0000 UTC m=+163.246795787" observedRunningTime="2026-04-22 21:12:03.422297547 +0000 UTC m=+164.148194160" watchObservedRunningTime="2026-04-22 21:12:03.423082126 +0000 UTC m=+164.148978726" Apr 22 21:12:06.058612 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:06.058582 2580 patch_prober.go:28] interesting pod/image-registry-7db9c69bcc-wxqq6 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Apr 22 21:12:06.059088 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:06.058634 2580 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" podUID="0f2eedec-34a3-43cc-8586-ec247b1fd8bc" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Apr 22 21:12:07.327593 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:07.327566 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7db9c69bcc-wxqq6" Apr 22 21:12:10.858739 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:10.858689 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rp9w5" Apr 22 21:12:10.861965 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:10.861947 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tq52g\"" Apr 22 21:12:10.870113 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:10.870095 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rp9w5" Apr 22 21:12:10.987171 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:10.987105 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rp9w5"] Apr 22 21:12:10.989433 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:12:10.989400 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ef207b1_3445_413b_8855_1e0c977efcf2.slice/crio-e575849d2ac852ae12ca11d6c9da4caece82ba6b5dc0a377b82b744207012a1b WatchSource:0}: Error finding container e575849d2ac852ae12ca11d6c9da4caece82ba6b5dc0a377b82b744207012a1b: Status 404 returned error can't find the container with id e575849d2ac852ae12ca11d6c9da4caece82ba6b5dc0a377b82b744207012a1b Apr 22 21:12:11.431177 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:11.431138 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rp9w5" event={"ID":"9ef207b1-3445-413b-8855-1e0c977efcf2","Type":"ContainerStarted","Data":"e575849d2ac852ae12ca11d6c9da4caece82ba6b5dc0a377b82b744207012a1b"} Apr 22 21:12:13.413487 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:13.413457 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9klhr" Apr 22 21:12:13.440344 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:13.440313 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rp9w5" event={"ID":"9ef207b1-3445-413b-8855-1e0c977efcf2","Type":"ContainerStarted","Data":"bc5b5efb42fe353ba7c0836f001230bc7168dd3026c3acdd4f4f26955237f4cb"} Apr 22 21:12:13.454975 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:13.454920 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rp9w5" podStartSLOduration=138.77552733 podStartE2EDuration="2m20.454902047s" podCreationTimestamp="2026-04-22 21:09:53 +0000 UTC" firstStartedPulling="2026-04-22 21:12:10.991687467 +0000 UTC m=+171.717584046" lastFinishedPulling="2026-04-22 21:12:12.671062182 +0000 UTC m=+173.396958763" observedRunningTime="2026-04-22 21:12:13.454160146 +0000 UTC m=+174.180056748" watchObservedRunningTime="2026-04-22 21:12:13.454902047 +0000 UTC m=+174.180798649" Apr 22 21:12:23.469507 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:23.469474 2580 generic.go:358] "Generic (PLEG): container finished" podID="a292639d-6620-4540-8ee0-322ee00ff2f7" containerID="f71e721c0db825e39e456b87ce58c70d790362802bc2977b7ae33011eab2a0b4" exitCode=0 Apr 22 21:12:23.470052 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:23.469550 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-xf75v" event={"ID":"a292639d-6620-4540-8ee0-322ee00ff2f7","Type":"ContainerDied","Data":"f71e721c0db825e39e456b87ce58c70d790362802bc2977b7ae33011eab2a0b4"} Apr 22 21:12:23.470052 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:23.469994 2580 scope.go:117] "RemoveContainer" containerID="f71e721c0db825e39e456b87ce58c70d790362802bc2977b7ae33011eab2a0b4" Apr 22 21:12:24.474036 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:24.474003 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-xf75v" event={"ID":"a292639d-6620-4540-8ee0-322ee00ff2f7","Type":"ContainerStarted","Data":"f36e072ec79aa159e1f545844745492061f14911a6da233658a1996589d4b375"} Apr 22 21:12:33.498802 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:33.498759 2580 generic.go:358] "Generic (PLEG): container finished" podID="7d912a63-fac1-4f77-b588-e42de7a51799" containerID="2eb6d14b0e8ea85e575a7e4e5f66946a060e010ed445f49c4067e80b83ae5dba" exitCode=0 Apr 22 21:12:33.499288 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:33.498829 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-585dfdc468-gwzzw" event={"ID":"7d912a63-fac1-4f77-b588-e42de7a51799","Type":"ContainerDied","Data":"2eb6d14b0e8ea85e575a7e4e5f66946a060e010ed445f49c4067e80b83ae5dba"} Apr 22 21:12:33.499288 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:33.499176 2580 scope.go:117] "RemoveContainer" containerID="2eb6d14b0e8ea85e575a7e4e5f66946a060e010ed445f49c4067e80b83ae5dba" Apr 22 21:12:34.504232 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:34.504190 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-585dfdc468-gwzzw" event={"ID":"7d912a63-fac1-4f77-b588-e42de7a51799","Type":"ContainerStarted","Data":"a9417b55acce3ef5192da17e1ca827bc13c3573a3750f54995a89fb2b5db793f"} Apr 22 21:12:38.518119 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:38.518084 2580 generic.go:358] "Generic (PLEG): container finished" podID="dcdefc0e-dace-496e-bccf-de663633bec9" containerID="6f5e2bdd58762749e3e7025c40cd5cde976256abcf4d9039bd1e489378bc286f" exitCode=0 Apr 22 21:12:38.518523 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:38.518138 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-j45pb" event={"ID":"dcdefc0e-dace-496e-bccf-de663633bec9","Type":"ContainerDied","Data":"6f5e2bdd58762749e3e7025c40cd5cde976256abcf4d9039bd1e489378bc286f"} Apr 22 21:12:38.518523 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:38.518437 2580 scope.go:117] "RemoveContainer" containerID="6f5e2bdd58762749e3e7025c40cd5cde976256abcf4d9039bd1e489378bc286f" Apr 22 21:12:39.523177 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:39.523139 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-j45pb" event={"ID":"dcdefc0e-dace-496e-bccf-de663633bec9","Type":"ContainerStarted","Data":"4bcee49656cee7d3cf074f047124ecd30decd230a00a36f4e853d4d9f62892be"} Apr 22 21:12:49.509625 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:49.509577 2580 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:12:49.525330 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:49.525299 2580 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:12:49.570563 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:12:49.570542 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:07.476215 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:07.476182 2580 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 22 21:13:07.476791 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:07.476614 2580 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerName="prometheus" containerID="cri-o://437903f98557692143da4e30c0d6e538c5015694bf8b8d65ac321b4c57ed3201" gracePeriod=600 Apr 22 21:13:07.476791 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:07.476663 2580 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerName="kube-rbac-proxy-thanos" containerID="cri-o://c41910aafdbe2ffd00897b22d06bd01c7d0ab29534f9d42c4e28598d0dfa7ba4" gracePeriod=600 Apr 22 21:13:07.476791 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:07.476711 2580 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerName="config-reloader" containerID="cri-o://129ec3dca79759e71f616e16ca7a3bb4d7aed852e286a93bef699183619ce294" gracePeriod=600 Apr 22 21:13:07.476791 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:07.476663 2580 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerName="kube-rbac-proxy" containerID="cri-o://2b9abfd2c0207e480996e62b704393dae3a5dcd273ad14cf1f093abb36451747" gracePeriod=600 Apr 22 21:13:07.477057 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:07.476676 2580 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerName="thanos-sidecar" containerID="cri-o://786758fdf24d3398ecb70b961c531b435d50172dce4d85179f4b477ee99b6f2b" gracePeriod=600 Apr 22 21:13:07.477057 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:07.476716 2580 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerName="kube-rbac-proxy-web" containerID="cri-o://6566a988615a9096fbb04663e61cdc64bcfc829859946fa8845baa4bde61c243" gracePeriod=600 Apr 22 21:13:07.608224 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:07.608195 2580 generic.go:358] "Generic (PLEG): container finished" podID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerID="c41910aafdbe2ffd00897b22d06bd01c7d0ab29534f9d42c4e28598d0dfa7ba4" exitCode=0 Apr 22 21:13:07.608224 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:07.608222 2580 generic.go:358] "Generic (PLEG): container finished" podID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerID="2b9abfd2c0207e480996e62b704393dae3a5dcd273ad14cf1f093abb36451747" exitCode=0 Apr 22 21:13:07.608380 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:07.608230 2580 generic.go:358] "Generic (PLEG): container finished" podID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerID="786758fdf24d3398ecb70b961c531b435d50172dce4d85179f4b477ee99b6f2b" exitCode=0 Apr 22 21:13:07.608380 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:07.608239 2580 generic.go:358] "Generic (PLEG): container finished" podID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerID="129ec3dca79759e71f616e16ca7a3bb4d7aed852e286a93bef699183619ce294" exitCode=0 Apr 22 21:13:07.608380 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:07.608246 2580 generic.go:358] "Generic (PLEG): container finished" podID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerID="437903f98557692143da4e30c0d6e538c5015694bf8b8d65ac321b4c57ed3201" exitCode=0 Apr 22 21:13:07.608380 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:07.608277 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc","Type":"ContainerDied","Data":"c41910aafdbe2ffd00897b22d06bd01c7d0ab29534f9d42c4e28598d0dfa7ba4"} Apr 22 21:13:07.608380 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:07.608313 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc","Type":"ContainerDied","Data":"2b9abfd2c0207e480996e62b704393dae3a5dcd273ad14cf1f093abb36451747"} Apr 22 21:13:07.608380 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:07.608326 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc","Type":"ContainerDied","Data":"786758fdf24d3398ecb70b961c531b435d50172dce4d85179f4b477ee99b6f2b"} Apr 22 21:13:07.608380 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:07.608335 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc","Type":"ContainerDied","Data":"129ec3dca79759e71f616e16ca7a3bb4d7aed852e286a93bef699183619ce294"} Apr 22 21:13:07.608380 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:07.608345 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc","Type":"ContainerDied","Data":"437903f98557692143da4e30c0d6e538c5015694bf8b8d65ac321b4c57ed3201"} Apr 22 21:13:08.809173 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.809150 2580 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:08.875533 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.875462 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-prometheus-k8s-rulefiles-0\") pod \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " Apr 22 21:13:08.875533 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.875497 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r25ln\" (UniqueName: \"kubernetes.io/projected/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-kube-api-access-r25ln\") pod \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " Apr 22 21:13:08.875751 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.875534 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-metrics-client-certs\") pod \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " Apr 22 21:13:08.875751 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.875717 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " Apr 22 21:13:08.875878 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.875758 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-web-config\") pod \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " Apr 22 21:13:08.875878 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.875793 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-config-out\") pod \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " Apr 22 21:13:08.875878 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.875822 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-config\") pod \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " Apr 22 21:13:08.875878 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.875873 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-prometheus-k8s-tls\") pod \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " Apr 22 21:13:08.876137 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.875907 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-configmap-serving-certs-ca-bundle\") pod \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " Apr 22 21:13:08.876137 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.875945 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " Apr 22 21:13:08.876137 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.875973 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-configmap-metrics-client-ca\") pod \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " Apr 22 21:13:08.876137 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.875998 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-thanos-prometheus-http-client-file\") pod \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " Apr 22 21:13:08.876137 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.876030 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-prometheus-k8s-db\") pod \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " Apr 22 21:13:08.876137 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.876080 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-configmap-kubelet-serving-ca-bundle\") pod \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " Apr 22 21:13:08.876137 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.876125 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-prometheus-trusted-ca-bundle\") pod \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " Apr 22 21:13:08.876644 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.876152 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-grpc-tls\") pod \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " Apr 22 21:13:08.876644 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.876190 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-kube-rbac-proxy\") pod \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " Apr 22 21:13:08.876644 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.876215 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-tls-assets\") pod \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\" (UID: \"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc\") " Apr 22 21:13:08.876644 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.876404 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" (UID: "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 22 21:13:08.877652 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.877082 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" (UID: "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 22 21:13:08.878615 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.878448 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" (UID: "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 22 21:13:08.878837 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.878809 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" (UID: "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 22 21:13:08.879029 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.878896 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" (UID: "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 22 21:13:08.879029 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.878893 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" (UID: "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 22 21:13:08.879029 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.878966 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-config" (OuterVolumeSpecName: "config") pod "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" (UID: "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 22 21:13:08.879504 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.879216 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" (UID: "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 22 21:13:08.880268 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.880102 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" (UID: "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 22 21:13:08.880268 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.880175 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-kube-api-access-r25ln" (OuterVolumeSpecName: "kube-api-access-r25ln") pod "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" (UID: "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc"). InnerVolumeSpecName "kube-api-access-r25ln". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 22 21:13:08.880268 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.880221 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-config-out" (OuterVolumeSpecName: "config-out") pod "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" (UID: "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 22 21:13:08.881314 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.881286 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" (UID: "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 22 21:13:08.881395 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.881323 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" (UID: "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 22 21:13:08.881781 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.881714 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" (UID: "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 22 21:13:08.881899 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.881797 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" (UID: "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 22 21:13:08.881899 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.881887 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" (UID: "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 22 21:13:08.882713 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.882695 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" (UID: "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 22 21:13:08.892592 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.892568 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-web-config" (OuterVolumeSpecName: "web-config") pod "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" (UID: "503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 22 21:13:08.977673 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.977638 2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-web-config\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:13:08.977673 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.977670 2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-config-out\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:13:08.977673 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.977679 2580 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-config\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:13:08.977961 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.977688 2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-prometheus-k8s-tls\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:13:08.977961 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.977697 2580 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:13:08.977961 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.977707 2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:13:08.977961 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.977716 2580 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-configmap-metrics-client-ca\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:13:08.977961 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.977724 2580 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-thanos-prometheus-http-client-file\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:13:08.977961 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.977733 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-prometheus-k8s-db\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:13:08.977961 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.977743 2580 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:13:08.977961 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.977753 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-prometheus-trusted-ca-bundle\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:13:08.977961 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.977761 2580 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-grpc-tls\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:13:08.977961 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.977769 2580 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-kube-rbac-proxy\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:13:08.977961 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.977777 2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-tls-assets\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:13:08.977961 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.977786 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:13:08.977961 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.977794 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r25ln\" (UniqueName: \"kubernetes.io/projected/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-kube-api-access-r25ln\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:13:08.977961 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.977804 2580 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-metrics-client-certs\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:13:08.977961 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:08.977813 2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:13:09.615813 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.615781 2580 generic.go:358] "Generic (PLEG): container finished" podID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerID="6566a988615a9096fbb04663e61cdc64bcfc829859946fa8845baa4bde61c243" exitCode=0 Apr 22 21:13:09.615958 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.615869 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc","Type":"ContainerDied","Data":"6566a988615a9096fbb04663e61cdc64bcfc829859946fa8845baa4bde61c243"} Apr 22 21:13:09.615958 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.615903 2580 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.615958 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.615919 2580 scope.go:117] "RemoveContainer" containerID="c41910aafdbe2ffd00897b22d06bd01c7d0ab29534f9d42c4e28598d0dfa7ba4" Apr 22 21:13:09.616101 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.615907 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc","Type":"ContainerDied","Data":"b9f2f94029f90547afc1ffa1c4808b05f3eb87f78f446afac39cb88c0b8d73da"} Apr 22 21:13:09.623417 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.623319 2580 scope.go:117] "RemoveContainer" containerID="2b9abfd2c0207e480996e62b704393dae3a5dcd273ad14cf1f093abb36451747" Apr 22 21:13:09.630036 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.629987 2580 scope.go:117] "RemoveContainer" containerID="6566a988615a9096fbb04663e61cdc64bcfc829859946fa8845baa4bde61c243" Apr 22 21:13:09.636768 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.636747 2580 scope.go:117] "RemoveContainer" containerID="786758fdf24d3398ecb70b961c531b435d50172dce4d85179f4b477ee99b6f2b" Apr 22 21:13:09.637587 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.637565 2580 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 22 21:13:09.644079 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.644064 2580 scope.go:117] "RemoveContainer" containerID="129ec3dca79759e71f616e16ca7a3bb4d7aed852e286a93bef699183619ce294" Apr 22 21:13:09.644961 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.644944 2580 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 22 21:13:09.650563 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.650545 2580 scope.go:117] "RemoveContainer" containerID="437903f98557692143da4e30c0d6e538c5015694bf8b8d65ac321b4c57ed3201" Apr 22 21:13:09.657262 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.657245 2580 scope.go:117] "RemoveContainer" containerID="0758042218fd8496ab7a7715f31a4dfe9b7b7d47fb65820ca5b293b9da3c394f" Apr 22 21:13:09.663445 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.663427 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 22 21:13:09.663702 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.663686 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerName="config-reloader" Apr 22 21:13:09.663746 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.663705 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerName="config-reloader" Apr 22 21:13:09.663746 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.663714 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerName="kube-rbac-proxy-thanos" Apr 22 21:13:09.663746 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.663720 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerName="kube-rbac-proxy-thanos" Apr 22 21:13:09.663746 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.663729 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerName="thanos-sidecar" Apr 22 21:13:09.663746 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.663736 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerName="thanos-sidecar" Apr 22 21:13:09.663746 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.663745 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerName="prometheus" Apr 22 21:13:09.663944 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.663750 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerName="prometheus" Apr 22 21:13:09.663944 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.663772 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerName="kube-rbac-proxy-web" Apr 22 21:13:09.663944 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.663778 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerName="kube-rbac-proxy-web" Apr 22 21:13:09.663944 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.663784 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerName="kube-rbac-proxy" Apr 22 21:13:09.663944 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.663789 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerName="kube-rbac-proxy" Apr 22 21:13:09.663944 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.663796 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerName="init-config-reloader" Apr 22 21:13:09.663944 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.663801 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerName="init-config-reloader" Apr 22 21:13:09.663944 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.663843 2580 memory_manager.go:356] "RemoveStaleState removing state" podUID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerName="kube-rbac-proxy-thanos" Apr 22 21:13:09.663944 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.663869 2580 memory_manager.go:356] "RemoveStaleState removing state" podUID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerName="config-reloader" Apr 22 21:13:09.663944 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.663876 2580 memory_manager.go:356] "RemoveStaleState removing state" podUID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerName="thanos-sidecar" Apr 22 21:13:09.663944 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.663882 2580 memory_manager.go:356] "RemoveStaleState removing state" podUID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerName="kube-rbac-proxy-web" Apr 22 21:13:09.663944 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.663887 2580 memory_manager.go:356] "RemoveStaleState removing state" podUID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerName="kube-rbac-proxy" Apr 22 21:13:09.663944 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.663894 2580 memory_manager.go:356] "RemoveStaleState removing state" podUID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" containerName="prometheus" Apr 22 21:13:09.664410 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.664196 2580 scope.go:117] "RemoveContainer" containerID="c41910aafdbe2ffd00897b22d06bd01c7d0ab29534f9d42c4e28598d0dfa7ba4" Apr 22 21:13:09.664504 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:13:09.664482 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c41910aafdbe2ffd00897b22d06bd01c7d0ab29534f9d42c4e28598d0dfa7ba4\": container with ID starting with c41910aafdbe2ffd00897b22d06bd01c7d0ab29534f9d42c4e28598d0dfa7ba4 not found: ID does not exist" containerID="c41910aafdbe2ffd00897b22d06bd01c7d0ab29534f9d42c4e28598d0dfa7ba4" Apr 22 21:13:09.664542 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.664505 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c41910aafdbe2ffd00897b22d06bd01c7d0ab29534f9d42c4e28598d0dfa7ba4"} err="failed to get container status \"c41910aafdbe2ffd00897b22d06bd01c7d0ab29534f9d42c4e28598d0dfa7ba4\": rpc error: code = NotFound desc = could not find container \"c41910aafdbe2ffd00897b22d06bd01c7d0ab29534f9d42c4e28598d0dfa7ba4\": container with ID starting with c41910aafdbe2ffd00897b22d06bd01c7d0ab29534f9d42c4e28598d0dfa7ba4 not found: ID does not exist" Apr 22 21:13:09.664542 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.664536 2580 scope.go:117] "RemoveContainer" containerID="2b9abfd2c0207e480996e62b704393dae3a5dcd273ad14cf1f093abb36451747" Apr 22 21:13:09.664778 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:13:09.664759 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b9abfd2c0207e480996e62b704393dae3a5dcd273ad14cf1f093abb36451747\": container with ID starting with 2b9abfd2c0207e480996e62b704393dae3a5dcd273ad14cf1f093abb36451747 not found: ID does not exist" containerID="2b9abfd2c0207e480996e62b704393dae3a5dcd273ad14cf1f093abb36451747" Apr 22 21:13:09.664819 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.664785 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b9abfd2c0207e480996e62b704393dae3a5dcd273ad14cf1f093abb36451747"} err="failed to get container status \"2b9abfd2c0207e480996e62b704393dae3a5dcd273ad14cf1f093abb36451747\": rpc error: code = NotFound desc = could not find container \"2b9abfd2c0207e480996e62b704393dae3a5dcd273ad14cf1f093abb36451747\": container with ID starting with 2b9abfd2c0207e480996e62b704393dae3a5dcd273ad14cf1f093abb36451747 not found: ID does not exist" Apr 22 21:13:09.664819 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.664802 2580 scope.go:117] "RemoveContainer" containerID="6566a988615a9096fbb04663e61cdc64bcfc829859946fa8845baa4bde61c243" Apr 22 21:13:09.665086 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:13:09.665061 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6566a988615a9096fbb04663e61cdc64bcfc829859946fa8845baa4bde61c243\": container with ID starting with 6566a988615a9096fbb04663e61cdc64bcfc829859946fa8845baa4bde61c243 not found: ID does not exist" containerID="6566a988615a9096fbb04663e61cdc64bcfc829859946fa8845baa4bde61c243" Apr 22 21:13:09.665170 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.665092 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6566a988615a9096fbb04663e61cdc64bcfc829859946fa8845baa4bde61c243"} err="failed to get container status \"6566a988615a9096fbb04663e61cdc64bcfc829859946fa8845baa4bde61c243\": rpc error: code = NotFound desc = could not find container \"6566a988615a9096fbb04663e61cdc64bcfc829859946fa8845baa4bde61c243\": container with ID starting with 6566a988615a9096fbb04663e61cdc64bcfc829859946fa8845baa4bde61c243 not found: ID does not exist" Apr 22 21:13:09.665170 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.665111 2580 scope.go:117] "RemoveContainer" containerID="786758fdf24d3398ecb70b961c531b435d50172dce4d85179f4b477ee99b6f2b" Apr 22 21:13:09.665322 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:13:09.665299 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"786758fdf24d3398ecb70b961c531b435d50172dce4d85179f4b477ee99b6f2b\": container with ID starting with 786758fdf24d3398ecb70b961c531b435d50172dce4d85179f4b477ee99b6f2b not found: ID does not exist" containerID="786758fdf24d3398ecb70b961c531b435d50172dce4d85179f4b477ee99b6f2b" Apr 22 21:13:09.665411 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.665325 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"786758fdf24d3398ecb70b961c531b435d50172dce4d85179f4b477ee99b6f2b"} err="failed to get container status \"786758fdf24d3398ecb70b961c531b435d50172dce4d85179f4b477ee99b6f2b\": rpc error: code = NotFound desc = could not find container \"786758fdf24d3398ecb70b961c531b435d50172dce4d85179f4b477ee99b6f2b\": container with ID starting with 786758fdf24d3398ecb70b961c531b435d50172dce4d85179f4b477ee99b6f2b not found: ID does not exist" Apr 22 21:13:09.665411 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.665338 2580 scope.go:117] "RemoveContainer" containerID="129ec3dca79759e71f616e16ca7a3bb4d7aed852e286a93bef699183619ce294" Apr 22 21:13:09.665613 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:13:09.665567 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"129ec3dca79759e71f616e16ca7a3bb4d7aed852e286a93bef699183619ce294\": container with ID starting with 129ec3dca79759e71f616e16ca7a3bb4d7aed852e286a93bef699183619ce294 not found: ID does not exist" containerID="129ec3dca79759e71f616e16ca7a3bb4d7aed852e286a93bef699183619ce294" Apr 22 21:13:09.665613 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.665591 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"129ec3dca79759e71f616e16ca7a3bb4d7aed852e286a93bef699183619ce294"} err="failed to get container status \"129ec3dca79759e71f616e16ca7a3bb4d7aed852e286a93bef699183619ce294\": rpc error: code = NotFound desc = could not find container \"129ec3dca79759e71f616e16ca7a3bb4d7aed852e286a93bef699183619ce294\": container with ID starting with 129ec3dca79759e71f616e16ca7a3bb4d7aed852e286a93bef699183619ce294 not found: ID does not exist" Apr 22 21:13:09.665613 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.665606 2580 scope.go:117] "RemoveContainer" containerID="437903f98557692143da4e30c0d6e538c5015694bf8b8d65ac321b4c57ed3201" Apr 22 21:13:09.665909 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:13:09.665845 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"437903f98557692143da4e30c0d6e538c5015694bf8b8d65ac321b4c57ed3201\": container with ID starting with 437903f98557692143da4e30c0d6e538c5015694bf8b8d65ac321b4c57ed3201 not found: ID does not exist" containerID="437903f98557692143da4e30c0d6e538c5015694bf8b8d65ac321b4c57ed3201" Apr 22 21:13:09.665959 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.665915 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"437903f98557692143da4e30c0d6e538c5015694bf8b8d65ac321b4c57ed3201"} err="failed to get container status \"437903f98557692143da4e30c0d6e538c5015694bf8b8d65ac321b4c57ed3201\": rpc error: code = NotFound desc = could not find container \"437903f98557692143da4e30c0d6e538c5015694bf8b8d65ac321b4c57ed3201\": container with ID starting with 437903f98557692143da4e30c0d6e538c5015694bf8b8d65ac321b4c57ed3201 not found: ID does not exist" Apr 22 21:13:09.665959 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.665929 2580 scope.go:117] "RemoveContainer" containerID="0758042218fd8496ab7a7715f31a4dfe9b7b7d47fb65820ca5b293b9da3c394f" Apr 22 21:13:09.666140 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:13:09.666124 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0758042218fd8496ab7a7715f31a4dfe9b7b7d47fb65820ca5b293b9da3c394f\": container with ID starting with 0758042218fd8496ab7a7715f31a4dfe9b7b7d47fb65820ca5b293b9da3c394f not found: ID does not exist" containerID="0758042218fd8496ab7a7715f31a4dfe9b7b7d47fb65820ca5b293b9da3c394f" Apr 22 21:13:09.666179 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.666146 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0758042218fd8496ab7a7715f31a4dfe9b7b7d47fb65820ca5b293b9da3c394f"} err="failed to get container status \"0758042218fd8496ab7a7715f31a4dfe9b7b7d47fb65820ca5b293b9da3c394f\": rpc error: code = NotFound desc = could not find container \"0758042218fd8496ab7a7715f31a4dfe9b7b7d47fb65820ca5b293b9da3c394f\": container with ID starting with 0758042218fd8496ab7a7715f31a4dfe9b7b7d47fb65820ca5b293b9da3c394f not found: ID does not exist" Apr 22 21:13:09.667148 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.667134 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.669739 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.669722 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Apr 22 21:13:09.669834 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.669744 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Apr 22 21:13:09.669834 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.669814 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Apr 22 21:13:09.669970 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.669951 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Apr 22 21:13:09.670026 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.670003 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Apr 22 21:13:09.670221 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.670187 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Apr 22 21:13:09.670221 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.670240 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Apr 22 21:13:09.670478 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.670267 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Apr 22 21:13:09.670478 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.670338 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-b6b78\"" Apr 22 21:13:09.670595 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.670554 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Apr 22 21:13:09.670595 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.670555 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Apr 22 21:13:09.670595 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.670558 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4s76p81ptr7as\"" Apr 22 21:13:09.670775 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.670736 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Apr 22 21:13:09.673803 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.673782 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Apr 22 21:13:09.681331 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.680889 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 22 21:13:09.682584 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.682562 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Apr 22 21:13:09.782172 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.782128 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.782172 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.782178 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.782350 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.782204 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.782350 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.782224 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.782350 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.782244 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-config-out\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.782350 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.782270 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.782350 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.782297 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.782350 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.782329 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.782350 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.782345 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.782554 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.782369 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-config\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.782554 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.782419 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.782554 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.782439 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.782554 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.782457 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.782554 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.782472 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.782554 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.782493 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.782554 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.782533 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-web-config\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.782783 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.782554 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.782783 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.782582 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvvqf\" (UniqueName: \"kubernetes.io/projected/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-kube-api-access-zvvqf\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.861700 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.861628 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc" path="/var/lib/kubelet/pods/503b81b1-1119-4a8d-b1bd-fa6cf6b9ecfc/volumes" Apr 22 21:13:09.883615 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.883592 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.883692 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.883621 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.883692 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.883642 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.883692 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.883659 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.883692 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.883679 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-config\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.883914 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.883704 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.883914 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.883730 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.883914 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.883892 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.884074 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.883929 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.884074 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.883958 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.884074 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.883995 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-web-config\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.884074 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.884028 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.884074 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.884063 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zvvqf\" (UniqueName: \"kubernetes.io/projected/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-kube-api-access-zvvqf\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.884321 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.884092 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.884321 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.884117 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.884321 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.884141 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.884321 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.884147 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.884321 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.884164 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.884321 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.884195 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-config-out\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.884799 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.884605 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.887021 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.886994 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.887144 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.887127 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-config-out\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.887212 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.887138 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.887212 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.887158 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-config\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.887322 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.887310 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.887372 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.887326 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.888602 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.887941 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.888602 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.888219 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.888602 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.888269 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.888602 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.888552 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.888876 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.888828 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.889686 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.889661 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-web-config\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.889803 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.889780 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.889901 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.889882 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.890448 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.890431 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.895164 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.895145 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvvqf\" (UniqueName: \"kubernetes.io/projected/e85fbea5-3c3a-4464-85d0-97d3aa56ea89-kube-api-access-zvvqf\") pod \"prometheus-k8s-0\" (UID: \"e85fbea5-3c3a-4464-85d0-97d3aa56ea89\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:09.979369 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:09.979339 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:13:10.108317 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:10.108294 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 22 21:13:10.110058 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:13:10.110030 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode85fbea5_3c3a_4464_85d0_97d3aa56ea89.slice/crio-bf27ec039a46d4e0908b956598bd4e651fd9d5379db174c452186d5673cbce34 WatchSource:0}: Error finding container bf27ec039a46d4e0908b956598bd4e651fd9d5379db174c452186d5673cbce34: Status 404 returned error can't find the container with id bf27ec039a46d4e0908b956598bd4e651fd9d5379db174c452186d5673cbce34 Apr 22 21:13:10.619997 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:10.619963 2580 generic.go:358] "Generic (PLEG): container finished" podID="e85fbea5-3c3a-4464-85d0-97d3aa56ea89" containerID="7f1a56cd7a3996d18e128592f7e3bc29603ffb8b7d7a146ee7f7d456604e0d67" exitCode=0 Apr 22 21:13:10.620161 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:10.620042 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e85fbea5-3c3a-4464-85d0-97d3aa56ea89","Type":"ContainerDied","Data":"7f1a56cd7a3996d18e128592f7e3bc29603ffb8b7d7a146ee7f7d456604e0d67"} Apr 22 21:13:10.620161 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:10.620070 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e85fbea5-3c3a-4464-85d0-97d3aa56ea89","Type":"ContainerStarted","Data":"bf27ec039a46d4e0908b956598bd4e651fd9d5379db174c452186d5673cbce34"} Apr 22 21:13:11.625295 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:11.625258 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e85fbea5-3c3a-4464-85d0-97d3aa56ea89","Type":"ContainerStarted","Data":"7aee392ff612ba34280ff5eb152c413ce259ef2965a9332dd36fe35912e0f608"} Apr 22 21:13:11.625295 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:11.625291 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e85fbea5-3c3a-4464-85d0-97d3aa56ea89","Type":"ContainerStarted","Data":"4e00eb866eeefd94302af77fcb46a2b60ba9e726c1c25f79385315ebcaa03e7e"} Apr 22 21:13:11.625295 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:11.625300 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e85fbea5-3c3a-4464-85d0-97d3aa56ea89","Type":"ContainerStarted","Data":"cc33ebd473915f53c068cfcea9e193bf32dfdb20336f1293e81f47e40831b7c5"} Apr 22 21:13:11.625891 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:11.625309 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e85fbea5-3c3a-4464-85d0-97d3aa56ea89","Type":"ContainerStarted","Data":"b33c9d882763c4437c86b54733bd63c70a0961d8362e8a8ca92770e432af8674"} Apr 22 21:13:11.625891 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:11.625319 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e85fbea5-3c3a-4464-85d0-97d3aa56ea89","Type":"ContainerStarted","Data":"b0614c7bf4b52e2d409be6718f826645fa91eb43b364fd2cf70caac9230cb9bb"} Apr 22 21:13:11.625891 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:11.625326 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e85fbea5-3c3a-4464-85d0-97d3aa56ea89","Type":"ContainerStarted","Data":"0b6902cddb5970886beef60b15a05b53c1ceb1044630e34b636f4a73446f2981"} Apr 22 21:13:11.654563 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:11.654453 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.654436462 podStartE2EDuration="2.654436462s" podCreationTimestamp="2026-04-22 21:13:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-22 21:13:11.651701567 +0000 UTC m=+232.377598167" watchObservedRunningTime="2026-04-22 21:13:11.654436462 +0000 UTC m=+232.380333060" Apr 22 21:13:14.979866 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:13:14.979804 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:14:09.980395 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:14:09.980363 2580 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:14:09.995822 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:14:09.995802 2580 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:14:10.805161 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:14:10.805133 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Apr 22 21:14:19.738656 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:14:19.738627 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ngls_b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b/ovn-acl-logging/0.log" Apr 22 21:14:19.739120 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:14:19.739083 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ngls_b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b/ovn-acl-logging/0.log" Apr 22 21:14:19.743669 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:14:19.743644 2580 kubelet.go:1628] "Image garbage collection succeeded" Apr 22 21:15:12.409994 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:15:12.409962 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-txlcq"] Apr 22 21:15:12.413412 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:15:12.413390 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txlcq" Apr 22 21:15:12.416532 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:15:12.416508 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Apr 22 21:15:12.418428 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:15:12.418406 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-txlcq"] Apr 22 21:15:12.488926 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:15:12.488893 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9efd7cea-0616-4971-9f61-d3b0e3e9c156-kubelet-config\") pod \"global-pull-secret-syncer-txlcq\" (UID: \"9efd7cea-0616-4971-9f61-d3b0e3e9c156\") " pod="kube-system/global-pull-secret-syncer-txlcq" Apr 22 21:15:12.489082 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:15:12.488945 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9efd7cea-0616-4971-9f61-d3b0e3e9c156-original-pull-secret\") pod \"global-pull-secret-syncer-txlcq\" (UID: \"9efd7cea-0616-4971-9f61-d3b0e3e9c156\") " pod="kube-system/global-pull-secret-syncer-txlcq" Apr 22 21:15:12.489082 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:15:12.488979 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9efd7cea-0616-4971-9f61-d3b0e3e9c156-dbus\") pod \"global-pull-secret-syncer-txlcq\" (UID: \"9efd7cea-0616-4971-9f61-d3b0e3e9c156\") " pod="kube-system/global-pull-secret-syncer-txlcq" Apr 22 21:15:12.589785 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:15:12.589753 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9efd7cea-0616-4971-9f61-d3b0e3e9c156-kubelet-config\") pod \"global-pull-secret-syncer-txlcq\" (UID: \"9efd7cea-0616-4971-9f61-d3b0e3e9c156\") " pod="kube-system/global-pull-secret-syncer-txlcq" Apr 22 21:15:12.589958 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:15:12.589795 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9efd7cea-0616-4971-9f61-d3b0e3e9c156-original-pull-secret\") pod \"global-pull-secret-syncer-txlcq\" (UID: \"9efd7cea-0616-4971-9f61-d3b0e3e9c156\") " pod="kube-system/global-pull-secret-syncer-txlcq" Apr 22 21:15:12.589958 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:15:12.589819 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9efd7cea-0616-4971-9f61-d3b0e3e9c156-dbus\") pod \"global-pull-secret-syncer-txlcq\" (UID: \"9efd7cea-0616-4971-9f61-d3b0e3e9c156\") " pod="kube-system/global-pull-secret-syncer-txlcq" Apr 22 21:15:12.589958 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:15:12.589906 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9efd7cea-0616-4971-9f61-d3b0e3e9c156-kubelet-config\") pod \"global-pull-secret-syncer-txlcq\" (UID: \"9efd7cea-0616-4971-9f61-d3b0e3e9c156\") " pod="kube-system/global-pull-secret-syncer-txlcq" Apr 22 21:15:12.590128 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:15:12.589993 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9efd7cea-0616-4971-9f61-d3b0e3e9c156-dbus\") pod \"global-pull-secret-syncer-txlcq\" (UID: \"9efd7cea-0616-4971-9f61-d3b0e3e9c156\") " pod="kube-system/global-pull-secret-syncer-txlcq" Apr 22 21:15:12.592234 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:15:12.592214 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9efd7cea-0616-4971-9f61-d3b0e3e9c156-original-pull-secret\") pod \"global-pull-secret-syncer-txlcq\" (UID: \"9efd7cea-0616-4971-9f61-d3b0e3e9c156\") " pod="kube-system/global-pull-secret-syncer-txlcq" Apr 22 21:15:12.723433 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:15:12.723363 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txlcq" Apr 22 21:15:12.839010 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:15:12.838991 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-txlcq"] Apr 22 21:15:12.840787 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:15:12.840761 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9efd7cea_0616_4971_9f61_d3b0e3e9c156.slice/crio-03da2563bc9c0cc2c2619b42477d428a75c97d8c82fbd2d2f6ba6fdba2ee5092 WatchSource:0}: Error finding container 03da2563bc9c0cc2c2619b42477d428a75c97d8c82fbd2d2f6ba6fdba2ee5092: Status 404 returned error can't find the container with id 03da2563bc9c0cc2c2619b42477d428a75c97d8c82fbd2d2f6ba6fdba2ee5092 Apr 22 21:15:12.842396 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:15:12.842380 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 22 21:15:12.949112 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:15:12.949068 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-txlcq" event={"ID":"9efd7cea-0616-4971-9f61-d3b0e3e9c156","Type":"ContainerStarted","Data":"03da2563bc9c0cc2c2619b42477d428a75c97d8c82fbd2d2f6ba6fdba2ee5092"} Apr 22 21:15:17.963380 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:15:17.963339 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-txlcq" event={"ID":"9efd7cea-0616-4971-9f61-d3b0e3e9c156","Type":"ContainerStarted","Data":"b51158053f563716d34a5a5b598e4b00fae315739325dc528b0bdb58611cc775"} Apr 22 21:15:17.975517 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:15:17.975421 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-txlcq" podStartSLOduration=1.655129488 podStartE2EDuration="5.975406181s" podCreationTimestamp="2026-04-22 21:15:12 +0000 UTC" firstStartedPulling="2026-04-22 21:15:12.842521877 +0000 UTC m=+353.568418455" lastFinishedPulling="2026-04-22 21:15:17.162798557 +0000 UTC m=+357.888695148" observedRunningTime="2026-04-22 21:15:17.975004816 +0000 UTC m=+358.700901417" watchObservedRunningTime="2026-04-22 21:15:17.975406181 +0000 UTC m=+358.701302795" Apr 22 21:16:26.768872 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:26.768776 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65d8664856-x586c"] Apr 22 21:16:26.770917 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:26.770899 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65d8664856-x586c" Apr 22 21:16:26.774900 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:26.774877 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Apr 22 21:16:26.775026 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:26.774948 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-s8gxl\"" Apr 22 21:16:26.775026 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:26.775002 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Apr 22 21:16:26.775145 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:26.775028 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Apr 22 21:16:26.775186 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:26.775148 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Apr 22 21:16:26.788703 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:26.788682 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65d8664856-x586c"] Apr 22 21:16:26.878829 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:26.878799 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xb4nn\" (UniqueName: \"kubernetes.io/projected/38f8d29f-76b8-4964-8085-5603aac1059e-kube-api-access-xb4nn\") pod \"opendatahub-operator-controller-manager-65d8664856-x586c\" (UID: \"38f8d29f-76b8-4964-8085-5603aac1059e\") " pod="opendatahub/opendatahub-operator-controller-manager-65d8664856-x586c" Apr 22 21:16:26.878829 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:26.878838 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/38f8d29f-76b8-4964-8085-5603aac1059e-webhook-cert\") pod \"opendatahub-operator-controller-manager-65d8664856-x586c\" (UID: \"38f8d29f-76b8-4964-8085-5603aac1059e\") " pod="opendatahub/opendatahub-operator-controller-manager-65d8664856-x586c" Apr 22 21:16:26.879072 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:26.878959 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/38f8d29f-76b8-4964-8085-5603aac1059e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65d8664856-x586c\" (UID: \"38f8d29f-76b8-4964-8085-5603aac1059e\") " pod="opendatahub/opendatahub-operator-controller-manager-65d8664856-x586c" Apr 22 21:16:26.980075 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:26.980048 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xb4nn\" (UniqueName: \"kubernetes.io/projected/38f8d29f-76b8-4964-8085-5603aac1059e-kube-api-access-xb4nn\") pod \"opendatahub-operator-controller-manager-65d8664856-x586c\" (UID: \"38f8d29f-76b8-4964-8085-5603aac1059e\") " pod="opendatahub/opendatahub-operator-controller-manager-65d8664856-x586c" Apr 22 21:16:26.980253 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:26.980085 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/38f8d29f-76b8-4964-8085-5603aac1059e-webhook-cert\") pod \"opendatahub-operator-controller-manager-65d8664856-x586c\" (UID: \"38f8d29f-76b8-4964-8085-5603aac1059e\") " pod="opendatahub/opendatahub-operator-controller-manager-65d8664856-x586c" Apr 22 21:16:26.980253 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:26.980127 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/38f8d29f-76b8-4964-8085-5603aac1059e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65d8664856-x586c\" (UID: \"38f8d29f-76b8-4964-8085-5603aac1059e\") " pod="opendatahub/opendatahub-operator-controller-manager-65d8664856-x586c" Apr 22 21:16:26.982633 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:26.982604 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/38f8d29f-76b8-4964-8085-5603aac1059e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65d8664856-x586c\" (UID: \"38f8d29f-76b8-4964-8085-5603aac1059e\") " pod="opendatahub/opendatahub-operator-controller-manager-65d8664856-x586c" Apr 22 21:16:26.982746 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:26.982677 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/38f8d29f-76b8-4964-8085-5603aac1059e-webhook-cert\") pod \"opendatahub-operator-controller-manager-65d8664856-x586c\" (UID: \"38f8d29f-76b8-4964-8085-5603aac1059e\") " pod="opendatahub/opendatahub-operator-controller-manager-65d8664856-x586c" Apr 22 21:16:26.987594 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:26.987567 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xb4nn\" (UniqueName: \"kubernetes.io/projected/38f8d29f-76b8-4964-8085-5603aac1059e-kube-api-access-xb4nn\") pod \"opendatahub-operator-controller-manager-65d8664856-x586c\" (UID: \"38f8d29f-76b8-4964-8085-5603aac1059e\") " pod="opendatahub/opendatahub-operator-controller-manager-65d8664856-x586c" Apr 22 21:16:27.081101 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:27.081035 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65d8664856-x586c" Apr 22 21:16:27.205153 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:27.205074 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65d8664856-x586c"] Apr 22 21:16:27.208006 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:16:27.207977 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38f8d29f_76b8_4964_8085_5603aac1059e.slice/crio-0e8294759ac1d560b84dbf7436dae32322dca21d8f977cc83c99dd8cc260814f WatchSource:0}: Error finding container 0e8294759ac1d560b84dbf7436dae32322dca21d8f977cc83c99dd8cc260814f: Status 404 returned error can't find the container with id 0e8294759ac1d560b84dbf7436dae32322dca21d8f977cc83c99dd8cc260814f Apr 22 21:16:28.158093 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:28.158054 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65d8664856-x586c" event={"ID":"38f8d29f-76b8-4964-8085-5603aac1059e","Type":"ContainerStarted","Data":"0e8294759ac1d560b84dbf7436dae32322dca21d8f977cc83c99dd8cc260814f"} Apr 22 21:16:30.165827 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:30.165792 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65d8664856-x586c" event={"ID":"38f8d29f-76b8-4964-8085-5603aac1059e","Type":"ContainerStarted","Data":"85093419fdd04971e3950ec8027498c9ba61987b06005e76add9d9a9e742f643"} Apr 22 21:16:30.166240 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:30.165946 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-65d8664856-x586c" Apr 22 21:16:30.186342 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:30.186297 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-65d8664856-x586c" podStartSLOduration=1.557924806 podStartE2EDuration="4.186283615s" podCreationTimestamp="2026-04-22 21:16:26 +0000 UTC" firstStartedPulling="2026-04-22 21:16:27.209624779 +0000 UTC m=+427.935521358" lastFinishedPulling="2026-04-22 21:16:29.837983589 +0000 UTC m=+430.563880167" observedRunningTime="2026-04-22 21:16:30.185659173 +0000 UTC m=+430.911555774" watchObservedRunningTime="2026-04-22 21:16:30.186283615 +0000 UTC m=+430.912180215" Apr 22 21:16:33.015481 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:33.015447 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5db7bf5949-jf9h6"] Apr 22 21:16:33.017586 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:33.017570 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5db7bf5949-jf9h6" Apr 22 21:16:33.020168 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:33.020146 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Apr 22 21:16:33.021552 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:33.021523 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Apr 22 21:16:33.021668 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:33.021630 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Apr 22 21:16:33.021742 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:33.021681 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Apr 22 21:16:33.021742 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:33.021730 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Apr 22 21:16:33.021876 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:33.021738 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-r8sgt\"" Apr 22 21:16:33.027898 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:33.027877 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5db7bf5949-jf9h6"] Apr 22 21:16:33.131025 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:33.130987 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/522c73e8-3953-4540-b97b-156bfea8e30a-manager-config\") pod \"lws-controller-manager-5db7bf5949-jf9h6\" (UID: \"522c73e8-3953-4540-b97b-156bfea8e30a\") " pod="openshift-lws-operator/lws-controller-manager-5db7bf5949-jf9h6" Apr 22 21:16:33.131219 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:33.131045 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/522c73e8-3953-4540-b97b-156bfea8e30a-metrics-cert\") pod \"lws-controller-manager-5db7bf5949-jf9h6\" (UID: \"522c73e8-3953-4540-b97b-156bfea8e30a\") " pod="openshift-lws-operator/lws-controller-manager-5db7bf5949-jf9h6" Apr 22 21:16:33.131219 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:33.131097 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/522c73e8-3953-4540-b97b-156bfea8e30a-cert\") pod \"lws-controller-manager-5db7bf5949-jf9h6\" (UID: \"522c73e8-3953-4540-b97b-156bfea8e30a\") " pod="openshift-lws-operator/lws-controller-manager-5db7bf5949-jf9h6" Apr 22 21:16:33.131219 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:33.131172 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f5jw\" (UniqueName: \"kubernetes.io/projected/522c73e8-3953-4540-b97b-156bfea8e30a-kube-api-access-4f5jw\") pod \"lws-controller-manager-5db7bf5949-jf9h6\" (UID: \"522c73e8-3953-4540-b97b-156bfea8e30a\") " pod="openshift-lws-operator/lws-controller-manager-5db7bf5949-jf9h6" Apr 22 21:16:33.232358 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:33.232317 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4f5jw\" (UniqueName: \"kubernetes.io/projected/522c73e8-3953-4540-b97b-156bfea8e30a-kube-api-access-4f5jw\") pod \"lws-controller-manager-5db7bf5949-jf9h6\" (UID: \"522c73e8-3953-4540-b97b-156bfea8e30a\") " pod="openshift-lws-operator/lws-controller-manager-5db7bf5949-jf9h6" Apr 22 21:16:33.232548 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:33.232370 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/522c73e8-3953-4540-b97b-156bfea8e30a-manager-config\") pod \"lws-controller-manager-5db7bf5949-jf9h6\" (UID: \"522c73e8-3953-4540-b97b-156bfea8e30a\") " pod="openshift-lws-operator/lws-controller-manager-5db7bf5949-jf9h6" Apr 22 21:16:33.232616 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:33.232557 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/522c73e8-3953-4540-b97b-156bfea8e30a-metrics-cert\") pod \"lws-controller-manager-5db7bf5949-jf9h6\" (UID: \"522c73e8-3953-4540-b97b-156bfea8e30a\") " pod="openshift-lws-operator/lws-controller-manager-5db7bf5949-jf9h6" Apr 22 21:16:33.232616 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:33.232603 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/522c73e8-3953-4540-b97b-156bfea8e30a-cert\") pod \"lws-controller-manager-5db7bf5949-jf9h6\" (UID: \"522c73e8-3953-4540-b97b-156bfea8e30a\") " pod="openshift-lws-operator/lws-controller-manager-5db7bf5949-jf9h6" Apr 22 21:16:33.233155 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:33.233127 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/522c73e8-3953-4540-b97b-156bfea8e30a-manager-config\") pod \"lws-controller-manager-5db7bf5949-jf9h6\" (UID: \"522c73e8-3953-4540-b97b-156bfea8e30a\") " pod="openshift-lws-operator/lws-controller-manager-5db7bf5949-jf9h6" Apr 22 21:16:33.235126 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:33.235106 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/522c73e8-3953-4540-b97b-156bfea8e30a-metrics-cert\") pod \"lws-controller-manager-5db7bf5949-jf9h6\" (UID: \"522c73e8-3953-4540-b97b-156bfea8e30a\") " pod="openshift-lws-operator/lws-controller-manager-5db7bf5949-jf9h6" Apr 22 21:16:33.235222 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:33.235125 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/522c73e8-3953-4540-b97b-156bfea8e30a-cert\") pod \"lws-controller-manager-5db7bf5949-jf9h6\" (UID: \"522c73e8-3953-4540-b97b-156bfea8e30a\") " pod="openshift-lws-operator/lws-controller-manager-5db7bf5949-jf9h6" Apr 22 21:16:33.241299 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:33.241273 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f5jw\" (UniqueName: \"kubernetes.io/projected/522c73e8-3953-4540-b97b-156bfea8e30a-kube-api-access-4f5jw\") pod \"lws-controller-manager-5db7bf5949-jf9h6\" (UID: \"522c73e8-3953-4540-b97b-156bfea8e30a\") " pod="openshift-lws-operator/lws-controller-manager-5db7bf5949-jf9h6" Apr 22 21:16:33.326490 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:33.326405 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5db7bf5949-jf9h6" Apr 22 21:16:33.445658 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:33.445631 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5db7bf5949-jf9h6"] Apr 22 21:16:33.448198 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:16:33.448170 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod522c73e8_3953_4540_b97b_156bfea8e30a.slice/crio-b9287677d7453f6ad5b43d4fa1fbaa2e6da225df51eab53653c65f2294858f38 WatchSource:0}: Error finding container b9287677d7453f6ad5b43d4fa1fbaa2e6da225df51eab53653c65f2294858f38: Status 404 returned error can't find the container with id b9287677d7453f6ad5b43d4fa1fbaa2e6da225df51eab53653c65f2294858f38 Apr 22 21:16:34.180099 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:34.180062 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5db7bf5949-jf9h6" event={"ID":"522c73e8-3953-4540-b97b-156bfea8e30a","Type":"ContainerStarted","Data":"b9287677d7453f6ad5b43d4fa1fbaa2e6da225df51eab53653c65f2294858f38"} Apr 22 21:16:36.188168 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:36.188132 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5db7bf5949-jf9h6" event={"ID":"522c73e8-3953-4540-b97b-156bfea8e30a","Type":"ContainerStarted","Data":"0f08adac9966dbbe961873564ae32331c6c85c8e04ac5e30f2dfd79815caf398"} Apr 22 21:16:36.188563 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:36.188293 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5db7bf5949-jf9h6" Apr 22 21:16:36.204688 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:36.204648 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5db7bf5949-jf9h6" podStartSLOduration=1.984306833 podStartE2EDuration="4.204637582s" podCreationTimestamp="2026-04-22 21:16:32 +0000 UTC" firstStartedPulling="2026-04-22 21:16:33.449941217 +0000 UTC m=+434.175837794" lastFinishedPulling="2026-04-22 21:16:35.670271966 +0000 UTC m=+436.396168543" observedRunningTime="2026-04-22 21:16:36.202561325 +0000 UTC m=+436.928457926" watchObservedRunningTime="2026-04-22 21:16:36.204637582 +0000 UTC m=+436.930534182" Apr 22 21:16:41.171452 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:41.171424 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-65d8664856-x586c" Apr 22 21:16:47.194105 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:16:47.194074 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5db7bf5949-jf9h6" Apr 22 21:17:28.757984 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.757947 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc"] Apr 22 21:17:28.761128 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.761106 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:28.766021 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.765997 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"istio-ca-root-cert\"" Apr 22 21:17:28.766155 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.766060 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-qjd75\"" Apr 22 21:17:28.781827 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.781802 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc"] Apr 22 21:17:28.843971 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.843936 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7d8ffd7-bf30-44b4-88af-5238fea731b6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-55cc67557f2z5tc\" (UID: \"d7d8ffd7-bf30-44b4-88af-5238fea731b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:28.844145 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.843980 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7d8ffd7-bf30-44b4-88af-5238fea731b6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-55cc67557f2z5tc\" (UID: \"d7d8ffd7-bf30-44b4-88af-5238fea731b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:28.844145 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.844043 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlhx9\" (UniqueName: \"kubernetes.io/projected/d7d8ffd7-bf30-44b4-88af-5238fea731b6-kube-api-access-vlhx9\") pod \"data-science-gateway-data-science-gateway-class-55cc67557f2z5tc\" (UID: \"d7d8ffd7-bf30-44b4-88af-5238fea731b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:28.844145 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.844113 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7d8ffd7-bf30-44b4-88af-5238fea731b6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-55cc67557f2z5tc\" (UID: \"d7d8ffd7-bf30-44b4-88af-5238fea731b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:28.844145 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.844137 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7d8ffd7-bf30-44b4-88af-5238fea731b6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-55cc67557f2z5tc\" (UID: \"d7d8ffd7-bf30-44b4-88af-5238fea731b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:28.844361 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.844160 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7d8ffd7-bf30-44b4-88af-5238fea731b6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-55cc67557f2z5tc\" (UID: \"d7d8ffd7-bf30-44b4-88af-5238fea731b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:28.844361 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.844191 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7d8ffd7-bf30-44b4-88af-5238fea731b6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-55cc67557f2z5tc\" (UID: \"d7d8ffd7-bf30-44b4-88af-5238fea731b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:28.844361 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.844209 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7d8ffd7-bf30-44b4-88af-5238fea731b6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-55cc67557f2z5tc\" (UID: \"d7d8ffd7-bf30-44b4-88af-5238fea731b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:28.844361 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.844237 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7d8ffd7-bf30-44b4-88af-5238fea731b6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-55cc67557f2z5tc\" (UID: \"d7d8ffd7-bf30-44b4-88af-5238fea731b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:28.945474 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.945433 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7d8ffd7-bf30-44b4-88af-5238fea731b6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-55cc67557f2z5tc\" (UID: \"d7d8ffd7-bf30-44b4-88af-5238fea731b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:28.945631 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.945490 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7d8ffd7-bf30-44b4-88af-5238fea731b6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-55cc67557f2z5tc\" (UID: \"d7d8ffd7-bf30-44b4-88af-5238fea731b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:28.945631 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.945520 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7d8ffd7-bf30-44b4-88af-5238fea731b6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-55cc67557f2z5tc\" (UID: \"d7d8ffd7-bf30-44b4-88af-5238fea731b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:28.945631 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.945559 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7d8ffd7-bf30-44b4-88af-5238fea731b6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-55cc67557f2z5tc\" (UID: \"d7d8ffd7-bf30-44b4-88af-5238fea731b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:28.945631 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.945590 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7d8ffd7-bf30-44b4-88af-5238fea731b6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-55cc67557f2z5tc\" (UID: \"d7d8ffd7-bf30-44b4-88af-5238fea731b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:28.945838 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.945729 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7d8ffd7-bf30-44b4-88af-5238fea731b6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-55cc67557f2z5tc\" (UID: \"d7d8ffd7-bf30-44b4-88af-5238fea731b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:28.945986 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.945965 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7d8ffd7-bf30-44b4-88af-5238fea731b6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-55cc67557f2z5tc\" (UID: \"d7d8ffd7-bf30-44b4-88af-5238fea731b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:28.946115 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.946091 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7d8ffd7-bf30-44b4-88af-5238fea731b6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-55cc67557f2z5tc\" (UID: \"d7d8ffd7-bf30-44b4-88af-5238fea731b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:28.946187 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.946107 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7d8ffd7-bf30-44b4-88af-5238fea731b6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-55cc67557f2z5tc\" (UID: \"d7d8ffd7-bf30-44b4-88af-5238fea731b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:28.946187 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.946166 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7d8ffd7-bf30-44b4-88af-5238fea731b6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-55cc67557f2z5tc\" (UID: \"d7d8ffd7-bf30-44b4-88af-5238fea731b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:28.946293 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.946201 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7d8ffd7-bf30-44b4-88af-5238fea731b6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-55cc67557f2z5tc\" (UID: \"d7d8ffd7-bf30-44b4-88af-5238fea731b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:28.946293 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.946202 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vlhx9\" (UniqueName: \"kubernetes.io/projected/d7d8ffd7-bf30-44b4-88af-5238fea731b6-kube-api-access-vlhx9\") pod \"data-science-gateway-data-science-gateway-class-55cc67557f2z5tc\" (UID: \"d7d8ffd7-bf30-44b4-88af-5238fea731b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:28.946293 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.946266 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7d8ffd7-bf30-44b4-88af-5238fea731b6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-55cc67557f2z5tc\" (UID: \"d7d8ffd7-bf30-44b4-88af-5238fea731b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:28.946464 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.946444 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7d8ffd7-bf30-44b4-88af-5238fea731b6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-55cc67557f2z5tc\" (UID: \"d7d8ffd7-bf30-44b4-88af-5238fea731b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:28.948304 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.948274 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7d8ffd7-bf30-44b4-88af-5238fea731b6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-55cc67557f2z5tc\" (UID: \"d7d8ffd7-bf30-44b4-88af-5238fea731b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:28.948806 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.948785 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7d8ffd7-bf30-44b4-88af-5238fea731b6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-55cc67557f2z5tc\" (UID: \"d7d8ffd7-bf30-44b4-88af-5238fea731b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:28.967586 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.967562 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlhx9\" (UniqueName: \"kubernetes.io/projected/d7d8ffd7-bf30-44b4-88af-5238fea731b6-kube-api-access-vlhx9\") pod \"data-science-gateway-data-science-gateway-class-55cc67557f2z5tc\" (UID: \"d7d8ffd7-bf30-44b4-88af-5238fea731b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:28.967808 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:28.967786 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7d8ffd7-bf30-44b4-88af-5238fea731b6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-55cc67557f2z5tc\" (UID: \"d7d8ffd7-bf30-44b4-88af-5238fea731b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:29.072680 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:29.072609 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:29.219087 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:29.219063 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc"] Apr 22 21:17:29.220739 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:17:29.220713 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7d8ffd7_bf30_44b4_88af_5238fea731b6.slice/crio-5aab3f5f8251ed209aab5ef266c8b0ec38d295ae28b2425cde01dbbab147b82c WatchSource:0}: Error finding container 5aab3f5f8251ed209aab5ef266c8b0ec38d295ae28b2425cde01dbbab147b82c: Status 404 returned error can't find the container with id 5aab3f5f8251ed209aab5ef266c8b0ec38d295ae28b2425cde01dbbab147b82c Apr 22 21:17:29.359687 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:29.359603 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" event={"ID":"d7d8ffd7-bf30-44b4-88af-5238fea731b6","Type":"ContainerStarted","Data":"5aab3f5f8251ed209aab5ef266c8b0ec38d295ae28b2425cde01dbbab147b82c"} Apr 22 21:17:31.791160 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:31.791119 2580 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Apr 22 21:17:31.791419 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:31.791193 2580 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Apr 22 21:17:31.791419 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:31.791223 2580 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Apr 22 21:17:32.370925 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:32.370885 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" event={"ID":"d7d8ffd7-bf30-44b4-88af-5238fea731b6","Type":"ContainerStarted","Data":"3ad82b10d7deec26d3566d05dbfd39d91097e4191471ee87a190ee9b5778e01e"} Apr 22 21:17:32.391870 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:32.391803 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" podStartSLOduration=1.823477853 podStartE2EDuration="4.391788524s" podCreationTimestamp="2026-04-22 21:17:28 +0000 UTC" firstStartedPulling="2026-04-22 21:17:29.222559231 +0000 UTC m=+489.948455812" lastFinishedPulling="2026-04-22 21:17:31.790869905 +0000 UTC m=+492.516766483" observedRunningTime="2026-04-22 21:17:32.390069625 +0000 UTC m=+493.115966225" watchObservedRunningTime="2026-04-22 21:17:32.391788524 +0000 UTC m=+493.117685186" Apr 22 21:17:33.073476 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:33.073441 2580 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:33.078220 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:33.078199 2580 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:33.374428 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:33.374350 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:33.375189 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:33.375171 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-55cc67557f2z5tc" Apr 22 21:17:44.979002 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:44.978968 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sw5vj"] Apr 22 21:17:44.984817 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:44.984789 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sw5vj" Apr 22 21:17:44.988397 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:44.987436 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Apr 22 21:17:44.988928 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:44.988902 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-vwbd8\"" Apr 22 21:17:44.989673 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:44.989654 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Apr 22 21:17:44.991384 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:44.991361 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sw5vj"] Apr 22 21:17:45.070401 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:45.070360 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzct7\" (UniqueName: \"kubernetes.io/projected/7bb5b705-2b88-495d-aaac-f0efd4c432c3-kube-api-access-pzct7\") pod \"kuadrant-operator-catalog-sw5vj\" (UID: \"7bb5b705-2b88-495d-aaac-f0efd4c432c3\") " pod="kuadrant-system/kuadrant-operator-catalog-sw5vj" Apr 22 21:17:45.171058 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:45.171022 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pzct7\" (UniqueName: \"kubernetes.io/projected/7bb5b705-2b88-495d-aaac-f0efd4c432c3-kube-api-access-pzct7\") pod \"kuadrant-operator-catalog-sw5vj\" (UID: \"7bb5b705-2b88-495d-aaac-f0efd4c432c3\") " pod="kuadrant-system/kuadrant-operator-catalog-sw5vj" Apr 22 21:17:45.179435 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:45.179410 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzct7\" (UniqueName: \"kubernetes.io/projected/7bb5b705-2b88-495d-aaac-f0efd4c432c3-kube-api-access-pzct7\") pod \"kuadrant-operator-catalog-sw5vj\" (UID: \"7bb5b705-2b88-495d-aaac-f0efd4c432c3\") " pod="kuadrant-system/kuadrant-operator-catalog-sw5vj" Apr 22 21:17:45.297602 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:45.297517 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sw5vj" Apr 22 21:17:45.359582 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:45.359527 2580 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sw5vj"] Apr 22 21:17:45.417990 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:45.417971 2580 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sw5vj"] Apr 22 21:17:45.420220 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:17:45.420186 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bb5b705_2b88_495d_aaac_f0efd4c432c3.slice/crio-49ffb489af41e135d50471a1b56a64be415248bf54a8e52d02d5824f2063c445 WatchSource:0}: Error finding container 49ffb489af41e135d50471a1b56a64be415248bf54a8e52d02d5824f2063c445: Status 404 returned error can't find the container with id 49ffb489af41e135d50471a1b56a64be415248bf54a8e52d02d5824f2063c445 Apr 22 21:17:45.561308 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:45.561232 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8zprt"] Apr 22 21:17:45.566357 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:45.566337 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-8zprt" Apr 22 21:17:45.570736 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:45.570713 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8zprt"] Apr 22 21:17:45.675049 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:45.675018 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qzlb\" (UniqueName: \"kubernetes.io/projected/1aa3b203-7b1e-42c4-8856-c8e6a43e56c3-kube-api-access-8qzlb\") pod \"kuadrant-operator-catalog-8zprt\" (UID: \"1aa3b203-7b1e-42c4-8856-c8e6a43e56c3\") " pod="kuadrant-system/kuadrant-operator-catalog-8zprt" Apr 22 21:17:45.775929 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:45.775900 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8qzlb\" (UniqueName: \"kubernetes.io/projected/1aa3b203-7b1e-42c4-8856-c8e6a43e56c3-kube-api-access-8qzlb\") pod \"kuadrant-operator-catalog-8zprt\" (UID: \"1aa3b203-7b1e-42c4-8856-c8e6a43e56c3\") " pod="kuadrant-system/kuadrant-operator-catalog-8zprt" Apr 22 21:17:45.785032 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:45.785006 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qzlb\" (UniqueName: \"kubernetes.io/projected/1aa3b203-7b1e-42c4-8856-c8e6a43e56c3-kube-api-access-8qzlb\") pod \"kuadrant-operator-catalog-8zprt\" (UID: \"1aa3b203-7b1e-42c4-8856-c8e6a43e56c3\") " pod="kuadrant-system/kuadrant-operator-catalog-8zprt" Apr 22 21:17:45.877405 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:45.877332 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-8zprt" Apr 22 21:17:45.997462 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:45.997440 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8zprt"] Apr 22 21:17:45.999826 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:17:45.999800 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aa3b203_7b1e_42c4_8856_c8e6a43e56c3.slice/crio-9951412f36955e92f427469ae7e3aabc36968651e68e7555a3d78e6c01ce12d1 WatchSource:0}: Error finding container 9951412f36955e92f427469ae7e3aabc36968651e68e7555a3d78e6c01ce12d1: Status 404 returned error can't find the container with id 9951412f36955e92f427469ae7e3aabc36968651e68e7555a3d78e6c01ce12d1 Apr 22 21:17:46.412839 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:46.412805 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-8zprt" event={"ID":"1aa3b203-7b1e-42c4-8856-c8e6a43e56c3","Type":"ContainerStarted","Data":"9951412f36955e92f427469ae7e3aabc36968651e68e7555a3d78e6c01ce12d1"} Apr 22 21:17:46.413915 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:46.413887 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sw5vj" event={"ID":"7bb5b705-2b88-495d-aaac-f0efd4c432c3","Type":"ContainerStarted","Data":"49ffb489af41e135d50471a1b56a64be415248bf54a8e52d02d5824f2063c445"} Apr 22 21:17:48.428071 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:48.428034 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sw5vj" event={"ID":"7bb5b705-2b88-495d-aaac-f0efd4c432c3","Type":"ContainerStarted","Data":"7f47c0fe45cc988c4bda2b74c5e593dfa624ae2e07b5d32c2852f690d74e0734"} Apr 22 21:17:48.428526 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:48.428063 2580 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-sw5vj" podUID="7bb5b705-2b88-495d-aaac-f0efd4c432c3" containerName="registry-server" containerID="cri-o://7f47c0fe45cc988c4bda2b74c5e593dfa624ae2e07b5d32c2852f690d74e0734" gracePeriod=2 Apr 22 21:17:48.429538 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:48.429515 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-8zprt" event={"ID":"1aa3b203-7b1e-42c4-8856-c8e6a43e56c3","Type":"ContainerStarted","Data":"290ef14290a5657a97a9b6103409a715b6003b34d6c8b39c11e4790541288210"} Apr 22 21:17:48.443041 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:48.443004 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-sw5vj" podStartSLOduration=2.381088393 podStartE2EDuration="4.442992331s" podCreationTimestamp="2026-04-22 21:17:44 +0000 UTC" firstStartedPulling="2026-04-22 21:17:45.421468978 +0000 UTC m=+506.147365556" lastFinishedPulling="2026-04-22 21:17:47.483372915 +0000 UTC m=+508.209269494" observedRunningTime="2026-04-22 21:17:48.441618955 +0000 UTC m=+509.167515555" watchObservedRunningTime="2026-04-22 21:17:48.442992331 +0000 UTC m=+509.168888931" Apr 22 21:17:48.456259 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:48.456222 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-8zprt" podStartSLOduration=1.973087576 podStartE2EDuration="3.456210936s" podCreationTimestamp="2026-04-22 21:17:45 +0000 UTC" firstStartedPulling="2026-04-22 21:17:46.001229025 +0000 UTC m=+506.727125603" lastFinishedPulling="2026-04-22 21:17:47.484352386 +0000 UTC m=+508.210248963" observedRunningTime="2026-04-22 21:17:48.454420614 +0000 UTC m=+509.180317224" watchObservedRunningTime="2026-04-22 21:17:48.456210936 +0000 UTC m=+509.182107535" Apr 22 21:17:48.671214 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:48.671185 2580 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sw5vj" Apr 22 21:17:48.806310 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:48.806233 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzct7\" (UniqueName: \"kubernetes.io/projected/7bb5b705-2b88-495d-aaac-f0efd4c432c3-kube-api-access-pzct7\") pod \"7bb5b705-2b88-495d-aaac-f0efd4c432c3\" (UID: \"7bb5b705-2b88-495d-aaac-f0efd4c432c3\") " Apr 22 21:17:48.808533 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:48.808504 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bb5b705-2b88-495d-aaac-f0efd4c432c3-kube-api-access-pzct7" (OuterVolumeSpecName: "kube-api-access-pzct7") pod "7bb5b705-2b88-495d-aaac-f0efd4c432c3" (UID: "7bb5b705-2b88-495d-aaac-f0efd4c432c3"). InnerVolumeSpecName "kube-api-access-pzct7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 22 21:17:48.907568 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:48.907535 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzct7\" (UniqueName: \"kubernetes.io/projected/7bb5b705-2b88-495d-aaac-f0efd4c432c3-kube-api-access-pzct7\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:17:49.433581 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:49.433551 2580 generic.go:358] "Generic (PLEG): container finished" podID="7bb5b705-2b88-495d-aaac-f0efd4c432c3" containerID="7f47c0fe45cc988c4bda2b74c5e593dfa624ae2e07b5d32c2852f690d74e0734" exitCode=0 Apr 22 21:17:49.434003 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:49.433602 2580 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sw5vj" Apr 22 21:17:49.434003 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:49.433611 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sw5vj" event={"ID":"7bb5b705-2b88-495d-aaac-f0efd4c432c3","Type":"ContainerDied","Data":"7f47c0fe45cc988c4bda2b74c5e593dfa624ae2e07b5d32c2852f690d74e0734"} Apr 22 21:17:49.434003 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:49.433648 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sw5vj" event={"ID":"7bb5b705-2b88-495d-aaac-f0efd4c432c3","Type":"ContainerDied","Data":"49ffb489af41e135d50471a1b56a64be415248bf54a8e52d02d5824f2063c445"} Apr 22 21:17:49.434003 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:49.433663 2580 scope.go:117] "RemoveContainer" containerID="7f47c0fe45cc988c4bda2b74c5e593dfa624ae2e07b5d32c2852f690d74e0734" Apr 22 21:17:49.442820 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:49.442805 2580 scope.go:117] "RemoveContainer" containerID="7f47c0fe45cc988c4bda2b74c5e593dfa624ae2e07b5d32c2852f690d74e0734" Apr 22 21:17:49.443107 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:17:49.443090 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f47c0fe45cc988c4bda2b74c5e593dfa624ae2e07b5d32c2852f690d74e0734\": container with ID starting with 7f47c0fe45cc988c4bda2b74c5e593dfa624ae2e07b5d32c2852f690d74e0734 not found: ID does not exist" containerID="7f47c0fe45cc988c4bda2b74c5e593dfa624ae2e07b5d32c2852f690d74e0734" Apr 22 21:17:49.443171 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:49.443120 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f47c0fe45cc988c4bda2b74c5e593dfa624ae2e07b5d32c2852f690d74e0734"} err="failed to get container status \"7f47c0fe45cc988c4bda2b74c5e593dfa624ae2e07b5d32c2852f690d74e0734\": rpc error: code = NotFound desc = could not find container \"7f47c0fe45cc988c4bda2b74c5e593dfa624ae2e07b5d32c2852f690d74e0734\": container with ID starting with 7f47c0fe45cc988c4bda2b74c5e593dfa624ae2e07b5d32c2852f690d74e0734 not found: ID does not exist" Apr 22 21:17:49.453244 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:49.453223 2580 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sw5vj"] Apr 22 21:17:49.457095 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:49.457075 2580 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sw5vj"] Apr 22 21:17:49.862458 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:49.862390 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bb5b705-2b88-495d-aaac-f0efd4c432c3" path="/var/lib/kubelet/pods/7bb5b705-2b88-495d-aaac-f0efd4c432c3/volumes" Apr 22 21:17:55.877532 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:55.877500 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-8zprt" Apr 22 21:17:55.877940 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:55.877542 2580 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-8zprt" Apr 22 21:17:55.898954 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:55.898924 2580 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-8zprt" Apr 22 21:17:56.477092 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:17:56.477063 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-8zprt" Apr 22 21:18:12.777059 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:12.777029 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m"] Apr 22 21:18:12.777527 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:12.777315 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bb5b705-2b88-495d-aaac-f0efd4c432c3" containerName="registry-server" Apr 22 21:18:12.777527 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:12.777325 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bb5b705-2b88-495d-aaac-f0efd4c432c3" containerName="registry-server" Apr 22 21:18:12.777527 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:12.777386 2580 memory_manager.go:356] "RemoveStaleState removing state" podUID="7bb5b705-2b88-495d-aaac-f0efd4c432c3" containerName="registry-server" Apr 22 21:18:12.780376 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:12.780360 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m" Apr 22 21:18:12.783196 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:12.783174 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-42bnf\"" Apr 22 21:18:12.791332 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:12.791309 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m"] Apr 22 21:18:12.803592 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:12.803572 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcx7n\" (UniqueName: \"kubernetes.io/projected/3dc16655-346d-4736-bc20-d788784a0b01-kube-api-access-tcx7n\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-ddk8m\" (UID: \"3dc16655-346d-4736-bc20-d788784a0b01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m" Apr 22 21:18:12.803678 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:12.803616 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3dc16655-346d-4736-bc20-d788784a0b01-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-ddk8m\" (UID: \"3dc16655-346d-4736-bc20-d788784a0b01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m" Apr 22 21:18:12.904627 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:12.904592 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tcx7n\" (UniqueName: \"kubernetes.io/projected/3dc16655-346d-4736-bc20-d788784a0b01-kube-api-access-tcx7n\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-ddk8m\" (UID: \"3dc16655-346d-4736-bc20-d788784a0b01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m" Apr 22 21:18:12.904817 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:12.904666 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3dc16655-346d-4736-bc20-d788784a0b01-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-ddk8m\" (UID: \"3dc16655-346d-4736-bc20-d788784a0b01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m" Apr 22 21:18:12.905149 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:12.905125 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3dc16655-346d-4736-bc20-d788784a0b01-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-ddk8m\" (UID: \"3dc16655-346d-4736-bc20-d788784a0b01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m" Apr 22 21:18:12.914833 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:12.914809 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcx7n\" (UniqueName: \"kubernetes.io/projected/3dc16655-346d-4736-bc20-d788784a0b01-kube-api-access-tcx7n\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-ddk8m\" (UID: \"3dc16655-346d-4736-bc20-d788784a0b01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m" Apr 22 21:18:13.090995 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:13.090898 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m" Apr 22 21:18:13.218253 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:13.218154 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m"] Apr 22 21:18:13.221069 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:18:13.221042 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dc16655_346d_4736_bc20_d788784a0b01.slice/crio-c61b65635afb7a49c34a1212d999167d6a04a5670fb8cbbd1ccd1431393d1b52 WatchSource:0}: Error finding container c61b65635afb7a49c34a1212d999167d6a04a5670fb8cbbd1ccd1431393d1b52: Status 404 returned error can't find the container with id c61b65635afb7a49c34a1212d999167d6a04a5670fb8cbbd1ccd1431393d1b52 Apr 22 21:18:13.508671 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:13.508618 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m" event={"ID":"3dc16655-346d-4736-bc20-d788784a0b01","Type":"ContainerStarted","Data":"c61b65635afb7a49c34a1212d999167d6a04a5670fb8cbbd1ccd1431393d1b52"} Apr 22 21:18:15.852311 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:15.852277 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-sbwf4"] Apr 22 21:18:15.855474 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:15.855458 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-sbwf4" Apr 22 21:18:15.858161 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:15.858140 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Apr 22 21:18:15.858438 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:15.858424 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-2qxmw\"" Apr 22 21:18:15.864298 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:15.864276 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-sbwf4"] Apr 22 21:18:15.931113 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:15.931081 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdzkq\" (UniqueName: \"kubernetes.io/projected/52d77edf-b5ee-4290-a5ed-e7a91ad20470-kube-api-access-xdzkq\") pod \"dns-operator-controller-manager-648d5c98bc-sbwf4\" (UID: \"52d77edf-b5ee-4290-a5ed-e7a91ad20470\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-sbwf4" Apr 22 21:18:16.031890 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:16.031841 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xdzkq\" (UniqueName: \"kubernetes.io/projected/52d77edf-b5ee-4290-a5ed-e7a91ad20470-kube-api-access-xdzkq\") pod \"dns-operator-controller-manager-648d5c98bc-sbwf4\" (UID: \"52d77edf-b5ee-4290-a5ed-e7a91ad20470\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-sbwf4" Apr 22 21:18:16.044987 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:16.044954 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdzkq\" (UniqueName: \"kubernetes.io/projected/52d77edf-b5ee-4290-a5ed-e7a91ad20470-kube-api-access-xdzkq\") pod \"dns-operator-controller-manager-648d5c98bc-sbwf4\" (UID: \"52d77edf-b5ee-4290-a5ed-e7a91ad20470\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-sbwf4" Apr 22 21:18:16.167322 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:16.167289 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-sbwf4" Apr 22 21:18:16.316835 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:16.316806 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-sbwf4"] Apr 22 21:18:16.320119 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:18:16.320089 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52d77edf_b5ee_4290_a5ed_e7a91ad20470.slice/crio-c7484e6efc51651825efc502596daa6b6618d008c4d37f56eb4d67ba547cf0c5 WatchSource:0}: Error finding container c7484e6efc51651825efc502596daa6b6618d008c4d37f56eb4d67ba547cf0c5: Status 404 returned error can't find the container with id c7484e6efc51651825efc502596daa6b6618d008c4d37f56eb4d67ba547cf0c5 Apr 22 21:18:16.519795 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:16.519707 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-sbwf4" event={"ID":"52d77edf-b5ee-4290-a5ed-e7a91ad20470","Type":"ContainerStarted","Data":"c7484e6efc51651825efc502596daa6b6618d008c4d37f56eb4d67ba547cf0c5"} Apr 22 21:18:18.529978 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:18.529941 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m" event={"ID":"3dc16655-346d-4736-bc20-d788784a0b01","Type":"ContainerStarted","Data":"d8891c21e18cff32ea8049bf8c7f080cc494ecc5a7ef9d656276dc3ea7055016"} Apr 22 21:18:18.530472 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:18.530079 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m" Apr 22 21:18:18.549467 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:18.549340 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m" podStartSLOduration=1.707935466 podStartE2EDuration="6.549322012s" podCreationTimestamp="2026-04-22 21:18:12 +0000 UTC" firstStartedPulling="2026-04-22 21:18:13.223407293 +0000 UTC m=+533.949303886" lastFinishedPulling="2026-04-22 21:18:18.064793854 +0000 UTC m=+538.790690432" observedRunningTime="2026-04-22 21:18:18.547587304 +0000 UTC m=+539.273483904" watchObservedRunningTime="2026-04-22 21:18:18.549322012 +0000 UTC m=+539.275218613" Apr 22 21:18:20.248790 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:20.248772 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Apr 22 21:18:20.537578 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:20.537484 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-sbwf4" event={"ID":"52d77edf-b5ee-4290-a5ed-e7a91ad20470","Type":"ContainerStarted","Data":"2623b28d10a9108fca9dc158d91524a261904197274545f2d1c112c4ba70cc17"} Apr 22 21:18:20.537578 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:20.537544 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-sbwf4" Apr 22 21:18:20.554390 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:20.554341 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-sbwf4" podStartSLOduration=1.630832196 podStartE2EDuration="5.554327651s" podCreationTimestamp="2026-04-22 21:18:15 +0000 UTC" firstStartedPulling="2026-04-22 21:18:16.322700124 +0000 UTC m=+537.048596702" lastFinishedPulling="2026-04-22 21:18:20.246195576 +0000 UTC m=+540.972092157" observedRunningTime="2026-04-22 21:18:20.552737958 +0000 UTC m=+541.278634559" watchObservedRunningTime="2026-04-22 21:18:20.554327651 +0000 UTC m=+541.280224250" Apr 22 21:18:21.211422 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:21.211383 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-g2njn"] Apr 22 21:18:21.219901 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:21.219875 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-g2njn" Apr 22 21:18:21.222400 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:21.222376 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-g2njn"] Apr 22 21:18:21.222954 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:21.222919 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5sldk\"" Apr 22 21:18:21.222954 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:21.222944 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Apr 22 21:18:21.223117 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:21.223028 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Apr 22 21:18:21.278006 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:21.277983 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vb56\" (UniqueName: \"kubernetes.io/projected/b1b12a7a-558f-4cb8-8856-0469f16c5555-kube-api-access-7vb56\") pod \"kuadrant-console-plugin-6cb54b5c86-g2njn\" (UID: \"b1b12a7a-558f-4cb8-8856-0469f16c5555\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-g2njn" Apr 22 21:18:21.278326 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:21.278019 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b1b12a7a-558f-4cb8-8856-0469f16c5555-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-g2njn\" (UID: \"b1b12a7a-558f-4cb8-8856-0469f16c5555\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-g2njn" Apr 22 21:18:21.278326 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:21.278045 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1b12a7a-558f-4cb8-8856-0469f16c5555-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-g2njn\" (UID: \"b1b12a7a-558f-4cb8-8856-0469f16c5555\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-g2njn" Apr 22 21:18:21.379363 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:21.379336 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7vb56\" (UniqueName: \"kubernetes.io/projected/b1b12a7a-558f-4cb8-8856-0469f16c5555-kube-api-access-7vb56\") pod \"kuadrant-console-plugin-6cb54b5c86-g2njn\" (UID: \"b1b12a7a-558f-4cb8-8856-0469f16c5555\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-g2njn" Apr 22 21:18:21.379363 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:21.379374 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b1b12a7a-558f-4cb8-8856-0469f16c5555-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-g2njn\" (UID: \"b1b12a7a-558f-4cb8-8856-0469f16c5555\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-g2njn" Apr 22 21:18:21.379650 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:21.379410 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1b12a7a-558f-4cb8-8856-0469f16c5555-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-g2njn\" (UID: \"b1b12a7a-558f-4cb8-8856-0469f16c5555\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-g2njn" Apr 22 21:18:21.380673 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:18:21.379927 2580 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Apr 22 21:18:21.380673 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:18:21.380022 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1b12a7a-558f-4cb8-8856-0469f16c5555-plugin-serving-cert podName:b1b12a7a-558f-4cb8-8856-0469f16c5555 nodeName:}" failed. No retries permitted until 2026-04-22 21:18:21.879989378 +0000 UTC m=+542.605885970 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/b1b12a7a-558f-4cb8-8856-0469f16c5555-plugin-serving-cert") pod "kuadrant-console-plugin-6cb54b5c86-g2njn" (UID: "b1b12a7a-558f-4cb8-8856-0469f16c5555") : secret "plugin-serving-cert" not found Apr 22 21:18:21.380673 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:21.380532 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b1b12a7a-558f-4cb8-8856-0469f16c5555-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-g2njn\" (UID: \"b1b12a7a-558f-4cb8-8856-0469f16c5555\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-g2njn" Apr 22 21:18:21.389140 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:21.389117 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vb56\" (UniqueName: \"kubernetes.io/projected/b1b12a7a-558f-4cb8-8856-0469f16c5555-kube-api-access-7vb56\") pod \"kuadrant-console-plugin-6cb54b5c86-g2njn\" (UID: \"b1b12a7a-558f-4cb8-8856-0469f16c5555\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-g2njn" Apr 22 21:18:21.884265 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:21.884234 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1b12a7a-558f-4cb8-8856-0469f16c5555-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-g2njn\" (UID: \"b1b12a7a-558f-4cb8-8856-0469f16c5555\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-g2njn" Apr 22 21:18:21.886695 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:21.886672 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1b12a7a-558f-4cb8-8856-0469f16c5555-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-g2njn\" (UID: \"b1b12a7a-558f-4cb8-8856-0469f16c5555\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-g2njn" Apr 22 21:18:22.145899 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:22.145777 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-g2njn" Apr 22 21:18:22.212022 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:22.211994 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-49mrf"] Apr 22 21:18:22.217464 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:22.217448 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-49mrf" Apr 22 21:18:22.221954 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:22.221929 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-z7sgz\"" Apr 22 21:18:22.222378 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:22.222360 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-49mrf"] Apr 22 21:18:22.269370 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:22.269309 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-g2njn"] Apr 22 21:18:22.271939 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:18:22.271910 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb1b12a7a_558f_4cb8_8856_0469f16c5555.slice/crio-2e1135dce5b81c7ae3aad47fb9f36f36a4eb8dc702f9584efeeefe2156831ab3 WatchSource:0}: Error finding container 2e1135dce5b81c7ae3aad47fb9f36f36a4eb8dc702f9584efeeefe2156831ab3: Status 404 returned error can't find the container with id 2e1135dce5b81c7ae3aad47fb9f36f36a4eb8dc702f9584efeeefe2156831ab3 Apr 22 21:18:22.288488 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:22.288464 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxqvk\" (UniqueName: \"kubernetes.io/projected/b4b91b3c-dcb7-4860-a395-11e9b651c24d-kube-api-access-pxqvk\") pod \"limitador-operator-controller-manager-85c4996f8c-49mrf\" (UID: \"b4b91b3c-dcb7-4860-a395-11e9b651c24d\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-49mrf" Apr 22 21:18:22.389304 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:22.389274 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pxqvk\" (UniqueName: \"kubernetes.io/projected/b4b91b3c-dcb7-4860-a395-11e9b651c24d-kube-api-access-pxqvk\") pod \"limitador-operator-controller-manager-85c4996f8c-49mrf\" (UID: \"b4b91b3c-dcb7-4860-a395-11e9b651c24d\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-49mrf" Apr 22 21:18:22.400390 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:22.400332 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxqvk\" (UniqueName: \"kubernetes.io/projected/b4b91b3c-dcb7-4860-a395-11e9b651c24d-kube-api-access-pxqvk\") pod \"limitador-operator-controller-manager-85c4996f8c-49mrf\" (UID: \"b4b91b3c-dcb7-4860-a395-11e9b651c24d\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-49mrf" Apr 22 21:18:22.531191 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:22.531156 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-49mrf" Apr 22 21:18:22.546039 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:22.546011 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-g2njn" event={"ID":"b1b12a7a-558f-4cb8-8856-0469f16c5555","Type":"ContainerStarted","Data":"2e1135dce5b81c7ae3aad47fb9f36f36a4eb8dc702f9584efeeefe2156831ab3"} Apr 22 21:18:22.652326 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:22.652271 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-49mrf"] Apr 22 21:18:22.654362 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:18:22.654329 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4b91b3c_dcb7_4860_a395_11e9b651c24d.slice/crio-5068d499f6fff5ce5b54dc72305a5f24b5a813fbecef2069a3a5c1410c45a9ed WatchSource:0}: Error finding container 5068d499f6fff5ce5b54dc72305a5f24b5a813fbecef2069a3a5c1410c45a9ed: Status 404 returned error can't find the container with id 5068d499f6fff5ce5b54dc72305a5f24b5a813fbecef2069a3a5c1410c45a9ed Apr 22 21:18:23.553309 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:23.553259 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-49mrf" event={"ID":"b4b91b3c-dcb7-4860-a395-11e9b651c24d","Type":"ContainerStarted","Data":"5068d499f6fff5ce5b54dc72305a5f24b5a813fbecef2069a3a5c1410c45a9ed"} Apr 22 21:18:25.562511 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:25.562474 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-49mrf" event={"ID":"b4b91b3c-dcb7-4860-a395-11e9b651c24d","Type":"ContainerStarted","Data":"6c4ea614304d041943812b9ad7ddb0a64479e367e43c3bf4503095aedcd6b30e"} Apr 22 21:18:25.563008 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:25.562641 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-49mrf" Apr 22 21:18:25.579373 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:25.579328 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-49mrf" podStartSLOduration=1.737920226 podStartE2EDuration="3.579315123s" podCreationTimestamp="2026-04-22 21:18:22 +0000 UTC" firstStartedPulling="2026-04-22 21:18:22.656264792 +0000 UTC m=+543.382161370" lastFinishedPulling="2026-04-22 21:18:24.497659676 +0000 UTC m=+545.223556267" observedRunningTime="2026-04-22 21:18:25.578679603 +0000 UTC m=+546.304576202" watchObservedRunningTime="2026-04-22 21:18:25.579315123 +0000 UTC m=+546.305211723" Apr 22 21:18:29.535139 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:29.535109 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m" Apr 22 21:18:31.076740 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.076680 2580 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m"] Apr 22 21:18:31.077218 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.077105 2580 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m" podUID="3dc16655-346d-4736-bc20-d788784a0b01" containerName="manager" containerID="cri-o://d8891c21e18cff32ea8049bf8c7f080cc494ecc5a7ef9d656276dc3ea7055016" gracePeriod=2 Apr 22 21:18:31.079071 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.079022 2580 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m"] Apr 22 21:18:31.079376 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.079343 2580 status_manager.go:895] "Failed to get status for pod" podUID="3dc16655-346d-4736-bc20-d788784a0b01" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-ddk8m\" is forbidden: User \"system:node:ip-10-0-136-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-20.ec2.internal' and this object" Apr 22 21:18:31.090107 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.090059 2580 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-49mrf"] Apr 22 21:18:31.090343 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.090317 2580 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-49mrf" podUID="b4b91b3c-dcb7-4860-a395-11e9b651c24d" containerName="manager" containerID="cri-o://6c4ea614304d041943812b9ad7ddb0a64479e367e43c3bf4503095aedcd6b30e" gracePeriod=2 Apr 22 21:18:31.092657 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.092540 2580 status_manager.go:895] "Failed to get status for pod" podUID="3dc16655-346d-4736-bc20-d788784a0b01" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-ddk8m\" is forbidden: User \"system:node:ip-10-0-136-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-20.ec2.internal' and this object" Apr 22 21:18:31.092657 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.092585 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-49mrf" Apr 22 21:18:31.097115 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.097078 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-t78zl"] Apr 22 21:18:31.098301 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.097590 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dc16655-346d-4736-bc20-d788784a0b01" containerName="manager" Apr 22 21:18:31.098301 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.097610 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dc16655-346d-4736-bc20-d788784a0b01" containerName="manager" Apr 22 21:18:31.098301 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.097709 2580 memory_manager.go:356] "RemoveStaleState removing state" podUID="3dc16655-346d-4736-bc20-d788784a0b01" containerName="manager" Apr 22 21:18:31.101259 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.100806 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-t78zl" Apr 22 21:18:31.101259 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.101073 2580 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-49mrf"] Apr 22 21:18:31.103921 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.103870 2580 status_manager.go:895] "Failed to get status for pod" podUID="b4b91b3c-dcb7-4860-a395-11e9b651c24d" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-49mrf" err="pods \"limitador-operator-controller-manager-85c4996f8c-49mrf\" is forbidden: User \"system:node:ip-10-0-136-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-20.ec2.internal' and this object" Apr 22 21:18:31.114167 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.114145 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-krpg7"] Apr 22 21:18:31.114514 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.114493 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4b91b3c-dcb7-4860-a395-11e9b651c24d" containerName="manager" Apr 22 21:18:31.114514 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.114510 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4b91b3c-dcb7-4860-a395-11e9b651c24d" containerName="manager" Apr 22 21:18:31.114676 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.114579 2580 memory_manager.go:356] "RemoveStaleState removing state" podUID="b4b91b3c-dcb7-4860-a395-11e9b651c24d" containerName="manager" Apr 22 21:18:31.119686 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.118564 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-t78zl"] Apr 22 21:18:31.119686 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.118699 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-krpg7" Apr 22 21:18:31.127026 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.127009 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-krpg7"] Apr 22 21:18:31.130904 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.130836 2580 status_manager.go:895] "Failed to get status for pod" podUID="3dc16655-346d-4736-bc20-d788784a0b01" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-ddk8m\" is forbidden: User \"system:node:ip-10-0-136-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-20.ec2.internal' and this object" Apr 22 21:18:31.154791 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.154757 2580 status_manager.go:895] "Failed to get status for pod" podUID="3dc16655-346d-4736-bc20-d788784a0b01" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-ddk8m\" is forbidden: User \"system:node:ip-10-0-136-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-20.ec2.internal' and this object" Apr 22 21:18:31.156912 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.156883 2580 status_manager.go:895] "Failed to get status for pod" podUID="b4b91b3c-dcb7-4860-a395-11e9b651c24d" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-49mrf" err="pods \"limitador-operator-controller-manager-85c4996f8c-49mrf\" is forbidden: User \"system:node:ip-10-0-136-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-20.ec2.internal' and this object" Apr 22 21:18:31.169150 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.169124 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdbqv\" (UniqueName: \"kubernetes.io/projected/a7570b12-6fd8-4228-9e12-9d9662a671a8-kube-api-access-wdbqv\") pod \"limitador-operator-controller-manager-85c4996f8c-krpg7\" (UID: \"a7570b12-6fd8-4228-9e12-9d9662a671a8\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-krpg7" Apr 22 21:18:31.169261 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.169189 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4052a2dc-1ea9-412b-be5f-28ba6a0606fc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-t78zl\" (UID: \"4052a2dc-1ea9-412b-be5f-28ba6a0606fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-t78zl" Apr 22 21:18:31.169322 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.169284 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5rg7\" (UniqueName: \"kubernetes.io/projected/4052a2dc-1ea9-412b-be5f-28ba6a0606fc-kube-api-access-x5rg7\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-t78zl\" (UID: \"4052a2dc-1ea9-412b-be5f-28ba6a0606fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-t78zl" Apr 22 21:18:31.270338 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.270310 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4052a2dc-1ea9-412b-be5f-28ba6a0606fc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-t78zl\" (UID: \"4052a2dc-1ea9-412b-be5f-28ba6a0606fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-t78zl" Apr 22 21:18:31.270461 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.270391 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x5rg7\" (UniqueName: \"kubernetes.io/projected/4052a2dc-1ea9-412b-be5f-28ba6a0606fc-kube-api-access-x5rg7\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-t78zl\" (UID: \"4052a2dc-1ea9-412b-be5f-28ba6a0606fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-t78zl" Apr 22 21:18:31.270521 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.270463 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wdbqv\" (UniqueName: \"kubernetes.io/projected/a7570b12-6fd8-4228-9e12-9d9662a671a8-kube-api-access-wdbqv\") pod \"limitador-operator-controller-manager-85c4996f8c-krpg7\" (UID: \"a7570b12-6fd8-4228-9e12-9d9662a671a8\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-krpg7" Apr 22 21:18:31.271017 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.270969 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4052a2dc-1ea9-412b-be5f-28ba6a0606fc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-t78zl\" (UID: \"4052a2dc-1ea9-412b-be5f-28ba6a0606fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-t78zl" Apr 22 21:18:31.280460 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.280407 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdbqv\" (UniqueName: \"kubernetes.io/projected/a7570b12-6fd8-4228-9e12-9d9662a671a8-kube-api-access-wdbqv\") pod \"limitador-operator-controller-manager-85c4996f8c-krpg7\" (UID: \"a7570b12-6fd8-4228-9e12-9d9662a671a8\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-krpg7" Apr 22 21:18:31.281024 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.281000 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5rg7\" (UniqueName: \"kubernetes.io/projected/4052a2dc-1ea9-412b-be5f-28ba6a0606fc-kube-api-access-x5rg7\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-t78zl\" (UID: \"4052a2dc-1ea9-412b-be5f-28ba6a0606fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-t78zl" Apr 22 21:18:31.327035 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.326983 2580 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m" Apr 22 21:18:31.329586 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.329559 2580 status_manager.go:895] "Failed to get status for pod" podUID="3dc16655-346d-4736-bc20-d788784a0b01" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-ddk8m\" is forbidden: User \"system:node:ip-10-0-136-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-20.ec2.internal' and this object" Apr 22 21:18:31.331640 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.331612 2580 status_manager.go:895] "Failed to get status for pod" podUID="b4b91b3c-dcb7-4860-a395-11e9b651c24d" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-49mrf" err="pods \"limitador-operator-controller-manager-85c4996f8c-49mrf\" is forbidden: User \"system:node:ip-10-0-136-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-20.ec2.internal' and this object" Apr 22 21:18:31.334228 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.334213 2580 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-49mrf" Apr 22 21:18:31.336643 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.336620 2580 status_manager.go:895] "Failed to get status for pod" podUID="3dc16655-346d-4736-bc20-d788784a0b01" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-ddk8m\" is forbidden: User \"system:node:ip-10-0-136-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-20.ec2.internal' and this object" Apr 22 21:18:31.339114 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.339087 2580 status_manager.go:895] "Failed to get status for pod" podUID="b4b91b3c-dcb7-4860-a395-11e9b651c24d" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-49mrf" err="pods \"limitador-operator-controller-manager-85c4996f8c-49mrf\" is forbidden: User \"system:node:ip-10-0-136-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-20.ec2.internal' and this object" Apr 22 21:18:31.371030 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.371004 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcx7n\" (UniqueName: \"kubernetes.io/projected/3dc16655-346d-4736-bc20-d788784a0b01-kube-api-access-tcx7n\") pod \"3dc16655-346d-4736-bc20-d788784a0b01\" (UID: \"3dc16655-346d-4736-bc20-d788784a0b01\") " Apr 22 21:18:31.371133 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.371108 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3dc16655-346d-4736-bc20-d788784a0b01-extensions-socket-volume\") pod \"3dc16655-346d-4736-bc20-d788784a0b01\" (UID: \"3dc16655-346d-4736-bc20-d788784a0b01\") " Apr 22 21:18:31.371498 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.371469 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dc16655-346d-4736-bc20-d788784a0b01-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3dc16655-346d-4736-bc20-d788784a0b01" (UID: "3dc16655-346d-4736-bc20-d788784a0b01"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 22 21:18:31.373514 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.373488 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dc16655-346d-4736-bc20-d788784a0b01-kube-api-access-tcx7n" (OuterVolumeSpecName: "kube-api-access-tcx7n") pod "3dc16655-346d-4736-bc20-d788784a0b01" (UID: "3dc16655-346d-4736-bc20-d788784a0b01"). InnerVolumeSpecName "kube-api-access-tcx7n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 22 21:18:31.472320 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.472291 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxqvk\" (UniqueName: \"kubernetes.io/projected/b4b91b3c-dcb7-4860-a395-11e9b651c24d-kube-api-access-pxqvk\") pod \"b4b91b3c-dcb7-4860-a395-11e9b651c24d\" (UID: \"b4b91b3c-dcb7-4860-a395-11e9b651c24d\") " Apr 22 21:18:31.472570 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.472552 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3dc16655-346d-4736-bc20-d788784a0b01-extensions-socket-volume\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:18:31.472620 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.472577 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcx7n\" (UniqueName: \"kubernetes.io/projected/3dc16655-346d-4736-bc20-d788784a0b01-kube-api-access-tcx7n\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:18:31.474697 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.474674 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4b91b3c-dcb7-4860-a395-11e9b651c24d-kube-api-access-pxqvk" (OuterVolumeSpecName: "kube-api-access-pxqvk") pod "b4b91b3c-dcb7-4860-a395-11e9b651c24d" (UID: "b4b91b3c-dcb7-4860-a395-11e9b651c24d"). InnerVolumeSpecName "kube-api-access-pxqvk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 22 21:18:31.518414 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.518387 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-t78zl" Apr 22 21:18:31.526281 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.526262 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-krpg7" Apr 22 21:18:31.545217 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.545196 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-sbwf4" Apr 22 21:18:31.572363 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.572266 2580 status_manager.go:895] "Failed to get status for pod" podUID="3dc16655-346d-4736-bc20-d788784a0b01" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-ddk8m\" is forbidden: User \"system:node:ip-10-0-136-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-20.ec2.internal' and this object" Apr 22 21:18:31.573766 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.573725 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxqvk\" (UniqueName: \"kubernetes.io/projected/b4b91b3c-dcb7-4860-a395-11e9b651c24d-kube-api-access-pxqvk\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:18:31.574701 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.574674 2580 status_manager.go:895] "Failed to get status for pod" podUID="b4b91b3c-dcb7-4860-a395-11e9b651c24d" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-49mrf" err="pods \"limitador-operator-controller-manager-85c4996f8c-49mrf\" is forbidden: User \"system:node:ip-10-0-136-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-20.ec2.internal' and this object" Apr 22 21:18:31.595303 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.595237 2580 generic.go:358] "Generic (PLEG): container finished" podID="3dc16655-346d-4736-bc20-d788784a0b01" containerID="d8891c21e18cff32ea8049bf8c7f080cc494ecc5a7ef9d656276dc3ea7055016" exitCode=0 Apr 22 21:18:31.595610 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.595488 2580 scope.go:117] "RemoveContainer" containerID="d8891c21e18cff32ea8049bf8c7f080cc494ecc5a7ef9d656276dc3ea7055016" Apr 22 21:18:31.598510 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.595733 2580 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m" Apr 22 21:18:31.600225 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.600191 2580 status_manager.go:895] "Failed to get status for pod" podUID="b4b91b3c-dcb7-4860-a395-11e9b651c24d" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-49mrf" err="pods \"limitador-operator-controller-manager-85c4996f8c-49mrf\" is forbidden: User \"system:node:ip-10-0-136-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-20.ec2.internal' and this object" Apr 22 21:18:31.602999 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.602963 2580 status_manager.go:895] "Failed to get status for pod" podUID="3dc16655-346d-4736-bc20-d788784a0b01" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-ddk8m\" is forbidden: User \"system:node:ip-10-0-136-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-20.ec2.internal' and this object" Apr 22 21:18:31.607074 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.603603 2580 generic.go:358] "Generic (PLEG): container finished" podID="b4b91b3c-dcb7-4860-a395-11e9b651c24d" containerID="6c4ea614304d041943812b9ad7ddb0a64479e367e43c3bf4503095aedcd6b30e" exitCode=0 Apr 22 21:18:31.607074 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.603743 2580 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-49mrf" Apr 22 21:18:31.609803 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.609688 2580 status_manager.go:895] "Failed to get status for pod" podUID="3dc16655-346d-4736-bc20-d788784a0b01" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-ddk8m\" is forbidden: User \"system:node:ip-10-0-136-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-20.ec2.internal' and this object" Apr 22 21:18:31.612718 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.612341 2580 status_manager.go:895] "Failed to get status for pod" podUID="b4b91b3c-dcb7-4860-a395-11e9b651c24d" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-49mrf" err="pods \"limitador-operator-controller-manager-85c4996f8c-49mrf\" is forbidden: User \"system:node:ip-10-0-136-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-20.ec2.internal' and this object" Apr 22 21:18:31.615737 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.615553 2580 status_manager.go:895] "Failed to get status for pod" podUID="3dc16655-346d-4736-bc20-d788784a0b01" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-ddk8m\" is forbidden: User \"system:node:ip-10-0-136-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-20.ec2.internal' and this object" Apr 22 21:18:31.618139 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.618106 2580 status_manager.go:895] "Failed to get status for pod" podUID="b4b91b3c-dcb7-4860-a395-11e9b651c24d" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-49mrf" err="pods \"limitador-operator-controller-manager-85c4996f8c-49mrf\" is forbidden: User \"system:node:ip-10-0-136-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-20.ec2.internal' and this object" Apr 22 21:18:31.623091 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.623027 2580 scope.go:117] "RemoveContainer" containerID="d8891c21e18cff32ea8049bf8c7f080cc494ecc5a7ef9d656276dc3ea7055016" Apr 22 21:18:31.623405 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:18:31.623366 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8891c21e18cff32ea8049bf8c7f080cc494ecc5a7ef9d656276dc3ea7055016\": container with ID starting with d8891c21e18cff32ea8049bf8c7f080cc494ecc5a7ef9d656276dc3ea7055016 not found: ID does not exist" containerID="d8891c21e18cff32ea8049bf8c7f080cc494ecc5a7ef9d656276dc3ea7055016" Apr 22 21:18:31.623496 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.623406 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8891c21e18cff32ea8049bf8c7f080cc494ecc5a7ef9d656276dc3ea7055016"} err="failed to get container status \"d8891c21e18cff32ea8049bf8c7f080cc494ecc5a7ef9d656276dc3ea7055016\": rpc error: code = NotFound desc = could not find container \"d8891c21e18cff32ea8049bf8c7f080cc494ecc5a7ef9d656276dc3ea7055016\": container with ID starting with d8891c21e18cff32ea8049bf8c7f080cc494ecc5a7ef9d656276dc3ea7055016 not found: ID does not exist" Apr 22 21:18:31.623496 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.623430 2580 scope.go:117] "RemoveContainer" containerID="6c4ea614304d041943812b9ad7ddb0a64479e367e43c3bf4503095aedcd6b30e" Apr 22 21:18:31.625819 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.625692 2580 status_manager.go:895] "Failed to get status for pod" podUID="3dc16655-346d-4736-bc20-d788784a0b01" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ddk8m" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-ddk8m\" is forbidden: User \"system:node:ip-10-0-136-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-20.ec2.internal' and this object" Apr 22 21:18:31.628328 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.628310 2580 status_manager.go:895] "Failed to get status for pod" podUID="b4b91b3c-dcb7-4860-a395-11e9b651c24d" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-49mrf" err="pods \"limitador-operator-controller-manager-85c4996f8c-49mrf\" is forbidden: User \"system:node:ip-10-0-136-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-20.ec2.internal' and this object" Apr 22 21:18:31.634989 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.634941 2580 scope.go:117] "RemoveContainer" containerID="6c4ea614304d041943812b9ad7ddb0a64479e367e43c3bf4503095aedcd6b30e" Apr 22 21:18:31.635500 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:18:31.635439 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c4ea614304d041943812b9ad7ddb0a64479e367e43c3bf4503095aedcd6b30e\": container with ID starting with 6c4ea614304d041943812b9ad7ddb0a64479e367e43c3bf4503095aedcd6b30e not found: ID does not exist" containerID="6c4ea614304d041943812b9ad7ddb0a64479e367e43c3bf4503095aedcd6b30e" Apr 22 21:18:31.635500 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.635475 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c4ea614304d041943812b9ad7ddb0a64479e367e43c3bf4503095aedcd6b30e"} err="failed to get container status \"6c4ea614304d041943812b9ad7ddb0a64479e367e43c3bf4503095aedcd6b30e\": rpc error: code = NotFound desc = could not find container \"6c4ea614304d041943812b9ad7ddb0a64479e367e43c3bf4503095aedcd6b30e\": container with ID starting with 6c4ea614304d041943812b9ad7ddb0a64479e367e43c3bf4503095aedcd6b30e not found: ID does not exist" Apr 22 21:18:31.691326 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.691302 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-t78zl"] Apr 22 21:18:31.693010 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:18:31.692985 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4052a2dc_1ea9_412b_be5f_28ba6a0606fc.slice/crio-076590a48dee73a0b793ed9dcf4e29ce5783be51e079d6ea77e60587671368b8 WatchSource:0}: Error finding container 076590a48dee73a0b793ed9dcf4e29ce5783be51e079d6ea77e60587671368b8: Status 404 returned error can't find the container with id 076590a48dee73a0b793ed9dcf4e29ce5783be51e079d6ea77e60587671368b8 Apr 22 21:18:31.705517 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.705496 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-krpg7"] Apr 22 21:18:31.707568 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:18:31.707544 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7570b12_6fd8_4228_9e12_9d9662a671a8.slice/crio-eb5a219b2d1f5c075547ab49ca6a9d651d60bcece38053d94e38d7efd12dff76 WatchSource:0}: Error finding container eb5a219b2d1f5c075547ab49ca6a9d651d60bcece38053d94e38d7efd12dff76: Status 404 returned error can't find the container with id eb5a219b2d1f5c075547ab49ca6a9d651d60bcece38053d94e38d7efd12dff76 Apr 22 21:18:31.863915 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.863819 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3dc16655-346d-4736-bc20-d788784a0b01" path="/var/lib/kubelet/pods/3dc16655-346d-4736-bc20-d788784a0b01/volumes" Apr 22 21:18:31.864290 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:31.864272 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4b91b3c-dcb7-4860-a395-11e9b651c24d" path="/var/lib/kubelet/pods/b4b91b3c-dcb7-4860-a395-11e9b651c24d/volumes" Apr 22 21:18:32.047517 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:32.047483 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-dzf7g"] Apr 22 21:18:32.051387 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:32.051360 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-dzf7g" Apr 22 21:18:32.061531 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:32.061506 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-dzf7g"] Apr 22 21:18:32.179781 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:32.179746 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d01cf883-823e-48de-80f8-4df566a38c28-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-dzf7g\" (UID: \"d01cf883-823e-48de-80f8-4df566a38c28\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-dzf7g" Apr 22 21:18:32.180273 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:32.179809 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcg8m\" (UniqueName: \"kubernetes.io/projected/d01cf883-823e-48de-80f8-4df566a38c28-kube-api-access-dcg8m\") pod \"kuadrant-operator-controller-manager-55c7f4c975-dzf7g\" (UID: \"d01cf883-823e-48de-80f8-4df566a38c28\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-dzf7g" Apr 22 21:18:32.280880 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:32.280804 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d01cf883-823e-48de-80f8-4df566a38c28-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-dzf7g\" (UID: \"d01cf883-823e-48de-80f8-4df566a38c28\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-dzf7g" Apr 22 21:18:32.281058 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:32.280897 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dcg8m\" (UniqueName: \"kubernetes.io/projected/d01cf883-823e-48de-80f8-4df566a38c28-kube-api-access-dcg8m\") pod \"kuadrant-operator-controller-manager-55c7f4c975-dzf7g\" (UID: \"d01cf883-823e-48de-80f8-4df566a38c28\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-dzf7g" Apr 22 21:18:32.281329 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:32.281301 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d01cf883-823e-48de-80f8-4df566a38c28-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-dzf7g\" (UID: \"d01cf883-823e-48de-80f8-4df566a38c28\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-dzf7g" Apr 22 21:18:32.290623 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:32.290592 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcg8m\" (UniqueName: \"kubernetes.io/projected/d01cf883-823e-48de-80f8-4df566a38c28-kube-api-access-dcg8m\") pod \"kuadrant-operator-controller-manager-55c7f4c975-dzf7g\" (UID: \"d01cf883-823e-48de-80f8-4df566a38c28\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-dzf7g" Apr 22 21:18:32.365106 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:32.365080 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-dzf7g" Apr 22 21:18:32.496004 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:32.495794 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-dzf7g"] Apr 22 21:18:32.498917 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:18:32.498888 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd01cf883_823e_48de_80f8_4df566a38c28.slice/crio-bb223610304c4187aeb0ea8c32407b58b82ecc736173ef444b15396cda646df8 WatchSource:0}: Error finding container bb223610304c4187aeb0ea8c32407b58b82ecc736173ef444b15396cda646df8: Status 404 returned error can't find the container with id bb223610304c4187aeb0ea8c32407b58b82ecc736173ef444b15396cda646df8 Apr 22 21:18:32.611307 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:32.611274 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-t78zl" event={"ID":"4052a2dc-1ea9-412b-be5f-28ba6a0606fc","Type":"ContainerStarted","Data":"fa7fbb4be56ea3fe39dbbd9fdc05f4f4e7acd5e94da3749843a338220846a4a6"} Apr 22 21:18:32.611307 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:32.611312 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-t78zl" event={"ID":"4052a2dc-1ea9-412b-be5f-28ba6a0606fc","Type":"ContainerStarted","Data":"076590a48dee73a0b793ed9dcf4e29ce5783be51e079d6ea77e60587671368b8"} Apr 22 21:18:32.611536 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:32.611362 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-t78zl" Apr 22 21:18:32.612929 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:32.612893 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-dzf7g" event={"ID":"d01cf883-823e-48de-80f8-4df566a38c28","Type":"ContainerStarted","Data":"67c06eec94d74a684a4d050c63a241fbf3e55bba8fd75bda84964825e2d4a1be"} Apr 22 21:18:32.612929 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:32.612920 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-dzf7g" event={"ID":"d01cf883-823e-48de-80f8-4df566a38c28","Type":"ContainerStarted","Data":"bb223610304c4187aeb0ea8c32407b58b82ecc736173ef444b15396cda646df8"} Apr 22 21:18:32.613118 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:32.613015 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-dzf7g" Apr 22 21:18:32.614609 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:32.614587 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-krpg7" event={"ID":"a7570b12-6fd8-4228-9e12-9d9662a671a8","Type":"ContainerStarted","Data":"906794bce88d44a61842cca2b859e3808b7432d12da4fc9630fbd1b175dd1a77"} Apr 22 21:18:32.614609 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:32.614616 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-krpg7" event={"ID":"a7570b12-6fd8-4228-9e12-9d9662a671a8","Type":"ContainerStarted","Data":"eb5a219b2d1f5c075547ab49ca6a9d651d60bcece38053d94e38d7efd12dff76"} Apr 22 21:18:32.614963 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:32.614943 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-krpg7" Apr 22 21:18:32.630056 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:32.630018 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-t78zl" podStartSLOduration=1.6300068859999999 podStartE2EDuration="1.630006886s" podCreationTimestamp="2026-04-22 21:18:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-22 21:18:32.627992522 +0000 UTC m=+553.353889122" watchObservedRunningTime="2026-04-22 21:18:32.630006886 +0000 UTC m=+553.355903463" Apr 22 21:18:32.644189 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:32.644157 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-dzf7g" podStartSLOduration=0.644145187 podStartE2EDuration="644.145187ms" podCreationTimestamp="2026-04-22 21:18:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-22 21:18:32.643157544 +0000 UTC m=+553.369054142" watchObservedRunningTime="2026-04-22 21:18:32.644145187 +0000 UTC m=+553.370041787" Apr 22 21:18:32.660683 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:32.660647 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-krpg7" podStartSLOduration=1.660638286 podStartE2EDuration="1.660638286s" podCreationTimestamp="2026-04-22 21:18:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-22 21:18:32.659284547 +0000 UTC m=+553.385181141" watchObservedRunningTime="2026-04-22 21:18:32.660638286 +0000 UTC m=+553.386534886" Apr 22 21:18:43.621617 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:43.621587 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-dzf7g" Apr 22 21:18:43.622084 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:43.621642 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-t78zl" Apr 22 21:18:43.700705 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:43.700670 2580 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-t78zl"] Apr 22 21:18:43.700985 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:43.700960 2580 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-t78zl" podUID="4052a2dc-1ea9-412b-be5f-28ba6a0606fc" containerName="manager" containerID="cri-o://fa7fbb4be56ea3fe39dbbd9fdc05f4f4e7acd5e94da3749843a338220846a4a6" gracePeriod=10 Apr 22 21:18:44.625812 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:44.625776 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-krpg7" Apr 22 21:18:47.681355 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:47.681328 2580 generic.go:358] "Generic (PLEG): container finished" podID="4052a2dc-1ea9-412b-be5f-28ba6a0606fc" containerID="fa7fbb4be56ea3fe39dbbd9fdc05f4f4e7acd5e94da3749843a338220846a4a6" exitCode=0 Apr 22 21:18:47.681675 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:47.681374 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-t78zl" event={"ID":"4052a2dc-1ea9-412b-be5f-28ba6a0606fc","Type":"ContainerDied","Data":"fa7fbb4be56ea3fe39dbbd9fdc05f4f4e7acd5e94da3749843a338220846a4a6"} Apr 22 21:18:47.750813 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:47.750789 2580 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-t78zl" Apr 22 21:18:47.823386 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:47.823358 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5rg7\" (UniqueName: \"kubernetes.io/projected/4052a2dc-1ea9-412b-be5f-28ba6a0606fc-kube-api-access-x5rg7\") pod \"4052a2dc-1ea9-412b-be5f-28ba6a0606fc\" (UID: \"4052a2dc-1ea9-412b-be5f-28ba6a0606fc\") " Apr 22 21:18:47.823503 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:47.823417 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4052a2dc-1ea9-412b-be5f-28ba6a0606fc-extensions-socket-volume\") pod \"4052a2dc-1ea9-412b-be5f-28ba6a0606fc\" (UID: \"4052a2dc-1ea9-412b-be5f-28ba6a0606fc\") " Apr 22 21:18:47.823768 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:47.823739 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4052a2dc-1ea9-412b-be5f-28ba6a0606fc-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4052a2dc-1ea9-412b-be5f-28ba6a0606fc" (UID: "4052a2dc-1ea9-412b-be5f-28ba6a0606fc"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 22 21:18:47.825461 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:47.825434 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4052a2dc-1ea9-412b-be5f-28ba6a0606fc-kube-api-access-x5rg7" (OuterVolumeSpecName: "kube-api-access-x5rg7") pod "4052a2dc-1ea9-412b-be5f-28ba6a0606fc" (UID: "4052a2dc-1ea9-412b-be5f-28ba6a0606fc"). InnerVolumeSpecName "kube-api-access-x5rg7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 22 21:18:47.925516 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:47.925465 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5rg7\" (UniqueName: \"kubernetes.io/projected/4052a2dc-1ea9-412b-be5f-28ba6a0606fc-kube-api-access-x5rg7\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:18:47.925516 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:47.925517 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4052a2dc-1ea9-412b-be5f-28ba6a0606fc-extensions-socket-volume\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:18:48.686672 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:48.686634 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-t78zl" event={"ID":"4052a2dc-1ea9-412b-be5f-28ba6a0606fc","Type":"ContainerDied","Data":"076590a48dee73a0b793ed9dcf4e29ce5783be51e079d6ea77e60587671368b8"} Apr 22 21:18:48.687102 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:48.686670 2580 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-t78zl" Apr 22 21:18:48.687102 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:48.686687 2580 scope.go:117] "RemoveContainer" containerID="fa7fbb4be56ea3fe39dbbd9fdc05f4f4e7acd5e94da3749843a338220846a4a6" Apr 22 21:18:48.688142 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:48.688116 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-g2njn" event={"ID":"b1b12a7a-558f-4cb8-8856-0469f16c5555","Type":"ContainerStarted","Data":"f01618f22201e562ce70697c29d94d016d5f395f582887179b33f8a6c161f8cc"} Apr 22 21:18:48.709303 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:48.709278 2580 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-t78zl"] Apr 22 21:18:48.712979 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:48.712958 2580 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-t78zl"] Apr 22 21:18:48.726116 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:48.726076 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-g2njn" podStartSLOduration=2.312610523 podStartE2EDuration="27.726065363s" podCreationTimestamp="2026-04-22 21:18:21 +0000 UTC" firstStartedPulling="2026-04-22 21:18:22.273492989 +0000 UTC m=+542.999389581" lastFinishedPulling="2026-04-22 21:18:47.686947829 +0000 UTC m=+568.412844421" observedRunningTime="2026-04-22 21:18:48.725173909 +0000 UTC m=+569.451070510" watchObservedRunningTime="2026-04-22 21:18:48.726065363 +0000 UTC m=+569.451961960" Apr 22 21:18:49.862966 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:18:49.862935 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4052a2dc-1ea9-412b-be5f-28ba6a0606fc" path="/var/lib/kubelet/pods/4052a2dc-1ea9-412b-be5f-28ba6a0606fc/volumes" Apr 22 21:19:04.705464 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:04.705429 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7498df8756-mf5p8"] Apr 22 21:19:04.705831 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:04.705750 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4052a2dc-1ea9-412b-be5f-28ba6a0606fc" containerName="manager" Apr 22 21:19:04.705831 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:04.705760 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4052a2dc-1ea9-412b-be5f-28ba6a0606fc" containerName="manager" Apr 22 21:19:04.705831 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:04.705815 2580 memory_manager.go:356] "RemoveStaleState removing state" podUID="4052a2dc-1ea9-412b-be5f-28ba6a0606fc" containerName="manager" Apr 22 21:19:04.723292 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:04.723261 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-mf5p8"] Apr 22 21:19:04.723431 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:04.723344 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-mf5p8" Apr 22 21:19:04.726031 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:04.726006 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-q9rz8\"" Apr 22 21:19:04.762691 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:04.762664 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wwvg\" (UniqueName: \"kubernetes.io/projected/f06f486f-0fab-4ce5-80f1-81792aaa53be-kube-api-access-7wwvg\") pod \"authorino-7498df8756-mf5p8\" (UID: \"f06f486f-0fab-4ce5-80f1-81792aaa53be\") " pod="kuadrant-system/authorino-7498df8756-mf5p8" Apr 22 21:19:04.863103 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:04.863077 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7wwvg\" (UniqueName: \"kubernetes.io/projected/f06f486f-0fab-4ce5-80f1-81792aaa53be-kube-api-access-7wwvg\") pod \"authorino-7498df8756-mf5p8\" (UID: \"f06f486f-0fab-4ce5-80f1-81792aaa53be\") " pod="kuadrant-system/authorino-7498df8756-mf5p8" Apr 22 21:19:04.873518 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:04.873497 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wwvg\" (UniqueName: \"kubernetes.io/projected/f06f486f-0fab-4ce5-80f1-81792aaa53be-kube-api-access-7wwvg\") pod \"authorino-7498df8756-mf5p8\" (UID: \"f06f486f-0fab-4ce5-80f1-81792aaa53be\") " pod="kuadrant-system/authorino-7498df8756-mf5p8" Apr 22 21:19:05.033177 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:05.033092 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-mf5p8" Apr 22 21:19:05.155581 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:05.155552 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-mf5p8"] Apr 22 21:19:05.158299 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:19:05.158268 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf06f486f_0fab_4ce5_80f1_81792aaa53be.slice/crio-2eeac84d0509e281d72335190a17e33c5269eeb68b8e127c2c32878beae509ea WatchSource:0}: Error finding container 2eeac84d0509e281d72335190a17e33c5269eeb68b8e127c2c32878beae509ea: Status 404 returned error can't find the container with id 2eeac84d0509e281d72335190a17e33c5269eeb68b8e127c2c32878beae509ea Apr 22 21:19:05.747815 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:05.747774 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-mf5p8" event={"ID":"f06f486f-0fab-4ce5-80f1-81792aaa53be","Type":"ContainerStarted","Data":"2eeac84d0509e281d72335190a17e33c5269eeb68b8e127c2c32878beae509ea"} Apr 22 21:19:09.765265 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:09.765221 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-mf5p8" event={"ID":"f06f486f-0fab-4ce5-80f1-81792aaa53be","Type":"ContainerStarted","Data":"e84a8b99e4c652d2b926f23a1d9ac89184e91eb09142cee408aa4cfeaa573ec0"} Apr 22 21:19:19.765723 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:19.765697 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ngls_b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b/ovn-acl-logging/0.log" Apr 22 21:19:19.767988 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:19.767968 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ngls_b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b/ovn-acl-logging/0.log" Apr 22 21:19:37.376140 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:37.376040 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-7498df8756-mf5p8" podStartSLOduration=29.6955112 podStartE2EDuration="33.376024428s" podCreationTimestamp="2026-04-22 21:19:04 +0000 UTC" firstStartedPulling="2026-04-22 21:19:05.159631353 +0000 UTC m=+585.885527931" lastFinishedPulling="2026-04-22 21:19:08.840144581 +0000 UTC m=+589.566041159" observedRunningTime="2026-04-22 21:19:09.77947728 +0000 UTC m=+590.505373877" watchObservedRunningTime="2026-04-22 21:19:37.376024428 +0000 UTC m=+618.101921073" Apr 22 21:19:37.376515 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:37.376149 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-649dc778fc-9tmpk"] Apr 22 21:19:37.379635 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:37.379615 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-649dc778fc-9tmpk" Apr 22 21:19:37.382382 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:37.382357 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Apr 22 21:19:37.385301 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:37.385281 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-649dc778fc-9tmpk"] Apr 22 21:19:37.431266 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:37.431231 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/13b470de-37e8-4e79-b13b-21e87974e098-tls-cert\") pod \"authorino-649dc778fc-9tmpk\" (UID: \"13b470de-37e8-4e79-b13b-21e87974e098\") " pod="kuadrant-system/authorino-649dc778fc-9tmpk" Apr 22 21:19:37.431420 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:37.431291 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54hd6\" (UniqueName: \"kubernetes.io/projected/13b470de-37e8-4e79-b13b-21e87974e098-kube-api-access-54hd6\") pod \"authorino-649dc778fc-9tmpk\" (UID: \"13b470de-37e8-4e79-b13b-21e87974e098\") " pod="kuadrant-system/authorino-649dc778fc-9tmpk" Apr 22 21:19:37.532546 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:37.532516 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/13b470de-37e8-4e79-b13b-21e87974e098-tls-cert\") pod \"authorino-649dc778fc-9tmpk\" (UID: \"13b470de-37e8-4e79-b13b-21e87974e098\") " pod="kuadrant-system/authorino-649dc778fc-9tmpk" Apr 22 21:19:37.532692 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:37.532566 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-54hd6\" (UniqueName: \"kubernetes.io/projected/13b470de-37e8-4e79-b13b-21e87974e098-kube-api-access-54hd6\") pod \"authorino-649dc778fc-9tmpk\" (UID: \"13b470de-37e8-4e79-b13b-21e87974e098\") " pod="kuadrant-system/authorino-649dc778fc-9tmpk" Apr 22 21:19:37.535105 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:37.535087 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/13b470de-37e8-4e79-b13b-21e87974e098-tls-cert\") pod \"authorino-649dc778fc-9tmpk\" (UID: \"13b470de-37e8-4e79-b13b-21e87974e098\") " pod="kuadrant-system/authorino-649dc778fc-9tmpk" Apr 22 21:19:37.540125 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:37.540102 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54hd6\" (UniqueName: \"kubernetes.io/projected/13b470de-37e8-4e79-b13b-21e87974e098-kube-api-access-54hd6\") pod \"authorino-649dc778fc-9tmpk\" (UID: \"13b470de-37e8-4e79-b13b-21e87974e098\") " pod="kuadrant-system/authorino-649dc778fc-9tmpk" Apr 22 21:19:37.689691 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:37.689667 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-649dc778fc-9tmpk" Apr 22 21:19:37.826926 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:37.826893 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-649dc778fc-9tmpk"] Apr 22 21:19:37.830038 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:19:37.830010 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod13b470de_37e8_4e79_b13b_21e87974e098.slice/crio-0e06fd1f0e94197234fc7ddeb07bed44a49ac9938a947fa1a79c6680babb35a6 WatchSource:0}: Error finding container 0e06fd1f0e94197234fc7ddeb07bed44a49ac9938a947fa1a79c6680babb35a6: Status 404 returned error can't find the container with id 0e06fd1f0e94197234fc7ddeb07bed44a49ac9938a947fa1a79c6680babb35a6 Apr 22 21:19:37.863626 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:37.863598 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-649dc778fc-9tmpk" event={"ID":"13b470de-37e8-4e79-b13b-21e87974e098","Type":"ContainerStarted","Data":"0e06fd1f0e94197234fc7ddeb07bed44a49ac9938a947fa1a79c6680babb35a6"} Apr 22 21:19:38.868689 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:38.868654 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-649dc778fc-9tmpk" event={"ID":"13b470de-37e8-4e79-b13b-21e87974e098","Type":"ContainerStarted","Data":"3c6b793655a9cdab1c3f77690850fb596c6722cf67f032d509acd7a250b45402"} Apr 22 21:19:38.885225 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:38.885183 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-649dc778fc-9tmpk" podStartSLOduration=1.541389767 podStartE2EDuration="1.885170204s" podCreationTimestamp="2026-04-22 21:19:37 +0000 UTC" firstStartedPulling="2026-04-22 21:19:37.831668731 +0000 UTC m=+618.557565313" lastFinishedPulling="2026-04-22 21:19:38.175449169 +0000 UTC m=+618.901345750" observedRunningTime="2026-04-22 21:19:38.884454066 +0000 UTC m=+619.610350665" watchObservedRunningTime="2026-04-22 21:19:38.885170204 +0000 UTC m=+619.611066801" Apr 22 21:19:38.909492 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:38.909465 2580 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-mf5p8"] Apr 22 21:19:38.909686 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:38.909654 2580 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-7498df8756-mf5p8" podUID="f06f486f-0fab-4ce5-80f1-81792aaa53be" containerName="authorino" containerID="cri-o://e84a8b99e4c652d2b926f23a1d9ac89184e91eb09142cee408aa4cfeaa573ec0" gracePeriod=30 Apr 22 21:19:39.149453 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:39.149433 2580 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-mf5p8" Apr 22 21:19:39.247900 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:39.247863 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wwvg\" (UniqueName: \"kubernetes.io/projected/f06f486f-0fab-4ce5-80f1-81792aaa53be-kube-api-access-7wwvg\") pod \"f06f486f-0fab-4ce5-80f1-81792aaa53be\" (UID: \"f06f486f-0fab-4ce5-80f1-81792aaa53be\") " Apr 22 21:19:39.250064 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:39.250034 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f06f486f-0fab-4ce5-80f1-81792aaa53be-kube-api-access-7wwvg" (OuterVolumeSpecName: "kube-api-access-7wwvg") pod "f06f486f-0fab-4ce5-80f1-81792aaa53be" (UID: "f06f486f-0fab-4ce5-80f1-81792aaa53be"). InnerVolumeSpecName "kube-api-access-7wwvg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 22 21:19:39.349165 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:39.349133 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wwvg\" (UniqueName: \"kubernetes.io/projected/f06f486f-0fab-4ce5-80f1-81792aaa53be-kube-api-access-7wwvg\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:19:39.872585 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:39.872546 2580 generic.go:358] "Generic (PLEG): container finished" podID="f06f486f-0fab-4ce5-80f1-81792aaa53be" containerID="e84a8b99e4c652d2b926f23a1d9ac89184e91eb09142cee408aa4cfeaa573ec0" exitCode=0 Apr 22 21:19:39.873175 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:39.872602 2580 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-mf5p8" Apr 22 21:19:39.873175 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:39.872587 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-mf5p8" event={"ID":"f06f486f-0fab-4ce5-80f1-81792aaa53be","Type":"ContainerDied","Data":"e84a8b99e4c652d2b926f23a1d9ac89184e91eb09142cee408aa4cfeaa573ec0"} Apr 22 21:19:39.873175 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:39.872707 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-mf5p8" event={"ID":"f06f486f-0fab-4ce5-80f1-81792aaa53be","Type":"ContainerDied","Data":"2eeac84d0509e281d72335190a17e33c5269eeb68b8e127c2c32878beae509ea"} Apr 22 21:19:39.873175 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:39.872744 2580 scope.go:117] "RemoveContainer" containerID="e84a8b99e4c652d2b926f23a1d9ac89184e91eb09142cee408aa4cfeaa573ec0" Apr 22 21:19:39.880965 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:39.880944 2580 scope.go:117] "RemoveContainer" containerID="e84a8b99e4c652d2b926f23a1d9ac89184e91eb09142cee408aa4cfeaa573ec0" Apr 22 21:19:39.881221 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:19:39.881200 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e84a8b99e4c652d2b926f23a1d9ac89184e91eb09142cee408aa4cfeaa573ec0\": container with ID starting with e84a8b99e4c652d2b926f23a1d9ac89184e91eb09142cee408aa4cfeaa573ec0 not found: ID does not exist" containerID="e84a8b99e4c652d2b926f23a1d9ac89184e91eb09142cee408aa4cfeaa573ec0" Apr 22 21:19:39.881286 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:39.881229 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e84a8b99e4c652d2b926f23a1d9ac89184e91eb09142cee408aa4cfeaa573ec0"} err="failed to get container status \"e84a8b99e4c652d2b926f23a1d9ac89184e91eb09142cee408aa4cfeaa573ec0\": rpc error: code = NotFound desc = could not find container \"e84a8b99e4c652d2b926f23a1d9ac89184e91eb09142cee408aa4cfeaa573ec0\": container with ID starting with e84a8b99e4c652d2b926f23a1d9ac89184e91eb09142cee408aa4cfeaa573ec0 not found: ID does not exist" Apr 22 21:19:39.902205 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:39.902175 2580 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-mf5p8"] Apr 22 21:19:39.908008 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:39.907988 2580 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7498df8756-mf5p8"] Apr 22 21:19:41.863077 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:19:41.863043 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f06f486f-0fab-4ce5-80f1-81792aaa53be" path="/var/lib/kubelet/pods/f06f486f-0fab-4ce5-80f1-81792aaa53be/volumes" Apr 22 21:20:18.170122 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.170082 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws"] Apr 22 21:20:18.172883 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.170439 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f06f486f-0fab-4ce5-80f1-81792aaa53be" containerName="authorino" Apr 22 21:20:18.172883 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.170454 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f06f486f-0fab-4ce5-80f1-81792aaa53be" containerName="authorino" Apr 22 21:20:18.172883 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.170527 2580 memory_manager.go:356] "RemoveStaleState removing state" podUID="f06f486f-0fab-4ce5-80f1-81792aaa53be" containerName="authorino" Apr 22 21:20:18.173803 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.173787 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws" Apr 22 21:20:18.176415 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.176396 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Apr 22 21:20:18.176534 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.176440 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-8wbgm\"" Apr 22 21:20:18.177713 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.177695 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Apr 22 21:20:18.177815 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.177770 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Apr 22 21:20:18.181444 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.181426 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws"] Apr 22 21:20:18.265301 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.265265 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0c29ddbf-acab-4c25-9c76-adf84a1ee551-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws\" (UID: \"0c29ddbf-acab-4c25-9c76-adf84a1ee551\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws" Apr 22 21:20:18.265301 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.265303 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c29ddbf-acab-4c25-9c76-adf84a1ee551-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws\" (UID: \"0c29ddbf-acab-4c25-9c76-adf84a1ee551\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws" Apr 22 21:20:18.265499 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.265323 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrcc8\" (UniqueName: \"kubernetes.io/projected/0c29ddbf-acab-4c25-9c76-adf84a1ee551-kube-api-access-lrcc8\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws\" (UID: \"0c29ddbf-acab-4c25-9c76-adf84a1ee551\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws" Apr 22 21:20:18.265499 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.265342 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c29ddbf-acab-4c25-9c76-adf84a1ee551-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws\" (UID: \"0c29ddbf-acab-4c25-9c76-adf84a1ee551\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws" Apr 22 21:20:18.265499 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.265396 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c29ddbf-acab-4c25-9c76-adf84a1ee551-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws\" (UID: \"0c29ddbf-acab-4c25-9c76-adf84a1ee551\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws" Apr 22 21:20:18.265499 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.265437 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c29ddbf-acab-4c25-9c76-adf84a1ee551-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws\" (UID: \"0c29ddbf-acab-4c25-9c76-adf84a1ee551\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws" Apr 22 21:20:18.366046 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.366014 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c29ddbf-acab-4c25-9c76-adf84a1ee551-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws\" (UID: \"0c29ddbf-acab-4c25-9c76-adf84a1ee551\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws" Apr 22 21:20:18.366209 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.366081 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c29ddbf-acab-4c25-9c76-adf84a1ee551-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws\" (UID: \"0c29ddbf-acab-4c25-9c76-adf84a1ee551\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws" Apr 22 21:20:18.366209 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.366134 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0c29ddbf-acab-4c25-9c76-adf84a1ee551-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws\" (UID: \"0c29ddbf-acab-4c25-9c76-adf84a1ee551\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws" Apr 22 21:20:18.366209 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.366155 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c29ddbf-acab-4c25-9c76-adf84a1ee551-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws\" (UID: \"0c29ddbf-acab-4c25-9c76-adf84a1ee551\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws" Apr 22 21:20:18.366209 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.366179 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lrcc8\" (UniqueName: \"kubernetes.io/projected/0c29ddbf-acab-4c25-9c76-adf84a1ee551-kube-api-access-lrcc8\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws\" (UID: \"0c29ddbf-acab-4c25-9c76-adf84a1ee551\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws" Apr 22 21:20:18.366209 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.366198 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c29ddbf-acab-4c25-9c76-adf84a1ee551-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws\" (UID: \"0c29ddbf-acab-4c25-9c76-adf84a1ee551\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws" Apr 22 21:20:18.366566 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.366458 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c29ddbf-acab-4c25-9c76-adf84a1ee551-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws\" (UID: \"0c29ddbf-acab-4c25-9c76-adf84a1ee551\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws" Apr 22 21:20:18.366566 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.366527 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c29ddbf-acab-4c25-9c76-adf84a1ee551-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws\" (UID: \"0c29ddbf-acab-4c25-9c76-adf84a1ee551\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws" Apr 22 21:20:18.366707 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.366581 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0c29ddbf-acab-4c25-9c76-adf84a1ee551-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws\" (UID: \"0c29ddbf-acab-4c25-9c76-adf84a1ee551\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws" Apr 22 21:20:18.368507 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.368488 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c29ddbf-acab-4c25-9c76-adf84a1ee551-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws\" (UID: \"0c29ddbf-acab-4c25-9c76-adf84a1ee551\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws" Apr 22 21:20:18.368884 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.368845 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c29ddbf-acab-4c25-9c76-adf84a1ee551-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws\" (UID: \"0c29ddbf-acab-4c25-9c76-adf84a1ee551\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws" Apr 22 21:20:18.373811 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.373790 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrcc8\" (UniqueName: \"kubernetes.io/projected/0c29ddbf-acab-4c25-9c76-adf84a1ee551-kube-api-access-lrcc8\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws\" (UID: \"0c29ddbf-acab-4c25-9c76-adf84a1ee551\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws" Apr 22 21:20:18.485212 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.485128 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws" Apr 22 21:20:18.608051 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.607954 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws"] Apr 22 21:20:18.610974 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:20:18.610940 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c29ddbf_acab_4c25_9c76_adf84a1ee551.slice/crio-31d26bd241f93a970f7a17d7490f1a828395de8b993cd1f628ab6f30b50b3672 WatchSource:0}: Error finding container 31d26bd241f93a970f7a17d7490f1a828395de8b993cd1f628ab6f30b50b3672: Status 404 returned error can't find the container with id 31d26bd241f93a970f7a17d7490f1a828395de8b993cd1f628ab6f30b50b3672 Apr 22 21:20:18.612719 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.612703 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 22 21:20:18.998364 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:18.998317 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws" event={"ID":"0c29ddbf-acab-4c25-9c76-adf84a1ee551","Type":"ContainerStarted","Data":"31d26bd241f93a970f7a17d7490f1a828395de8b993cd1f628ab6f30b50b3672"} Apr 22 21:20:24.018542 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:24.018505 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws" event={"ID":"0c29ddbf-acab-4c25-9c76-adf84a1ee551","Type":"ContainerStarted","Data":"b5288fd5d05cea553c204c6f66ecceed2c6f03794a949e338d29ef6f41bd9486"} Apr 22 21:20:29.036713 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:29.036671 2580 generic.go:358] "Generic (PLEG): container finished" podID="0c29ddbf-acab-4c25-9c76-adf84a1ee551" containerID="b5288fd5d05cea553c204c6f66ecceed2c6f03794a949e338d29ef6f41bd9486" exitCode=0 Apr 22 21:20:29.037302 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:29.036750 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws" event={"ID":"0c29ddbf-acab-4c25-9c76-adf84a1ee551","Type":"ContainerDied","Data":"b5288fd5d05cea553c204c6f66ecceed2c6f03794a949e338d29ef6f41bd9486"} Apr 22 21:20:31.046294 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:31.046259 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws" event={"ID":"0c29ddbf-acab-4c25-9c76-adf84a1ee551","Type":"ContainerStarted","Data":"67effa1e5ad868fd8849d1d45f0e788d563998d8bd4abb1ef841fe826f54b0d4"} Apr 22 21:20:31.046764 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:31.046488 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws" Apr 22 21:20:31.066350 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:31.066297 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws" podStartSLOduration=1.566967026 podStartE2EDuration="13.066283633s" podCreationTimestamp="2026-04-22 21:20:18 +0000 UTC" firstStartedPulling="2026-04-22 21:20:18.612891518 +0000 UTC m=+659.338788096" lastFinishedPulling="2026-04-22 21:20:30.112208123 +0000 UTC m=+670.838104703" observedRunningTime="2026-04-22 21:20:31.064385903 +0000 UTC m=+671.790282517" watchObservedRunningTime="2026-04-22 21:20:31.066283633 +0000 UTC m=+671.792180262" Apr 22 21:20:31.078566 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:31.078541 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25"] Apr 22 21:20:31.082090 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:31.082070 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25" Apr 22 21:20:31.084696 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:31.084645 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Apr 22 21:20:31.094159 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:31.094136 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25"] Apr 22 21:20:31.183278 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:31.183251 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqtgp\" (UniqueName: \"kubernetes.io/projected/af92076a-1f97-4b23-be10-ccda02660902-kube-api-access-gqtgp\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-qdp25\" (UID: \"af92076a-1f97-4b23-be10-ccda02660902\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25" Apr 22 21:20:31.183427 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:31.183288 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/af92076a-1f97-4b23-be10-ccda02660902-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-qdp25\" (UID: \"af92076a-1f97-4b23-be10-ccda02660902\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25" Apr 22 21:20:31.183427 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:31.183339 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af92076a-1f97-4b23-be10-ccda02660902-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-qdp25\" (UID: \"af92076a-1f97-4b23-be10-ccda02660902\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25" Apr 22 21:20:31.183427 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:31.183392 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af92076a-1f97-4b23-be10-ccda02660902-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-qdp25\" (UID: \"af92076a-1f97-4b23-be10-ccda02660902\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25" Apr 22 21:20:31.183555 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:31.183443 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af92076a-1f97-4b23-be10-ccda02660902-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-qdp25\" (UID: \"af92076a-1f97-4b23-be10-ccda02660902\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25" Apr 22 21:20:31.183555 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:31.183472 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af92076a-1f97-4b23-be10-ccda02660902-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-qdp25\" (UID: \"af92076a-1f97-4b23-be10-ccda02660902\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25" Apr 22 21:20:31.284183 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:31.284155 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af92076a-1f97-4b23-be10-ccda02660902-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-qdp25\" (UID: \"af92076a-1f97-4b23-be10-ccda02660902\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25" Apr 22 21:20:31.284334 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:31.284191 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af92076a-1f97-4b23-be10-ccda02660902-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-qdp25\" (UID: \"af92076a-1f97-4b23-be10-ccda02660902\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25" Apr 22 21:20:31.284334 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:31.284219 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af92076a-1f97-4b23-be10-ccda02660902-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-qdp25\" (UID: \"af92076a-1f97-4b23-be10-ccda02660902\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25" Apr 22 21:20:31.284334 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:31.284248 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gqtgp\" (UniqueName: \"kubernetes.io/projected/af92076a-1f97-4b23-be10-ccda02660902-kube-api-access-gqtgp\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-qdp25\" (UID: \"af92076a-1f97-4b23-be10-ccda02660902\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25" Apr 22 21:20:31.284334 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:31.284271 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/af92076a-1f97-4b23-be10-ccda02660902-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-qdp25\" (UID: \"af92076a-1f97-4b23-be10-ccda02660902\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25" Apr 22 21:20:31.284334 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:31.284288 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af92076a-1f97-4b23-be10-ccda02660902-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-qdp25\" (UID: \"af92076a-1f97-4b23-be10-ccda02660902\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25" Apr 22 21:20:31.284597 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:31.284570 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af92076a-1f97-4b23-be10-ccda02660902-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-qdp25\" (UID: \"af92076a-1f97-4b23-be10-ccda02660902\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25" Apr 22 21:20:31.284736 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:31.284613 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/af92076a-1f97-4b23-be10-ccda02660902-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-qdp25\" (UID: \"af92076a-1f97-4b23-be10-ccda02660902\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25" Apr 22 21:20:31.284736 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:31.284654 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af92076a-1f97-4b23-be10-ccda02660902-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-qdp25\" (UID: \"af92076a-1f97-4b23-be10-ccda02660902\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25" Apr 22 21:20:31.286687 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:31.286666 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af92076a-1f97-4b23-be10-ccda02660902-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-qdp25\" (UID: \"af92076a-1f97-4b23-be10-ccda02660902\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25" Apr 22 21:20:31.286820 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:31.286804 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af92076a-1f97-4b23-be10-ccda02660902-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-qdp25\" (UID: \"af92076a-1f97-4b23-be10-ccda02660902\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25" Apr 22 21:20:31.292067 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:31.292047 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqtgp\" (UniqueName: \"kubernetes.io/projected/af92076a-1f97-4b23-be10-ccda02660902-kube-api-access-gqtgp\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-qdp25\" (UID: \"af92076a-1f97-4b23-be10-ccda02660902\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25" Apr 22 21:20:31.394997 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:31.394972 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25" Apr 22 21:20:31.517945 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:31.517924 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25"] Apr 22 21:20:31.519929 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:20:31.519900 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf92076a_1f97_4b23_be10_ccda02660902.slice/crio-cc96714f33590358c140adce6a20a03376d1dfde128589912f3584525b60ad78 WatchSource:0}: Error finding container cc96714f33590358c140adce6a20a03376d1dfde128589912f3584525b60ad78: Status 404 returned error can't find the container with id cc96714f33590358c140adce6a20a03376d1dfde128589912f3584525b60ad78 Apr 22 21:20:32.051503 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:32.051462 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25" event={"ID":"af92076a-1f97-4b23-be10-ccda02660902","Type":"ContainerStarted","Data":"8b7edaf02f3e69aa7695c7459c8c560c692e4e8899d37aba1531c4e981c5253d"} Apr 22 21:20:32.051503 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:32.051508 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25" event={"ID":"af92076a-1f97-4b23-be10-ccda02660902","Type":"ContainerStarted","Data":"cc96714f33590358c140adce6a20a03376d1dfde128589912f3584525b60ad78"} Apr 22 21:20:40.088586 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:40.088503 2580 generic.go:358] "Generic (PLEG): container finished" podID="af92076a-1f97-4b23-be10-ccda02660902" containerID="8b7edaf02f3e69aa7695c7459c8c560c692e4e8899d37aba1531c4e981c5253d" exitCode=0 Apr 22 21:20:40.088586 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:40.088571 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25" event={"ID":"af92076a-1f97-4b23-be10-ccda02660902","Type":"ContainerDied","Data":"8b7edaf02f3e69aa7695c7459c8c560c692e4e8899d37aba1531c4e981c5253d"} Apr 22 21:20:41.093429 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:41.093344 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25" event={"ID":"af92076a-1f97-4b23-be10-ccda02660902","Type":"ContainerStarted","Data":"683b06bc159860596b7cef4c6d4fa30bfcf62dc8ac81b2e0c0203834f2b71d4d"} Apr 22 21:20:41.093828 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:41.093553 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25" Apr 22 21:20:41.112775 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:41.112728 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25" podStartSLOduration=9.552544984 podStartE2EDuration="10.112714553s" podCreationTimestamp="2026-04-22 21:20:31 +0000 UTC" firstStartedPulling="2026-04-22 21:20:40.089277716 +0000 UTC m=+680.815174295" lastFinishedPulling="2026-04-22 21:20:40.649447287 +0000 UTC m=+681.375343864" observedRunningTime="2026-04-22 21:20:41.11119191 +0000 UTC m=+681.837088513" watchObservedRunningTime="2026-04-22 21:20:41.112714553 +0000 UTC m=+681.838611152" Apr 22 21:20:42.064414 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:42.064381 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws" Apr 22 21:20:52.111202 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:20:52.111158 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-qdp25" Apr 22 21:21:45.602256 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:21:45.602213 2580 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-649dc778fc-9tmpk"] Apr 22 21:21:45.602884 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:21:45.602524 2580 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-649dc778fc-9tmpk" podUID="13b470de-37e8-4e79-b13b-21e87974e098" containerName="authorino" containerID="cri-o://3c6b793655a9cdab1c3f77690850fb596c6722cf67f032d509acd7a250b45402" gracePeriod=30 Apr 22 21:21:45.837067 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:21:45.837038 2580 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-649dc778fc-9tmpk" Apr 22 21:21:45.943059 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:21:45.943021 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54hd6\" (UniqueName: \"kubernetes.io/projected/13b470de-37e8-4e79-b13b-21e87974e098-kube-api-access-54hd6\") pod \"13b470de-37e8-4e79-b13b-21e87974e098\" (UID: \"13b470de-37e8-4e79-b13b-21e87974e098\") " Apr 22 21:21:45.943059 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:21:45.943061 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/13b470de-37e8-4e79-b13b-21e87974e098-tls-cert\") pod \"13b470de-37e8-4e79-b13b-21e87974e098\" (UID: \"13b470de-37e8-4e79-b13b-21e87974e098\") " Apr 22 21:21:45.945255 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:21:45.945230 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13b470de-37e8-4e79-b13b-21e87974e098-kube-api-access-54hd6" (OuterVolumeSpecName: "kube-api-access-54hd6") pod "13b470de-37e8-4e79-b13b-21e87974e098" (UID: "13b470de-37e8-4e79-b13b-21e87974e098"). InnerVolumeSpecName "kube-api-access-54hd6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 22 21:21:45.952838 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:21:45.952816 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13b470de-37e8-4e79-b13b-21e87974e098-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "13b470de-37e8-4e79-b13b-21e87974e098" (UID: "13b470de-37e8-4e79-b13b-21e87974e098"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 22 21:21:46.044579 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:21:46.044547 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54hd6\" (UniqueName: \"kubernetes.io/projected/13b470de-37e8-4e79-b13b-21e87974e098-kube-api-access-54hd6\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:21:46.044579 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:21:46.044577 2580 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/13b470de-37e8-4e79-b13b-21e87974e098-tls-cert\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:21:46.317879 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:21:46.317775 2580 generic.go:358] "Generic (PLEG): container finished" podID="13b470de-37e8-4e79-b13b-21e87974e098" containerID="3c6b793655a9cdab1c3f77690850fb596c6722cf67f032d509acd7a250b45402" exitCode=0 Apr 22 21:21:46.317879 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:21:46.317829 2580 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-649dc778fc-9tmpk" Apr 22 21:21:46.318097 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:21:46.317887 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-649dc778fc-9tmpk" event={"ID":"13b470de-37e8-4e79-b13b-21e87974e098","Type":"ContainerDied","Data":"3c6b793655a9cdab1c3f77690850fb596c6722cf67f032d509acd7a250b45402"} Apr 22 21:21:46.318097 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:21:46.317927 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-649dc778fc-9tmpk" event={"ID":"13b470de-37e8-4e79-b13b-21e87974e098","Type":"ContainerDied","Data":"0e06fd1f0e94197234fc7ddeb07bed44a49ac9938a947fa1a79c6680babb35a6"} Apr 22 21:21:46.318097 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:21:46.317950 2580 scope.go:117] "RemoveContainer" containerID="3c6b793655a9cdab1c3f77690850fb596c6722cf67f032d509acd7a250b45402" Apr 22 21:21:46.326623 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:21:46.326602 2580 scope.go:117] "RemoveContainer" containerID="3c6b793655a9cdab1c3f77690850fb596c6722cf67f032d509acd7a250b45402" Apr 22 21:21:46.326847 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:21:46.326827 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c6b793655a9cdab1c3f77690850fb596c6722cf67f032d509acd7a250b45402\": container with ID starting with 3c6b793655a9cdab1c3f77690850fb596c6722cf67f032d509acd7a250b45402 not found: ID does not exist" containerID="3c6b793655a9cdab1c3f77690850fb596c6722cf67f032d509acd7a250b45402" Apr 22 21:21:46.326951 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:21:46.326874 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c6b793655a9cdab1c3f77690850fb596c6722cf67f032d509acd7a250b45402"} err="failed to get container status \"3c6b793655a9cdab1c3f77690850fb596c6722cf67f032d509acd7a250b45402\": rpc error: code = NotFound desc = could not find container \"3c6b793655a9cdab1c3f77690850fb596c6722cf67f032d509acd7a250b45402\": container with ID starting with 3c6b793655a9cdab1c3f77690850fb596c6722cf67f032d509acd7a250b45402 not found: ID does not exist" Apr 22 21:21:46.338043 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:21:46.338022 2580 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-649dc778fc-9tmpk"] Apr 22 21:21:46.340218 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:21:46.340198 2580 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-649dc778fc-9tmpk"] Apr 22 21:21:47.863216 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:21:47.863185 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13b470de-37e8-4e79-b13b-21e87974e098" path="/var/lib/kubelet/pods/13b470de-37e8-4e79-b13b-21e87974e098/volumes" Apr 22 21:24:19.791685 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:24:19.791655 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ngls_b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b/ovn-acl-logging/0.log" Apr 22 21:24:19.799289 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:24:19.799265 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ngls_b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b/ovn-acl-logging/0.log" Apr 22 21:29:19.823494 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:29:19.823468 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ngls_b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b/ovn-acl-logging/0.log" Apr 22 21:29:19.830406 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:29:19.830387 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ngls_b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b/ovn-acl-logging/0.log" Apr 22 21:30:00.130755 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:30:00.130663 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29614890-vd42l"] Apr 22 21:30:00.131258 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:30:00.131139 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13b470de-37e8-4e79-b13b-21e87974e098" containerName="authorino" Apr 22 21:30:00.131258 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:30:00.131155 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="13b470de-37e8-4e79-b13b-21e87974e098" containerName="authorino" Apr 22 21:30:00.131258 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:30:00.131208 2580 memory_manager.go:356] "RemoveStaleState removing state" podUID="13b470de-37e8-4e79-b13b-21e87974e098" containerName="authorino" Apr 22 21:30:00.134371 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:30:00.134353 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29614890-vd42l" Apr 22 21:30:00.136962 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:30:00.136945 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-p9m6q\"" Apr 22 21:30:00.148022 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:30:00.147998 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29614890-vd42l"] Apr 22 21:30:00.222237 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:30:00.222203 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whtfx\" (UniqueName: \"kubernetes.io/projected/d05fb114-d12a-49ab-9e24-3a48c75527bf-kube-api-access-whtfx\") pod \"maas-api-key-cleanup-29614890-vd42l\" (UID: \"d05fb114-d12a-49ab-9e24-3a48c75527bf\") " pod="opendatahub/maas-api-key-cleanup-29614890-vd42l" Apr 22 21:30:00.322684 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:30:00.322655 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-whtfx\" (UniqueName: \"kubernetes.io/projected/d05fb114-d12a-49ab-9e24-3a48c75527bf-kube-api-access-whtfx\") pod \"maas-api-key-cleanup-29614890-vd42l\" (UID: \"d05fb114-d12a-49ab-9e24-3a48c75527bf\") " pod="opendatahub/maas-api-key-cleanup-29614890-vd42l" Apr 22 21:30:00.331721 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:30:00.331698 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whtfx\" (UniqueName: \"kubernetes.io/projected/d05fb114-d12a-49ab-9e24-3a48c75527bf-kube-api-access-whtfx\") pod \"maas-api-key-cleanup-29614890-vd42l\" (UID: \"d05fb114-d12a-49ab-9e24-3a48c75527bf\") " pod="opendatahub/maas-api-key-cleanup-29614890-vd42l" Apr 22 21:30:00.444067 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:30:00.443983 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29614890-vd42l" Apr 22 21:30:00.773450 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:30:00.773426 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29614890-vd42l"] Apr 22 21:30:00.775737 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:30:00.775707 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd05fb114_d12a_49ab_9e24_3a48c75527bf.slice/crio-cada8e80623697dbb3dc638b24e8211c1e789e377bea3551ed35690a863eedad WatchSource:0}: Error finding container cada8e80623697dbb3dc638b24e8211c1e789e377bea3551ed35690a863eedad: Status 404 returned error can't find the container with id cada8e80623697dbb3dc638b24e8211c1e789e377bea3551ed35690a863eedad Apr 22 21:30:00.777486 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:30:00.777466 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 22 21:30:00.983563 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:30:00.983517 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29614890-vd42l" event={"ID":"d05fb114-d12a-49ab-9e24-3a48c75527bf","Type":"ContainerStarted","Data":"cada8e80623697dbb3dc638b24e8211c1e789e377bea3551ed35690a863eedad"} Apr 22 21:30:03.995742 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:30:03.995698 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29614890-vd42l" event={"ID":"d05fb114-d12a-49ab-9e24-3a48c75527bf","Type":"ContainerStarted","Data":"027a2d806f5ce165adf78848a66995e3530833e88835bbcda087d7dcf43a6679"} Apr 22 21:30:04.010302 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:30:04.010263 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29614890-vd42l" podStartSLOduration=1.519961645 podStartE2EDuration="4.010251276s" podCreationTimestamp="2026-04-22 21:30:00 +0000 UTC" firstStartedPulling="2026-04-22 21:30:00.777598858 +0000 UTC m=+1241.503495436" lastFinishedPulling="2026-04-22 21:30:03.267888485 +0000 UTC m=+1243.993785067" observedRunningTime="2026-04-22 21:30:04.009059728 +0000 UTC m=+1244.734956327" watchObservedRunningTime="2026-04-22 21:30:04.010251276 +0000 UTC m=+1244.736147876" Apr 22 21:30:24.063613 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:30:24.063581 2580 generic.go:358] "Generic (PLEG): container finished" podID="d05fb114-d12a-49ab-9e24-3a48c75527bf" containerID="027a2d806f5ce165adf78848a66995e3530833e88835bbcda087d7dcf43a6679" exitCode=6 Apr 22 21:30:24.063968 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:30:24.063656 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29614890-vd42l" event={"ID":"d05fb114-d12a-49ab-9e24-3a48c75527bf","Type":"ContainerDied","Data":"027a2d806f5ce165adf78848a66995e3530833e88835bbcda087d7dcf43a6679"} Apr 22 21:30:24.064047 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:30:24.064026 2580 scope.go:117] "RemoveContainer" containerID="027a2d806f5ce165adf78848a66995e3530833e88835bbcda087d7dcf43a6679" Apr 22 21:30:25.068832 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:30:25.068799 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29614890-vd42l" event={"ID":"d05fb114-d12a-49ab-9e24-3a48c75527bf","Type":"ContainerStarted","Data":"19220d0b0b854df6e96ef78d7b0ec436e45d58ed5b630bd02041a24fab96846d"} Apr 22 21:30:45.139677 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:30:45.139640 2580 generic.go:358] "Generic (PLEG): container finished" podID="d05fb114-d12a-49ab-9e24-3a48c75527bf" containerID="19220d0b0b854df6e96ef78d7b0ec436e45d58ed5b630bd02041a24fab96846d" exitCode=6 Apr 22 21:30:45.140129 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:30:45.139714 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29614890-vd42l" event={"ID":"d05fb114-d12a-49ab-9e24-3a48c75527bf","Type":"ContainerDied","Data":"19220d0b0b854df6e96ef78d7b0ec436e45d58ed5b630bd02041a24fab96846d"} Apr 22 21:30:45.140129 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:30:45.139756 2580 scope.go:117] "RemoveContainer" containerID="027a2d806f5ce165adf78848a66995e3530833e88835bbcda087d7dcf43a6679" Apr 22 21:30:45.140129 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:30:45.140087 2580 scope.go:117] "RemoveContainer" containerID="19220d0b0b854df6e96ef78d7b0ec436e45d58ed5b630bd02041a24fab96846d" Apr 22 21:30:45.140323 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:30:45.140301 2580 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29614890-vd42l_opendatahub(d05fb114-d12a-49ab-9e24-3a48c75527bf)\"" pod="opendatahub/maas-api-key-cleanup-29614890-vd42l" podUID="d05fb114-d12a-49ab-9e24-3a48c75527bf" Apr 22 21:30:57.858632 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:30:57.858599 2580 scope.go:117] "RemoveContainer" containerID="19220d0b0b854df6e96ef78d7b0ec436e45d58ed5b630bd02041a24fab96846d" Apr 22 21:30:59.195229 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:30:59.195195 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29614890-vd42l" event={"ID":"d05fb114-d12a-49ab-9e24-3a48c75527bf","Type":"ContainerStarted","Data":"4b60955a47141f9264ae3b26b8d2627e27588b2ba9a7b87d997dd985c096b74c"} Apr 22 21:31:00.009743 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:31:00.009703 2580 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-29614890-vd42l"] Apr 22 21:31:00.198820 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:31:00.198786 2580 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-api-key-cleanup-29614890-vd42l" podUID="d05fb114-d12a-49ab-9e24-3a48c75527bf" containerName="cleanup" containerID="cri-o://4b60955a47141f9264ae3b26b8d2627e27588b2ba9a7b87d997dd985c096b74c" gracePeriod=30 Apr 22 21:31:18.643890 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:31:18.643841 2580 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29614890-vd42l" Apr 22 21:31:18.779624 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:31:18.779541 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whtfx\" (UniqueName: \"kubernetes.io/projected/d05fb114-d12a-49ab-9e24-3a48c75527bf-kube-api-access-whtfx\") pod \"d05fb114-d12a-49ab-9e24-3a48c75527bf\" (UID: \"d05fb114-d12a-49ab-9e24-3a48c75527bf\") " Apr 22 21:31:18.781918 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:31:18.781886 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d05fb114-d12a-49ab-9e24-3a48c75527bf-kube-api-access-whtfx" (OuterVolumeSpecName: "kube-api-access-whtfx") pod "d05fb114-d12a-49ab-9e24-3a48c75527bf" (UID: "d05fb114-d12a-49ab-9e24-3a48c75527bf"). InnerVolumeSpecName "kube-api-access-whtfx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 22 21:31:18.880211 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:31:18.880171 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whtfx\" (UniqueName: \"kubernetes.io/projected/d05fb114-d12a-49ab-9e24-3a48c75527bf-kube-api-access-whtfx\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:31:19.266453 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:31:19.266419 2580 generic.go:358] "Generic (PLEG): container finished" podID="d05fb114-d12a-49ab-9e24-3a48c75527bf" containerID="4b60955a47141f9264ae3b26b8d2627e27588b2ba9a7b87d997dd985c096b74c" exitCode=6 Apr 22 21:31:19.266646 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:31:19.266480 2580 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29614890-vd42l" Apr 22 21:31:19.266646 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:31:19.266505 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29614890-vd42l" event={"ID":"d05fb114-d12a-49ab-9e24-3a48c75527bf","Type":"ContainerDied","Data":"4b60955a47141f9264ae3b26b8d2627e27588b2ba9a7b87d997dd985c096b74c"} Apr 22 21:31:19.266646 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:31:19.266550 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29614890-vd42l" event={"ID":"d05fb114-d12a-49ab-9e24-3a48c75527bf","Type":"ContainerDied","Data":"cada8e80623697dbb3dc638b24e8211c1e789e377bea3551ed35690a863eedad"} Apr 22 21:31:19.266646 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:31:19.266567 2580 scope.go:117] "RemoveContainer" containerID="4b60955a47141f9264ae3b26b8d2627e27588b2ba9a7b87d997dd985c096b74c" Apr 22 21:31:19.275413 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:31:19.275392 2580 scope.go:117] "RemoveContainer" containerID="19220d0b0b854df6e96ef78d7b0ec436e45d58ed5b630bd02041a24fab96846d" Apr 22 21:31:19.282760 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:31:19.282738 2580 scope.go:117] "RemoveContainer" containerID="4b60955a47141f9264ae3b26b8d2627e27588b2ba9a7b87d997dd985c096b74c" Apr 22 21:31:19.283062 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:31:19.283041 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b60955a47141f9264ae3b26b8d2627e27588b2ba9a7b87d997dd985c096b74c\": container with ID starting with 4b60955a47141f9264ae3b26b8d2627e27588b2ba9a7b87d997dd985c096b74c not found: ID does not exist" containerID="4b60955a47141f9264ae3b26b8d2627e27588b2ba9a7b87d997dd985c096b74c" Apr 22 21:31:19.283152 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:31:19.283073 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b60955a47141f9264ae3b26b8d2627e27588b2ba9a7b87d997dd985c096b74c"} err="failed to get container status \"4b60955a47141f9264ae3b26b8d2627e27588b2ba9a7b87d997dd985c096b74c\": rpc error: code = NotFound desc = could not find container \"4b60955a47141f9264ae3b26b8d2627e27588b2ba9a7b87d997dd985c096b74c\": container with ID starting with 4b60955a47141f9264ae3b26b8d2627e27588b2ba9a7b87d997dd985c096b74c not found: ID does not exist" Apr 22 21:31:19.283152 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:31:19.283095 2580 scope.go:117] "RemoveContainer" containerID="19220d0b0b854df6e96ef78d7b0ec436e45d58ed5b630bd02041a24fab96846d" Apr 22 21:31:19.283341 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:31:19.283323 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19220d0b0b854df6e96ef78d7b0ec436e45d58ed5b630bd02041a24fab96846d\": container with ID starting with 19220d0b0b854df6e96ef78d7b0ec436e45d58ed5b630bd02041a24fab96846d not found: ID does not exist" containerID="19220d0b0b854df6e96ef78d7b0ec436e45d58ed5b630bd02041a24fab96846d" Apr 22 21:31:19.283379 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:31:19.283348 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19220d0b0b854df6e96ef78d7b0ec436e45d58ed5b630bd02041a24fab96846d"} err="failed to get container status \"19220d0b0b854df6e96ef78d7b0ec436e45d58ed5b630bd02041a24fab96846d\": rpc error: code = NotFound desc = could not find container \"19220d0b0b854df6e96ef78d7b0ec436e45d58ed5b630bd02041a24fab96846d\": container with ID starting with 19220d0b0b854df6e96ef78d7b0ec436e45d58ed5b630bd02041a24fab96846d not found: ID does not exist" Apr 22 21:31:19.287311 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:31:19.287288 2580 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-29614890-vd42l"] Apr 22 21:31:19.288905 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:31:19.288886 2580 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-29614890-vd42l"] Apr 22 21:31:19.862463 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:31:19.862435 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d05fb114-d12a-49ab-9e24-3a48c75527bf" path="/var/lib/kubelet/pods/d05fb114-d12a-49ab-9e24-3a48c75527bf/volumes" Apr 22 21:33:17.321252 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:33:17.321215 2580 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-dzf7g"] Apr 22 21:33:17.321680 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:33:17.321522 2580 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-dzf7g" podUID="d01cf883-823e-48de-80f8-4df566a38c28" containerName="manager" containerID="cri-o://67c06eec94d74a684a4d050c63a241fbf3e55bba8fd75bda84964825e2d4a1be" gracePeriod=10 Apr 22 21:33:17.569264 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:33:17.569243 2580 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-dzf7g" Apr 22 21:33:17.671611 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:33:17.671580 2580 generic.go:358] "Generic (PLEG): container finished" podID="d01cf883-823e-48de-80f8-4df566a38c28" containerID="67c06eec94d74a684a4d050c63a241fbf3e55bba8fd75bda84964825e2d4a1be" exitCode=0 Apr 22 21:33:17.671752 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:33:17.671665 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-dzf7g" event={"ID":"d01cf883-823e-48de-80f8-4df566a38c28","Type":"ContainerDied","Data":"67c06eec94d74a684a4d050c63a241fbf3e55bba8fd75bda84964825e2d4a1be"} Apr 22 21:33:17.671752 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:33:17.671692 2580 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-dzf7g" Apr 22 21:33:17.671752 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:33:17.671703 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-dzf7g" event={"ID":"d01cf883-823e-48de-80f8-4df566a38c28","Type":"ContainerDied","Data":"bb223610304c4187aeb0ea8c32407b58b82ecc736173ef444b15396cda646df8"} Apr 22 21:33:17.671752 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:33:17.671728 2580 scope.go:117] "RemoveContainer" containerID="67c06eec94d74a684a4d050c63a241fbf3e55bba8fd75bda84964825e2d4a1be" Apr 22 21:33:17.671960 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:33:17.671929 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d01cf883-823e-48de-80f8-4df566a38c28-extensions-socket-volume\") pod \"d01cf883-823e-48de-80f8-4df566a38c28\" (UID: \"d01cf883-823e-48de-80f8-4df566a38c28\") " Apr 22 21:33:17.672078 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:33:17.672056 2580 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dcg8m\" (UniqueName: \"kubernetes.io/projected/d01cf883-823e-48de-80f8-4df566a38c28-kube-api-access-dcg8m\") pod \"d01cf883-823e-48de-80f8-4df566a38c28\" (UID: \"d01cf883-823e-48de-80f8-4df566a38c28\") " Apr 22 21:33:17.672342 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:33:17.672317 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d01cf883-823e-48de-80f8-4df566a38c28-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d01cf883-823e-48de-80f8-4df566a38c28" (UID: "d01cf883-823e-48de-80f8-4df566a38c28"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 22 21:33:17.674346 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:33:17.674322 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d01cf883-823e-48de-80f8-4df566a38c28-kube-api-access-dcg8m" (OuterVolumeSpecName: "kube-api-access-dcg8m") pod "d01cf883-823e-48de-80f8-4df566a38c28" (UID: "d01cf883-823e-48de-80f8-4df566a38c28"). InnerVolumeSpecName "kube-api-access-dcg8m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 22 21:33:17.684555 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:33:17.684539 2580 scope.go:117] "RemoveContainer" containerID="67c06eec94d74a684a4d050c63a241fbf3e55bba8fd75bda84964825e2d4a1be" Apr 22 21:33:17.684816 ip-10-0-136-20 kubenswrapper[2580]: E0422 21:33:17.684787 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67c06eec94d74a684a4d050c63a241fbf3e55bba8fd75bda84964825e2d4a1be\": container with ID starting with 67c06eec94d74a684a4d050c63a241fbf3e55bba8fd75bda84964825e2d4a1be not found: ID does not exist" containerID="67c06eec94d74a684a4d050c63a241fbf3e55bba8fd75bda84964825e2d4a1be" Apr 22 21:33:17.684888 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:33:17.684812 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67c06eec94d74a684a4d050c63a241fbf3e55bba8fd75bda84964825e2d4a1be"} err="failed to get container status \"67c06eec94d74a684a4d050c63a241fbf3e55bba8fd75bda84964825e2d4a1be\": rpc error: code = NotFound desc = could not find container \"67c06eec94d74a684a4d050c63a241fbf3e55bba8fd75bda84964825e2d4a1be\": container with ID starting with 67c06eec94d74a684a4d050c63a241fbf3e55bba8fd75bda84964825e2d4a1be not found: ID does not exist" Apr 22 21:33:17.773278 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:33:17.773257 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcg8m\" (UniqueName: \"kubernetes.io/projected/d01cf883-823e-48de-80f8-4df566a38c28-kube-api-access-dcg8m\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:33:17.773278 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:33:17.773277 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d01cf883-823e-48de-80f8-4df566a38c28-extensions-socket-volume\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Apr 22 21:33:17.987645 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:33:17.987566 2580 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-dzf7g"] Apr 22 21:33:17.990838 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:33:17.990811 2580 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-dzf7g"] Apr 22 21:33:19.863415 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:33:19.863383 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d01cf883-823e-48de-80f8-4df566a38c28" path="/var/lib/kubelet/pods/d01cf883-823e-48de-80f8-4df566a38c28/volumes" Apr 22 21:34:19.845467 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:19.845438 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ngls_b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b/ovn-acl-logging/0.log" Apr 22 21:34:19.860367 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:19.860342 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ngls_b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b/ovn-acl-logging/0.log" Apr 22 21:34:23.414331 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:23.414299 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-f6s65"] Apr 22 21:34:23.414697 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:23.414638 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d01cf883-823e-48de-80f8-4df566a38c28" containerName="manager" Apr 22 21:34:23.414697 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:23.414650 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d01cf883-823e-48de-80f8-4df566a38c28" containerName="manager" Apr 22 21:34:23.414697 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:23.414659 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d05fb114-d12a-49ab-9e24-3a48c75527bf" containerName="cleanup" Apr 22 21:34:23.414697 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:23.414665 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d05fb114-d12a-49ab-9e24-3a48c75527bf" containerName="cleanup" Apr 22 21:34:23.414697 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:23.414697 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d05fb114-d12a-49ab-9e24-3a48c75527bf" containerName="cleanup" Apr 22 21:34:23.414888 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:23.414703 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d05fb114-d12a-49ab-9e24-3a48c75527bf" containerName="cleanup" Apr 22 21:34:23.414888 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:23.414754 2580 memory_manager.go:356] "RemoveStaleState removing state" podUID="d01cf883-823e-48de-80f8-4df566a38c28" containerName="manager" Apr 22 21:34:23.414888 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:23.414764 2580 memory_manager.go:356] "RemoveStaleState removing state" podUID="d05fb114-d12a-49ab-9e24-3a48c75527bf" containerName="cleanup" Apr 22 21:34:23.414888 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:23.414771 2580 memory_manager.go:356] "RemoveStaleState removing state" podUID="d05fb114-d12a-49ab-9e24-3a48c75527bf" containerName="cleanup" Apr 22 21:34:23.414888 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:23.414778 2580 memory_manager.go:356] "RemoveStaleState removing state" podUID="d05fb114-d12a-49ab-9e24-3a48c75527bf" containerName="cleanup" Apr 22 21:34:23.417373 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:23.417356 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-f6s65" Apr 22 21:34:23.420340 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:23.420316 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-42bnf\"" Apr 22 21:34:23.428517 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:23.428498 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-f6s65"] Apr 22 21:34:23.538080 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:23.538047 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a1b70b4-b01d-4e8b-8b48-ea2feeec3620-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-f6s65\" (UID: \"6a1b70b4-b01d-4e8b-8b48-ea2feeec3620\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-f6s65" Apr 22 21:34:23.538235 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:23.538088 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmwh6\" (UniqueName: \"kubernetes.io/projected/6a1b70b4-b01d-4e8b-8b48-ea2feeec3620-kube-api-access-tmwh6\") pod \"kuadrant-operator-controller-manager-55c7f4c975-f6s65\" (UID: \"6a1b70b4-b01d-4e8b-8b48-ea2feeec3620\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-f6s65" Apr 22 21:34:23.638693 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:23.638666 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a1b70b4-b01d-4e8b-8b48-ea2feeec3620-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-f6s65\" (UID: \"6a1b70b4-b01d-4e8b-8b48-ea2feeec3620\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-f6s65" Apr 22 21:34:23.638817 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:23.638696 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tmwh6\" (UniqueName: \"kubernetes.io/projected/6a1b70b4-b01d-4e8b-8b48-ea2feeec3620-kube-api-access-tmwh6\") pod \"kuadrant-operator-controller-manager-55c7f4c975-f6s65\" (UID: \"6a1b70b4-b01d-4e8b-8b48-ea2feeec3620\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-f6s65" Apr 22 21:34:23.639042 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:23.639022 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a1b70b4-b01d-4e8b-8b48-ea2feeec3620-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-f6s65\" (UID: \"6a1b70b4-b01d-4e8b-8b48-ea2feeec3620\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-f6s65" Apr 22 21:34:23.647087 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:23.647067 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmwh6\" (UniqueName: \"kubernetes.io/projected/6a1b70b4-b01d-4e8b-8b48-ea2feeec3620-kube-api-access-tmwh6\") pod \"kuadrant-operator-controller-manager-55c7f4c975-f6s65\" (UID: \"6a1b70b4-b01d-4e8b-8b48-ea2feeec3620\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-f6s65" Apr 22 21:34:23.727637 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:23.727583 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-f6s65" Apr 22 21:34:24.065137 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:24.065109 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-f6s65"] Apr 22 21:34:24.066762 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:34:24.066735 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a1b70b4_b01d_4e8b_8b48_ea2feeec3620.slice/crio-750b3abf0b865a411257d1b17627307728e29905f8bc2eeb58005fdd7c34862b WatchSource:0}: Error finding container 750b3abf0b865a411257d1b17627307728e29905f8bc2eeb58005fdd7c34862b: Status 404 returned error can't find the container with id 750b3abf0b865a411257d1b17627307728e29905f8bc2eeb58005fdd7c34862b Apr 22 21:34:24.898597 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:24.898563 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-f6s65" event={"ID":"6a1b70b4-b01d-4e8b-8b48-ea2feeec3620","Type":"ContainerStarted","Data":"6fede64997fbdb94546741cc0ad1637ae30f9ff6fb2ab01310fffe33a8d3ece7"} Apr 22 21:34:24.898597 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:24.898597 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-f6s65" event={"ID":"6a1b70b4-b01d-4e8b-8b48-ea2feeec3620","Type":"ContainerStarted","Data":"750b3abf0b865a411257d1b17627307728e29905f8bc2eeb58005fdd7c34862b"} Apr 22 21:34:24.899045 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:24.898708 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-f6s65" Apr 22 21:34:24.920064 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:24.920015 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-f6s65" podStartSLOduration=1.919997803 podStartE2EDuration="1.919997803s" podCreationTimestamp="2026-04-22 21:34:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-22 21:34:24.918066131 +0000 UTC m=+1505.643962732" watchObservedRunningTime="2026-04-22 21:34:24.919997803 +0000 UTC m=+1505.645894403" Apr 22 21:34:35.904264 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:34:35.904193 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-f6s65" Apr 22 21:39:19.875216 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:39:19.875187 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ngls_b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b/ovn-acl-logging/0.log" Apr 22 21:39:19.889393 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:39:19.889368 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ngls_b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b/ovn-acl-logging/0.log" Apr 22 21:44:06.709102 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:06.709071 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65d8664856-x586c_38f8d29f-76b8-4964-8085-5603aac1059e/manager/0.log" Apr 22 21:44:08.269650 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:08.269619 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-sbwf4_52d77edf-b5ee-4290-a5ed-e7a91ad20470/manager/0.log" Apr 22 21:44:08.371503 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:08.371479 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-g2njn_b1b12a7a-558f-4cb8-8856-0469f16c5555/kuadrant-console-plugin/0.log" Apr 22 21:44:08.488428 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:08.488402 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-8zprt_1aa3b203-7b1e-42c4-8856-c8e6a43e56c3/registry-server/0.log" Apr 22 21:44:08.605044 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:08.604972 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-f6s65_6a1b70b4-b01d-4e8b-8b48-ea2feeec3620/manager/0.log" Apr 22 21:44:08.835805 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:08.835777 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-krpg7_a7570b12-6fd8-4228-9e12-9d9662a671a8/manager/0.log" Apr 22 21:44:09.160823 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:09.160789 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-55cc67557f2z5tc_d7d8ffd7-bf30-44b4-88af-5238fea731b6/istio-proxy/0.log" Apr 22 21:44:09.720189 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:09.720160 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-8594858796-49pkv_39daacd1-808b-4be7-95be-bccd8a0817e7/router/0.log" Apr 22 21:44:10.488901 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:10.488873 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws_0c29ddbf-acab-4c25-9c76-adf84a1ee551/storage-initializer/0.log" Apr 22 21:44:10.495131 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:10.495111 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-b7bws_0c29ddbf-acab-4c25-9c76-adf84a1ee551/main/0.log" Apr 22 21:44:10.596810 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:10.596781 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-6b97b89985-qdp25_af92076a-1f97-4b23-be10-ccda02660902/storage-initializer/0.log" Apr 22 21:44:10.602723 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:10.602701 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-6b97b89985-qdp25_af92076a-1f97-4b23-be10-ccda02660902/main/0.log" Apr 22 21:44:17.254990 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:17.254954 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-txlcq_9efd7cea-0616-4971-9f61-d3b0e3e9c156/global-pull-secret-syncer/0.log" Apr 22 21:44:17.300736 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:17.300706 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7xt7x_72b91440-69fd-4335-a6e6-e8f741ad2da8/konnectivity-agent/0.log" Apr 22 21:44:17.423255 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:17.423224 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-20.ec2.internal_7702f8c6e9277cd7f5964f802e4caebb/haproxy/0.log" Apr 22 21:44:19.903078 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:19.903048 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ngls_b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b/ovn-acl-logging/0.log" Apr 22 21:44:19.914601 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:19.914582 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ngls_b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b/ovn-acl-logging/0.log" Apr 22 21:44:21.674810 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:21.674778 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-sbwf4_52d77edf-b5ee-4290-a5ed-e7a91ad20470/manager/0.log" Apr 22 21:44:21.695972 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:21.695952 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-g2njn_b1b12a7a-558f-4cb8-8856-0469f16c5555/kuadrant-console-plugin/0.log" Apr 22 21:44:21.732409 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:21.732391 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-8zprt_1aa3b203-7b1e-42c4-8856-c8e6a43e56c3/registry-server/0.log" Apr 22 21:44:21.788007 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:21.787981 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-f6s65_6a1b70b4-b01d-4e8b-8b48-ea2feeec3620/manager/0.log" Apr 22 21:44:21.861325 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:21.861300 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-krpg7_a7570b12-6fd8-4228-9e12-9d9662a671a8/manager/0.log" Apr 22 21:44:23.571138 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:23.571110 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7dccd58f55-4p2k6_10bb0162-71a7-451d-bc0c-578af94b5b8a/monitoring-plugin/0.log" Apr 22 21:44:23.663968 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:23.663936 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rcwjz_bcd6d2ac-f291-46f0-b1d1-f15f26c41c79/node-exporter/0.log" Apr 22 21:44:23.679568 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:23.679544 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rcwjz_bcd6d2ac-f291-46f0-b1d1-f15f26c41c79/kube-rbac-proxy/0.log" Apr 22 21:44:23.695657 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:23.695635 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rcwjz_bcd6d2ac-f291-46f0-b1d1-f15f26c41c79/init-textfile/0.log" Apr 22 21:44:23.865269 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:23.865203 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e85fbea5-3c3a-4464-85d0-97d3aa56ea89/prometheus/0.log" Apr 22 21:44:23.886162 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:23.886140 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e85fbea5-3c3a-4464-85d0-97d3aa56ea89/config-reloader/0.log" Apr 22 21:44:23.915892 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:23.915868 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e85fbea5-3c3a-4464-85d0-97d3aa56ea89/thanos-sidecar/0.log" Apr 22 21:44:23.937522 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:23.937501 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e85fbea5-3c3a-4464-85d0-97d3aa56ea89/kube-rbac-proxy-web/0.log" Apr 22 21:44:23.960644 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:23.960619 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e85fbea5-3c3a-4464-85d0-97d3aa56ea89/kube-rbac-proxy/0.log" Apr 22 21:44:24.015827 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:24.015805 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e85fbea5-3c3a-4464-85d0-97d3aa56ea89/kube-rbac-proxy-thanos/0.log" Apr 22 21:44:24.036820 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:24.036802 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e85fbea5-3c3a-4464-85d0-97d3aa56ea89/init-config-reloader/0.log" Apr 22 21:44:24.153493 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:24.153467 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-57cf98b594-gsv7g_a23ef821-2192-402d-91ae-c95fc61fad45/prometheus-operator-admission-webhook/0.log" Apr 22 21:44:26.109720 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.109693 2580 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ddjjb/perf-node-gather-daemonset-mprsw"] Apr 22 21:44:26.110087 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.110048 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d05fb114-d12a-49ab-9e24-3a48c75527bf" containerName="cleanup" Apr 22 21:44:26.110087 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.110060 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d05fb114-d12a-49ab-9e24-3a48c75527bf" containerName="cleanup" Apr 22 21:44:26.113129 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.113110 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ddjjb/perf-node-gather-daemonset-mprsw" Apr 22 21:44:26.115918 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.115900 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ddjjb\"/\"kube-root-ca.crt\"" Apr 22 21:44:26.116034 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.115955 2580 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ddjjb\"/\"openshift-service-ca.crt\"" Apr 22 21:44:26.117167 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.117148 2580 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ddjjb\"/\"default-dockercfg-kv5l8\"" Apr 22 21:44:26.123767 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.123746 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ddjjb/perf-node-gather-daemonset-mprsw"] Apr 22 21:44:26.150283 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.150265 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnkgp\" (UniqueName: \"kubernetes.io/projected/6ac79121-b7d7-4dba-93cc-55329008881b-kube-api-access-dnkgp\") pod \"perf-node-gather-daemonset-mprsw\" (UID: \"6ac79121-b7d7-4dba-93cc-55329008881b\") " pod="openshift-must-gather-ddjjb/perf-node-gather-daemonset-mprsw" Apr 22 21:44:26.150393 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.150295 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ac79121-b7d7-4dba-93cc-55329008881b-sys\") pod \"perf-node-gather-daemonset-mprsw\" (UID: \"6ac79121-b7d7-4dba-93cc-55329008881b\") " pod="openshift-must-gather-ddjjb/perf-node-gather-daemonset-mprsw" Apr 22 21:44:26.150393 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.150322 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6ac79121-b7d7-4dba-93cc-55329008881b-podres\") pod \"perf-node-gather-daemonset-mprsw\" (UID: \"6ac79121-b7d7-4dba-93cc-55329008881b\") " pod="openshift-must-gather-ddjjb/perf-node-gather-daemonset-mprsw" Apr 22 21:44:26.150478 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.150456 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6ac79121-b7d7-4dba-93cc-55329008881b-proc\") pod \"perf-node-gather-daemonset-mprsw\" (UID: \"6ac79121-b7d7-4dba-93cc-55329008881b\") " pod="openshift-must-gather-ddjjb/perf-node-gather-daemonset-mprsw" Apr 22 21:44:26.150478 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.150471 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6ac79121-b7d7-4dba-93cc-55329008881b-lib-modules\") pod \"perf-node-gather-daemonset-mprsw\" (UID: \"6ac79121-b7d7-4dba-93cc-55329008881b\") " pod="openshift-must-gather-ddjjb/perf-node-gather-daemonset-mprsw" Apr 22 21:44:26.251447 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.251416 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dnkgp\" (UniqueName: \"kubernetes.io/projected/6ac79121-b7d7-4dba-93cc-55329008881b-kube-api-access-dnkgp\") pod \"perf-node-gather-daemonset-mprsw\" (UID: \"6ac79121-b7d7-4dba-93cc-55329008881b\") " pod="openshift-must-gather-ddjjb/perf-node-gather-daemonset-mprsw" Apr 22 21:44:26.251447 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.251451 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ac79121-b7d7-4dba-93cc-55329008881b-sys\") pod \"perf-node-gather-daemonset-mprsw\" (UID: \"6ac79121-b7d7-4dba-93cc-55329008881b\") " pod="openshift-must-gather-ddjjb/perf-node-gather-daemonset-mprsw" Apr 22 21:44:26.251660 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.251530 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ac79121-b7d7-4dba-93cc-55329008881b-sys\") pod \"perf-node-gather-daemonset-mprsw\" (UID: \"6ac79121-b7d7-4dba-93cc-55329008881b\") " pod="openshift-must-gather-ddjjb/perf-node-gather-daemonset-mprsw" Apr 22 21:44:26.251660 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.251576 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6ac79121-b7d7-4dba-93cc-55329008881b-podres\") pod \"perf-node-gather-daemonset-mprsw\" (UID: \"6ac79121-b7d7-4dba-93cc-55329008881b\") " pod="openshift-must-gather-ddjjb/perf-node-gather-daemonset-mprsw" Apr 22 21:44:26.251732 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.251658 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6ac79121-b7d7-4dba-93cc-55329008881b-proc\") pod \"perf-node-gather-daemonset-mprsw\" (UID: \"6ac79121-b7d7-4dba-93cc-55329008881b\") " pod="openshift-must-gather-ddjjb/perf-node-gather-daemonset-mprsw" Apr 22 21:44:26.251732 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.251678 2580 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6ac79121-b7d7-4dba-93cc-55329008881b-lib-modules\") pod \"perf-node-gather-daemonset-mprsw\" (UID: \"6ac79121-b7d7-4dba-93cc-55329008881b\") " pod="openshift-must-gather-ddjjb/perf-node-gather-daemonset-mprsw" Apr 22 21:44:26.251796 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.251754 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6ac79121-b7d7-4dba-93cc-55329008881b-podres\") pod \"perf-node-gather-daemonset-mprsw\" (UID: \"6ac79121-b7d7-4dba-93cc-55329008881b\") " pod="openshift-must-gather-ddjjb/perf-node-gather-daemonset-mprsw" Apr 22 21:44:26.251796 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.251764 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6ac79121-b7d7-4dba-93cc-55329008881b-proc\") pod \"perf-node-gather-daemonset-mprsw\" (UID: \"6ac79121-b7d7-4dba-93cc-55329008881b\") " pod="openshift-must-gather-ddjjb/perf-node-gather-daemonset-mprsw" Apr 22 21:44:26.251883 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.251813 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6ac79121-b7d7-4dba-93cc-55329008881b-lib-modules\") pod \"perf-node-gather-daemonset-mprsw\" (UID: \"6ac79121-b7d7-4dba-93cc-55329008881b\") " pod="openshift-must-gather-ddjjb/perf-node-gather-daemonset-mprsw" Apr 22 21:44:26.259348 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.259329 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnkgp\" (UniqueName: \"kubernetes.io/projected/6ac79121-b7d7-4dba-93cc-55329008881b-kube-api-access-dnkgp\") pod \"perf-node-gather-daemonset-mprsw\" (UID: \"6ac79121-b7d7-4dba-93cc-55329008881b\") " pod="openshift-must-gather-ddjjb/perf-node-gather-daemonset-mprsw" Apr 22 21:44:26.424634 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.424596 2580 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ddjjb/perf-node-gather-daemonset-mprsw" Apr 22 21:44:26.548896 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.548865 2580 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ddjjb/perf-node-gather-daemonset-mprsw"] Apr 22 21:44:26.552164 ip-10-0-136-20 kubenswrapper[2580]: W0422 21:44:26.552134 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6ac79121_b7d7_4dba_93cc_55329008881b.slice/crio-c20b50ae35a2b32b924a254fffd63f246cb997edfc31575f5bad03c6b84c1fa6 WatchSource:0}: Error finding container c20b50ae35a2b32b924a254fffd63f246cb997edfc31575f5bad03c6b84c1fa6: Status 404 returned error can't find the container with id c20b50ae35a2b32b924a254fffd63f246cb997edfc31575f5bad03c6b84c1fa6 Apr 22 21:44:26.553663 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.553646 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 22 21:44:26.950709 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.950623 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ddjjb/perf-node-gather-daemonset-mprsw" event={"ID":"6ac79121-b7d7-4dba-93cc-55329008881b","Type":"ContainerStarted","Data":"fd1fd844320feb04faf00d4239a816491d00894b8512de9688b0d99c89f2e0e7"} Apr 22 21:44:26.950709 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.950661 2580 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ddjjb/perf-node-gather-daemonset-mprsw" event={"ID":"6ac79121-b7d7-4dba-93cc-55329008881b","Type":"ContainerStarted","Data":"c20b50ae35a2b32b924a254fffd63f246cb997edfc31575f5bad03c6b84c1fa6"} Apr 22 21:44:26.950955 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.950767 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ddjjb/perf-node-gather-daemonset-mprsw" Apr 22 21:44:26.968029 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:26.967988 2580 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-ddjjb/perf-node-gather-daemonset-mprsw" podStartSLOduration=0.967975065 podStartE2EDuration="967.975065ms" podCreationTimestamp="2026-04-22 21:44:26 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-22 21:44:26.965734151 +0000 UTC m=+2107.691630735" watchObservedRunningTime="2026-04-22 21:44:26.967975065 +0000 UTC m=+2107.693871659" Apr 22 21:44:27.053702 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:27.053676 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-7c6cbb6c87-jhsqb_aa9a495b-78dc-4501-9952-5b4900126489/volume-data-source-validator/0.log" Apr 22 21:44:27.892599 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:27.892555 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9klhr_e92ccf2d-8b03-40ec-81d7-e0752b8f6c78/dns/0.log" Apr 22 21:44:27.909050 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:27.909029 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9klhr_e92ccf2d-8b03-40ec-81d7-e0752b8f6c78/kube-rbac-proxy/0.log" Apr 22 21:44:28.026720 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:28.026686 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-49dpb_9e4ebb4a-2ec6-41ad-ac25-a5d1d100711f/dns-node-resolver/0.log" Apr 22 21:44:28.492698 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:28.492670 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7db9c69bcc-wxqq6_0f2eedec-34a3-43cc-8586-ec247b1fd8bc/registry/0.log" Apr 22 21:44:28.550056 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:28.550032 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wkk24_13fe92d8-fa95-45da-8e47-5b754f77385c/node-ca/0.log" Apr 22 21:44:29.265628 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:29.265597 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-55cc67557f2z5tc_d7d8ffd7-bf30-44b4-88af-5238fea731b6/istio-proxy/0.log" Apr 22 21:44:29.539881 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:29.539783 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-8594858796-49pkv_39daacd1-808b-4be7-95be-bccd8a0817e7/router/0.log" Apr 22 21:44:30.051793 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:30.051764 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rp9w5_9ef207b1-3445-413b-8855-1e0c977efcf2/serve-healthcheck-canary/0.log" Apr 22 21:44:30.526798 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:30.526766 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-585dfdc468-gwzzw_7d912a63-fac1-4f77-b588-e42de7a51799/insights-operator/0.log" Apr 22 21:44:30.527270 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:30.527088 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-585dfdc468-gwzzw_7d912a63-fac1-4f77-b588-e42de7a51799/insights-operator/1.log" Apr 22 21:44:30.546492 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:30.546469 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5d2jm_f1048e6f-786c-4048-a184-b4e8454b570d/kube-rbac-proxy/0.log" Apr 22 21:44:30.564427 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:30.564405 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5d2jm_f1048e6f-786c-4048-a184-b4e8454b570d/exporter/0.log" Apr 22 21:44:30.581560 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:30.581536 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5d2jm_f1048e6f-786c-4048-a184-b4e8454b570d/extractor/0.log" Apr 22 21:44:32.783770 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:32.783736 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65d8664856-x586c_38f8d29f-76b8-4964-8085-5603aac1059e/manager/0.log" Apr 22 21:44:32.964086 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:32.964061 2580 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ddjjb/perf-node-gather-daemonset-mprsw" Apr 22 21:44:33.919996 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:33.919969 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5db7bf5949-jf9h6_522c73e8-3953-4540-b97b-156bfea8e30a/manager/0.log" Apr 22 21:44:38.617392 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:38.617350 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6769c5d45-j45pb_dcdefc0e-dace-496e-bccf-de663633bec9/kube-storage-version-migrator-operator/1.log" Apr 22 21:44:38.618451 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:38.618432 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6769c5d45-j45pb_dcdefc0e-dace-496e-bccf-de663633bec9/kube-storage-version-migrator-operator/0.log" Apr 22 21:44:39.651909 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:39.651876 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qvlsp_efc4b2c1-b3de-4aa6-abe3-c37e3517897f/kube-multus-additional-cni-plugins/0.log" Apr 22 21:44:39.667613 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:39.667582 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qvlsp_efc4b2c1-b3de-4aa6-abe3-c37e3517897f/egress-router-binary-copy/0.log" Apr 22 21:44:39.684019 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:39.684001 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qvlsp_efc4b2c1-b3de-4aa6-abe3-c37e3517897f/cni-plugins/0.log" Apr 22 21:44:39.701478 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:39.701457 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qvlsp_efc4b2c1-b3de-4aa6-abe3-c37e3517897f/bond-cni-plugin/0.log" Apr 22 21:44:39.717781 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:39.717757 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qvlsp_efc4b2c1-b3de-4aa6-abe3-c37e3517897f/routeoverride-cni/0.log" Apr 22 21:44:39.733147 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:39.733131 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qvlsp_efc4b2c1-b3de-4aa6-abe3-c37e3517897f/whereabouts-cni-bincopy/0.log" Apr 22 21:44:39.750306 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:39.750288 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qvlsp_efc4b2c1-b3de-4aa6-abe3-c37e3517897f/whereabouts-cni/0.log" Apr 22 21:44:39.949074 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:39.949007 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-zwlx7_408567c2-c6f7-4edd-bea3-ff7695a4d06e/kube-multus/0.log" Apr 22 21:44:39.964271 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:39.964247 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fx8k2_843da596-cc50-4a3e-817c-4ee63a89c94a/network-metrics-daemon/0.log" Apr 22 21:44:39.979128 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:39.979079 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fx8k2_843da596-cc50-4a3e-817c-4ee63a89c94a/kube-rbac-proxy/0.log" Apr 22 21:44:40.841129 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:40.841100 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ngls_b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b/ovn-controller/0.log" Apr 22 21:44:40.855230 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:40.855208 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ngls_b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b/ovn-acl-logging/0.log" Apr 22 21:44:40.864182 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:40.864157 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ngls_b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b/ovn-acl-logging/1.log" Apr 22 21:44:40.882310 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:40.882247 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ngls_b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b/kube-rbac-proxy-node/0.log" Apr 22 21:44:40.899426 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:40.899398 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ngls_b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b/kube-rbac-proxy-ovn-metrics/0.log" Apr 22 21:44:40.913864 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:40.913831 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ngls_b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b/northd/0.log" Apr 22 21:44:40.930496 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:40.930473 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ngls_b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b/nbdb/0.log" Apr 22 21:44:40.946247 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:40.946224 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ngls_b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b/sbdb/0.log" Apr 22 21:44:41.038102 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:41.038068 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8ngls_b619ae54-70a8-4b9f-b5ff-4008c1fdbc9b/ovnkube-controller/0.log" Apr 22 21:44:42.668989 ip-10-0-136-20 kubenswrapper[2580]: I0422 21:44:42.668954 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-lp7mc_516cb834-5ee4-4971-80a9-149fd935fc80/network-check-target-container/0.log"