May 21 00:54:40.975328 ip-10-0-138-101 systemd[1]: Starting Kubernetes Kubelet... May 21 00:54:41.460810 ip-10-0-138-101 kubenswrapper[2549]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 21 00:54:41.460810 ip-10-0-138-101 kubenswrapper[2549]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. May 21 00:54:41.460810 ip-10-0-138-101 kubenswrapper[2549]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 21 00:54:41.460810 ip-10-0-138-101 kubenswrapper[2549]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. May 21 00:54:41.460810 ip-10-0-138-101 kubenswrapper[2549]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 21 00:54:41.462446 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.462370 2549 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" May 21 00:54:41.468442 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468418 2549 feature_gate.go:328] unrecognized feature gate: NewOLM May 21 00:54:41.468442 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468438 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 21 00:54:41.468442 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468445 2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 21 00:54:41.468442 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468449 2549 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 21 00:54:41.468691 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468453 2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 21 00:54:41.468691 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468457 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 21 00:54:41.468691 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468461 2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 21 00:54:41.468691 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468465 2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 21 00:54:41.468691 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468468 2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 21 00:54:41.468691 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468472 2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 21 00:54:41.468691 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468475 2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 21 00:54:41.468691 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468479 2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 21 00:54:41.468691 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468482 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 21 00:54:41.468691 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468486 2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 21 00:54:41.468691 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468490 2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 21 00:54:41.468691 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468506 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 21 00:54:41.468691 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468511 2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 21 00:54:41.468691 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468514 2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 21 00:54:41.468691 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468518 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 21 00:54:41.468691 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468522 2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 21 00:54:41.468691 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468526 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 21 00:54:41.468691 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468530 2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 21 00:54:41.468691 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468534 2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 21 00:54:41.468691 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468538 2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 21 00:54:41.469564 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468542 2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 21 00:54:41.469564 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468546 2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 21 00:54:41.469564 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468551 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 21 00:54:41.469564 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468555 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 21 00:54:41.469564 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468559 2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 21 00:54:41.469564 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468563 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 21 00:54:41.469564 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468567 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 21 00:54:41.469564 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468572 2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 21 00:54:41.469564 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468576 2549 feature_gate.go:328] unrecognized feature gate: SignatureStores May 21 00:54:41.469564 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468581 2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 21 00:54:41.469564 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468584 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 21 00:54:41.469564 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468589 2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 21 00:54:41.469564 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468593 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 21 00:54:41.469564 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468598 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 21 00:54:41.469564 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468602 2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 21 00:54:41.469564 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468608 2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 21 00:54:41.469564 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468613 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 21 00:54:41.469564 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468617 2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 21 00:54:41.469564 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468622 2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 21 00:54:41.469564 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468626 2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 21 00:54:41.470300 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468630 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 21 00:54:41.470300 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468643 2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 21 00:54:41.470300 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468650 2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 21 00:54:41.470300 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468657 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 21 00:54:41.470300 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468661 2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 21 00:54:41.470300 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468665 2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 21 00:54:41.470300 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468670 2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 21 00:54:41.470300 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468674 2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 21 00:54:41.470300 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468679 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 21 00:54:41.470300 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468683 2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 21 00:54:41.470300 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468687 2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 21 00:54:41.470300 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468691 2549 feature_gate.go:328] unrecognized feature gate: PinnedImages May 21 00:54:41.470300 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468695 2549 feature_gate.go:328] unrecognized feature gate: DualReplica May 21 00:54:41.470300 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468699 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 21 00:54:41.470300 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468703 2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 21 00:54:41.470300 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468707 2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 21 00:54:41.470300 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468711 2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 21 00:54:41.470300 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468715 2549 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 21 00:54:41.470300 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468719 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 21 00:54:41.470758 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468723 2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 21 00:54:41.470758 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468727 2549 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 21 00:54:41.470758 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468732 2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 21 00:54:41.470758 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468736 2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 21 00:54:41.470758 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468740 2549 feature_gate.go:328] unrecognized feature gate: Example2 May 21 00:54:41.470758 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468744 2549 feature_gate.go:328] unrecognized feature gate: Example May 21 00:54:41.470758 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468750 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 21 00:54:41.470758 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468756 2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 21 00:54:41.470758 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468765 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 21 00:54:41.470758 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468770 2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 21 00:54:41.470758 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468775 2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 21 00:54:41.470758 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468780 2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 21 00:54:41.470758 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468784 2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 21 00:54:41.470758 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468788 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 21 00:54:41.470758 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468793 2549 feature_gate.go:328] unrecognized feature gate: OVNObservability May 21 00:54:41.470758 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468805 2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 21 00:54:41.470758 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468810 2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 21 00:54:41.470758 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468814 2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 21 00:54:41.470758 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468818 2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 21 00:54:41.470758 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468822 2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 21 00:54:41.471271 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468827 2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 21 00:54:41.471271 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468831 2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 21 00:54:41.471271 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.468835 2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 21 00:54:41.471271 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469509 2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 21 00:54:41.471271 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469519 2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 21 00:54:41.471271 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469525 2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 21 00:54:41.471271 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469530 2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 21 00:54:41.471271 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469534 2549 feature_gate.go:328] unrecognized feature gate: DualReplica May 21 00:54:41.471271 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469538 2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 21 00:54:41.471271 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469546 2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 21 00:54:41.471271 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469552 2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 21 00:54:41.471271 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469557 2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 21 00:54:41.471271 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469561 2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 21 00:54:41.471271 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469565 2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 21 00:54:41.471271 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469570 2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 21 00:54:41.471271 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469574 2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 21 00:54:41.471271 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469578 2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 21 00:54:41.471271 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469583 2549 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 21 00:54:41.471271 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469588 2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 21 00:54:41.472034 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469592 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 21 00:54:41.472034 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469597 2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 21 00:54:41.472034 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469602 2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 21 00:54:41.472034 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469606 2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 21 00:54:41.472034 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469611 2549 feature_gate.go:328] unrecognized feature gate: OVNObservability May 21 00:54:41.472034 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469615 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 21 00:54:41.472034 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469619 2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 21 00:54:41.472034 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469623 2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 21 00:54:41.472034 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469635 2549 feature_gate.go:328] unrecognized feature gate: Example May 21 00:54:41.472034 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469640 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 21 00:54:41.472034 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469645 2549 feature_gate.go:328] unrecognized feature gate: PinnedImages May 21 00:54:41.472034 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469649 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 21 00:54:41.472034 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469653 2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 21 00:54:41.472034 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469657 2549 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 21 00:54:41.472034 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469661 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 21 00:54:41.472034 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469665 2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 21 00:54:41.472034 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469669 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 21 00:54:41.472034 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469673 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 21 00:54:41.472034 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469678 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 21 00:54:41.472034 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469683 2549 feature_gate.go:328] unrecognized feature gate: Example2 May 21 00:54:41.472920 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469688 2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 21 00:54:41.472920 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469692 2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 21 00:54:41.472920 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469696 2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 21 00:54:41.472920 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469711 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 21 00:54:41.472920 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469715 2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 21 00:54:41.472920 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469719 2549 feature_gate.go:328] unrecognized feature gate: NewOLM May 21 00:54:41.472920 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469723 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 21 00:54:41.472920 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469727 2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 21 00:54:41.472920 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469732 2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 21 00:54:41.472920 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469736 2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 21 00:54:41.472920 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469740 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 21 00:54:41.472920 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469746 2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 21 00:54:41.472920 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469750 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 21 00:54:41.472920 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469755 2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 21 00:54:41.472920 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469759 2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 21 00:54:41.472920 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469763 2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 21 00:54:41.472920 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469767 2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 21 00:54:41.472920 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469771 2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 21 00:54:41.472920 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469775 2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 21 00:54:41.472920 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469780 2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 21 00:54:41.473567 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469784 2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 21 00:54:41.473567 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469796 2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 21 00:54:41.473567 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469803 2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 21 00:54:41.473567 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469810 2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 21 00:54:41.473567 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469814 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 21 00:54:41.473567 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469818 2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 21 00:54:41.473567 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469822 2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 21 00:54:41.473567 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469826 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 21 00:54:41.473567 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469830 2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 21 00:54:41.473567 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469834 2549 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 21 00:54:41.473567 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469838 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 21 00:54:41.473567 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469842 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 21 00:54:41.473567 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469846 2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 21 00:54:41.473567 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469850 2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 21 00:54:41.473567 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469854 2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 21 00:54:41.473567 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469858 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 21 00:54:41.473567 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469862 2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 21 00:54:41.473567 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469866 2549 feature_gate.go:328] unrecognized feature gate: SignatureStores May 21 00:54:41.473567 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469870 2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 21 00:54:41.474240 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469874 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 21 00:54:41.474240 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469880 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 21 00:54:41.474240 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469884 2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 21 00:54:41.474240 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469888 2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 21 00:54:41.474240 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469893 2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 21 00:54:41.474240 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469897 2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 21 00:54:41.474240 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469901 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 21 00:54:41.474240 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469906 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 21 00:54:41.474240 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469910 2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 21 00:54:41.474240 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469914 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 21 00:54:41.474240 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.469918 2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 21 00:54:41.474240 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471345 2549 flags.go:64] FLAG: --address="0.0.0.0" May 21 00:54:41.474240 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471361 2549 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" May 21 00:54:41.474240 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471375 2549 flags.go:64] FLAG: --anonymous-auth="true" May 21 00:54:41.474240 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471381 2549 flags.go:64] FLAG: --application-metrics-count-limit="100" May 21 00:54:41.474240 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471397 2549 flags.go:64] FLAG: --authentication-token-webhook="false" May 21 00:54:41.474240 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471403 2549 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" May 21 00:54:41.474240 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471409 2549 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" May 21 00:54:41.474240 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471416 2549 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" May 21 00:54:41.474240 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471421 2549 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" May 21 00:54:41.474240 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471426 2549 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" May 21 00:54:41.474776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471431 2549 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" May 21 00:54:41.474776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471437 2549 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" May 21 00:54:41.474776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471442 2549 flags.go:64] FLAG: --cgroup-driver="cgroupfs" May 21 00:54:41.474776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471447 2549 flags.go:64] FLAG: --cgroup-root="" May 21 00:54:41.474776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471452 2549 flags.go:64] FLAG: --cgroups-per-qos="true" May 21 00:54:41.474776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471457 2549 flags.go:64] FLAG: --client-ca-file="" May 21 00:54:41.474776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471461 2549 flags.go:64] FLAG: --cloud-config="" May 21 00:54:41.474776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471466 2549 flags.go:64] FLAG: --cloud-provider="external" May 21 00:54:41.474776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471471 2549 flags.go:64] FLAG: --cluster-dns="[]" May 21 00:54:41.474776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471481 2549 flags.go:64] FLAG: --cluster-domain="" May 21 00:54:41.474776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471487 2549 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" May 21 00:54:41.474776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471492 2549 flags.go:64] FLAG: --config-dir="" May 21 00:54:41.474776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471497 2549 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" May 21 00:54:41.474776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471502 2549 flags.go:64] FLAG: --container-log-max-files="5" May 21 00:54:41.474776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471509 2549 flags.go:64] FLAG: --container-log-max-size="10Mi" May 21 00:54:41.474776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471515 2549 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" May 21 00:54:41.474776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471520 2549 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" May 21 00:54:41.474776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471526 2549 flags.go:64] FLAG: --containerd-namespace="k8s.io" May 21 00:54:41.474776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471531 2549 flags.go:64] FLAG: --contention-profiling="false" May 21 00:54:41.474776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471535 2549 flags.go:64] FLAG: --cpu-cfs-quota="true" May 21 00:54:41.474776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471540 2549 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" May 21 00:54:41.474776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471545 2549 flags.go:64] FLAG: --cpu-manager-policy="none" May 21 00:54:41.474776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471550 2549 flags.go:64] FLAG: --cpu-manager-policy-options="" May 21 00:54:41.474776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471575 2549 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" May 21 00:54:41.474776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471582 2549 flags.go:64] FLAG: --enable-controller-attach-detach="true" May 21 00:54:41.475500 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471587 2549 flags.go:64] FLAG: --enable-debugging-handlers="true" May 21 00:54:41.475500 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471592 2549 flags.go:64] FLAG: --enable-load-reader="false" May 21 00:54:41.475500 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471605 2549 flags.go:64] FLAG: --enable-server="true" May 21 00:54:41.475500 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471611 2549 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" May 21 00:54:41.475500 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471620 2549 flags.go:64] FLAG: --event-burst="100" May 21 00:54:41.475500 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471633 2549 flags.go:64] FLAG: --event-qps="50" May 21 00:54:41.475500 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471639 2549 flags.go:64] FLAG: --event-storage-age-limit="default=0" May 21 00:54:41.475500 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471644 2549 flags.go:64] FLAG: --event-storage-event-limit="default=0" May 21 00:54:41.475500 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471649 2549 flags.go:64] FLAG: --eviction-hard="" May 21 00:54:41.475500 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471655 2549 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" May 21 00:54:41.475500 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471660 2549 flags.go:64] FLAG: --eviction-minimum-reclaim="" May 21 00:54:41.475500 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471665 2549 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" May 21 00:54:41.475500 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471669 2549 flags.go:64] FLAG: --eviction-soft="" May 21 00:54:41.475500 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471675 2549 flags.go:64] FLAG: --eviction-soft-grace-period="" May 21 00:54:41.475500 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471680 2549 flags.go:64] FLAG: --exit-on-lock-contention="false" May 21 00:54:41.475500 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471685 2549 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" May 21 00:54:41.475500 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471690 2549 flags.go:64] FLAG: --experimental-mounter-path="" May 21 00:54:41.475500 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471695 2549 flags.go:64] FLAG: --fail-cgroupv1="false" May 21 00:54:41.475500 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471699 2549 flags.go:64] FLAG: --fail-swap-on="true" May 21 00:54:41.475500 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471704 2549 flags.go:64] FLAG: --feature-gates="" May 21 00:54:41.475500 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471710 2549 flags.go:64] FLAG: --file-check-frequency="20s" May 21 00:54:41.475500 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471714 2549 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" May 21 00:54:41.475500 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471720 2549 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" May 21 00:54:41.475500 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471725 2549 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" May 21 00:54:41.475500 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471730 2549 flags.go:64] FLAG: --healthz-port="10248" May 21 00:54:41.475500 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471735 2549 flags.go:64] FLAG: --help="false" May 21 00:54:41.476181 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471740 2549 flags.go:64] FLAG: --hostname-override="ip-10-0-138-101.ec2.internal" May 21 00:54:41.476181 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471745 2549 flags.go:64] FLAG: --housekeeping-interval="10s" May 21 00:54:41.476181 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471750 2549 flags.go:64] FLAG: --http-check-frequency="20s" May 21 00:54:41.476181 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471755 2549 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" May 21 00:54:41.476181 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471761 2549 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" May 21 00:54:41.476181 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471767 2549 flags.go:64] FLAG: --image-gc-high-threshold="85" May 21 00:54:41.476181 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471771 2549 flags.go:64] FLAG: --image-gc-low-threshold="80" May 21 00:54:41.476181 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471776 2549 flags.go:64] FLAG: --image-service-endpoint="" May 21 00:54:41.476181 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471781 2549 flags.go:64] FLAG: --kernel-memcg-notification="false" May 21 00:54:41.476181 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471794 2549 flags.go:64] FLAG: --kube-api-burst="100" May 21 00:54:41.476181 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471799 2549 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" May 21 00:54:41.476181 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471806 2549 flags.go:64] FLAG: --kube-api-qps="50" May 21 00:54:41.476181 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471810 2549 flags.go:64] FLAG: --kube-reserved="" May 21 00:54:41.476181 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471815 2549 flags.go:64] FLAG: --kube-reserved-cgroup="" May 21 00:54:41.476181 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471820 2549 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" May 21 00:54:41.476181 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471825 2549 flags.go:64] FLAG: --kubelet-cgroups="" May 21 00:54:41.476181 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471830 2549 flags.go:64] FLAG: --local-storage-capacity-isolation="true" May 21 00:54:41.476181 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471834 2549 flags.go:64] FLAG: --lock-file="" May 21 00:54:41.476181 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471839 2549 flags.go:64] FLAG: --log-cadvisor-usage="false" May 21 00:54:41.476181 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471844 2549 flags.go:64] FLAG: --log-flush-frequency="5s" May 21 00:54:41.476181 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471849 2549 flags.go:64] FLAG: --log-json-info-buffer-size="0" May 21 00:54:41.476181 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471857 2549 flags.go:64] FLAG: --log-json-split-stream="false" May 21 00:54:41.476181 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471862 2549 flags.go:64] FLAG: --log-text-info-buffer-size="0" May 21 00:54:41.476727 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471867 2549 flags.go:64] FLAG: --log-text-split-stream="false" May 21 00:54:41.476727 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471871 2549 flags.go:64] FLAG: --logging-format="text" May 21 00:54:41.476727 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471876 2549 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" May 21 00:54:41.476727 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471881 2549 flags.go:64] FLAG: --make-iptables-util-chains="true" May 21 00:54:41.476727 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471886 2549 flags.go:64] FLAG: --manifest-url="" May 21 00:54:41.476727 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471891 2549 flags.go:64] FLAG: --manifest-url-header="" May 21 00:54:41.476727 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471898 2549 flags.go:64] FLAG: --max-housekeeping-interval="15s" May 21 00:54:41.476727 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471903 2549 flags.go:64] FLAG: --max-open-files="1000000" May 21 00:54:41.476727 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471909 2549 flags.go:64] FLAG: --max-pods="110" May 21 00:54:41.476727 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471914 2549 flags.go:64] FLAG: --maximum-dead-containers="-1" May 21 00:54:41.476727 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471919 2549 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" May 21 00:54:41.476727 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471924 2549 flags.go:64] FLAG: --memory-manager-policy="None" May 21 00:54:41.476727 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471929 2549 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" May 21 00:54:41.476727 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471933 2549 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" May 21 00:54:41.476727 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471938 2549 flags.go:64] FLAG: --node-ip="0.0.0.0" May 21 00:54:41.476727 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471943 2549 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" May 21 00:54:41.476727 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471954 2549 flags.go:64] FLAG: --node-status-max-images="50" May 21 00:54:41.476727 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471959 2549 flags.go:64] FLAG: --node-status-update-frequency="10s" May 21 00:54:41.476727 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471963 2549 flags.go:64] FLAG: --oom-score-adj="-999" May 21 00:54:41.476727 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471977 2549 flags.go:64] FLAG: --pod-cidr="" May 21 00:54:41.476727 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471982 2549 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc" May 21 00:54:41.476727 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471990 2549 flags.go:64] FLAG: --pod-manifest-path="" May 21 00:54:41.476727 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.471995 2549 flags.go:64] FLAG: --pod-max-pids="-1" May 21 00:54:41.476727 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472000 2549 flags.go:64] FLAG: --pods-per-core="0" May 21 00:54:41.477338 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472005 2549 flags.go:64] FLAG: --port="10250" May 21 00:54:41.477338 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472010 2549 flags.go:64] FLAG: --protect-kernel-defaults="false" May 21 00:54:41.477338 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472015 2549 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07c4a1b2940724ea6" May 21 00:54:41.477338 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472020 2549 flags.go:64] FLAG: --qos-reserved="" May 21 00:54:41.477338 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472025 2549 flags.go:64] FLAG: --read-only-port="10255" May 21 00:54:41.477338 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472030 2549 flags.go:64] FLAG: --register-node="true" May 21 00:54:41.477338 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472035 2549 flags.go:64] FLAG: --register-schedulable="true" May 21 00:54:41.477338 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472040 2549 flags.go:64] FLAG: --register-with-taints="" May 21 00:54:41.477338 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472045 2549 flags.go:64] FLAG: --registry-burst="10" May 21 00:54:41.477338 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472050 2549 flags.go:64] FLAG: --registry-qps="5" May 21 00:54:41.477338 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472054 2549 flags.go:64] FLAG: --reserved-cpus="" May 21 00:54:41.477338 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472075 2549 flags.go:64] FLAG: --reserved-memory="" May 21 00:54:41.477338 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472081 2549 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" May 21 00:54:41.477338 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472086 2549 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" May 21 00:54:41.477338 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472091 2549 flags.go:64] FLAG: --rotate-certificates="false" May 21 00:54:41.477338 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472096 2549 flags.go:64] FLAG: --rotate-server-certificates="false" May 21 00:54:41.477338 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472100 2549 flags.go:64] FLAG: --runonce="false" May 21 00:54:41.477338 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472105 2549 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" May 21 00:54:41.477338 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472110 2549 flags.go:64] FLAG: --runtime-request-timeout="2m0s" May 21 00:54:41.477338 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472114 2549 flags.go:64] FLAG: --seccomp-default="false" May 21 00:54:41.477338 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472119 2549 flags.go:64] FLAG: --serialize-image-pulls="true" May 21 00:54:41.477338 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472123 2549 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" May 21 00:54:41.477338 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472129 2549 flags.go:64] FLAG: --storage-driver-db="cadvisor" May 21 00:54:41.477338 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472133 2549 flags.go:64] FLAG: --storage-driver-host="localhost:8086" May 21 00:54:41.477338 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472138 2549 flags.go:64] FLAG: --storage-driver-password="root" May 21 00:54:41.477338 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472143 2549 flags.go:64] FLAG: --storage-driver-secure="false" May 21 00:54:41.478033 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472148 2549 flags.go:64] FLAG: --storage-driver-table="stats" May 21 00:54:41.478033 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472153 2549 flags.go:64] FLAG: --storage-driver-user="root" May 21 00:54:41.478033 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472168 2549 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" May 21 00:54:41.478033 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472174 2549 flags.go:64] FLAG: --sync-frequency="1m0s" May 21 00:54:41.478033 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472179 2549 flags.go:64] FLAG: --system-cgroups="" May 21 00:54:41.478033 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472184 2549 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" May 21 00:54:41.478033 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472193 2549 flags.go:64] FLAG: --system-reserved-cgroup="" May 21 00:54:41.478033 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472198 2549 flags.go:64] FLAG: --tls-cert-file="" May 21 00:54:41.478033 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472202 2549 flags.go:64] FLAG: --tls-cipher-suites="[]" May 21 00:54:41.478033 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472213 2549 flags.go:64] FLAG: --tls-min-version="" May 21 00:54:41.478033 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472219 2549 flags.go:64] FLAG: --tls-private-key-file="" May 21 00:54:41.478033 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472224 2549 flags.go:64] FLAG: --topology-manager-policy="none" May 21 00:54:41.478033 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472229 2549 flags.go:64] FLAG: --topology-manager-policy-options="" May 21 00:54:41.478033 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472234 2549 flags.go:64] FLAG: --topology-manager-scope="container" May 21 00:54:41.478033 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472239 2549 flags.go:64] FLAG: --v="2" May 21 00:54:41.478033 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472250 2549 flags.go:64] FLAG: --version="false" May 21 00:54:41.478033 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472257 2549 flags.go:64] FLAG: --vmodule="" May 21 00:54:41.478033 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472263 2549 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" May 21 00:54:41.478033 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472269 2549 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" May 21 00:54:41.478033 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472444 2549 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 21 00:54:41.478033 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472453 2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 21 00:54:41.478033 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472459 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 21 00:54:41.478033 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472465 2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 21 00:54:41.478618 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472470 2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 21 00:54:41.478618 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472475 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 21 00:54:41.478618 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472481 2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 21 00:54:41.478618 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472488 2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 21 00:54:41.478618 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472492 2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 21 00:54:41.478618 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472497 2549 feature_gate.go:328] unrecognized feature gate: Example2 May 21 00:54:41.478618 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472501 2549 feature_gate.go:328] unrecognized feature gate: OVNObservability May 21 00:54:41.478618 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472505 2549 feature_gate.go:328] unrecognized feature gate: PinnedImages May 21 00:54:41.478618 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472510 2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 21 00:54:41.478618 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472515 2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 21 00:54:41.478618 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472519 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 21 00:54:41.478618 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472523 2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 21 00:54:41.478618 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472537 2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 21 00:54:41.478618 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472541 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 21 00:54:41.478618 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472545 2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 21 00:54:41.478618 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472550 2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 21 00:54:41.478618 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472554 2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 21 00:54:41.478618 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472558 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 21 00:54:41.478618 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472563 2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 21 00:54:41.478618 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472572 2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 21 00:54:41.479213 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472576 2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 21 00:54:41.479213 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472580 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 21 00:54:41.479213 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472584 2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 21 00:54:41.479213 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472589 2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 21 00:54:41.479213 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472593 2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 21 00:54:41.479213 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472598 2549 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 21 00:54:41.479213 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472602 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 21 00:54:41.479213 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472606 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 21 00:54:41.479213 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472610 2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 21 00:54:41.479213 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472615 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 21 00:54:41.479213 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472619 2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 21 00:54:41.479213 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472623 2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 21 00:54:41.479213 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472627 2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 21 00:54:41.479213 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472631 2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 21 00:54:41.479213 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472636 2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 21 00:54:41.479213 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472640 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 21 00:54:41.479213 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472645 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 21 00:54:41.479213 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472648 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 21 00:54:41.479213 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472652 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 21 00:54:41.479714 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472657 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 21 00:54:41.479714 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472667 2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 21 00:54:41.479714 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472672 2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 21 00:54:41.479714 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472676 2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 21 00:54:41.479714 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472680 2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 21 00:54:41.479714 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472684 2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 21 00:54:41.479714 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472696 2549 feature_gate.go:328] unrecognized feature gate: NewOLM May 21 00:54:41.479714 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472700 2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 21 00:54:41.479714 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472704 2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 21 00:54:41.479714 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472708 2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 21 00:54:41.479714 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472713 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 21 00:54:41.479714 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472717 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 21 00:54:41.479714 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472723 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 21 00:54:41.479714 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472727 2549 feature_gate.go:328] unrecognized feature gate: Example May 21 00:54:41.479714 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472732 2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 21 00:54:41.479714 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472736 2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 21 00:54:41.479714 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472740 2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 21 00:54:41.479714 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472744 2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 21 00:54:41.479714 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472748 2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 21 00:54:41.479714 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472752 2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 21 00:54:41.480321 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472756 2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 21 00:54:41.480321 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472760 2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 21 00:54:41.480321 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472764 2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 21 00:54:41.480321 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472769 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 21 00:54:41.480321 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472773 2549 feature_gate.go:328] unrecognized feature gate: SignatureStores May 21 00:54:41.480321 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472777 2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 21 00:54:41.480321 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472781 2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 21 00:54:41.480321 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472785 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 21 00:54:41.480321 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472789 2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 21 00:54:41.480321 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472793 2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 21 00:54:41.480321 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472797 2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 21 00:54:41.480321 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472801 2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 21 00:54:41.480321 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472805 2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 21 00:54:41.480321 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472812 2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 21 00:54:41.480321 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472816 2549 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 21 00:54:41.480321 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472820 2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 21 00:54:41.480321 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472824 2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 21 00:54:41.480321 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472828 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 21 00:54:41.480321 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472832 2549 feature_gate.go:328] unrecognized feature gate: DualReplica May 21 00:54:41.480321 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472843 2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 21 00:54:41.480802 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472848 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 21 00:54:41.480802 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472852 2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 21 00:54:41.480802 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.472856 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 21 00:54:41.480802 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.472865 2549 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 21 00:54:41.480802 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.480243 2549 server.go:530] "Kubelet version" kubeletVersion="v1.33.11" May 21 00:54:41.480802 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.480258 2549 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" May 21 00:54:41.480802 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480304 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 21 00:54:41.480802 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480310 2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 21 00:54:41.480802 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480313 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 21 00:54:41.480802 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480316 2549 feature_gate.go:328] unrecognized feature gate: NewOLM May 21 00:54:41.480802 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480319 2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 21 00:54:41.480802 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480322 2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 21 00:54:41.480802 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480325 2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 21 00:54:41.480802 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480328 2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 21 00:54:41.480802 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480330 2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 21 00:54:41.480802 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480334 2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 21 00:54:41.481222 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480337 2549 feature_gate.go:328] unrecognized feature gate: OVNObservability May 21 00:54:41.481222 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480340 2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 21 00:54:41.481222 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480343 2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 21 00:54:41.481222 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480346 2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 21 00:54:41.481222 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480348 2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 21 00:54:41.481222 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480351 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 21 00:54:41.481222 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480353 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 21 00:54:41.481222 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480356 2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 21 00:54:41.481222 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480359 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 21 00:54:41.481222 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480362 2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 21 00:54:41.481222 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480365 2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 21 00:54:41.481222 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480367 2549 feature_gate.go:328] unrecognized feature gate: SignatureStores May 21 00:54:41.481222 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480370 2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 21 00:54:41.481222 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480372 2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 21 00:54:41.481222 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480375 2549 feature_gate.go:328] unrecognized feature gate: PinnedImages May 21 00:54:41.481222 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480378 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 21 00:54:41.481222 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480380 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 21 00:54:41.481222 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480383 2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 21 00:54:41.481222 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480385 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 21 00:54:41.481222 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480388 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 21 00:54:41.481718 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480390 2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 21 00:54:41.481718 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480401 2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 21 00:54:41.481718 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480404 2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 21 00:54:41.481718 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480407 2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 21 00:54:41.481718 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480410 2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 21 00:54:41.481718 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480413 2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 21 00:54:41.481718 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480415 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 21 00:54:41.481718 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480418 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 21 00:54:41.481718 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480421 2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 21 00:54:41.481718 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480423 2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 21 00:54:41.481718 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480426 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 21 00:54:41.481718 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480429 2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 21 00:54:41.481718 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480431 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 21 00:54:41.481718 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480434 2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 21 00:54:41.481718 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480436 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 21 00:54:41.481718 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480439 2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 21 00:54:41.481718 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480443 2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 21 00:54:41.481718 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480447 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 21 00:54:41.481718 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480449 2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 21 00:54:41.482199 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480452 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 21 00:54:41.482199 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480455 2549 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 21 00:54:41.482199 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480458 2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 21 00:54:41.482199 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480460 2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 21 00:54:41.482199 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480463 2549 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 21 00:54:41.482199 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480466 2549 feature_gate.go:328] unrecognized feature gate: Example2 May 21 00:54:41.482199 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480469 2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 21 00:54:41.482199 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480471 2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 21 00:54:41.482199 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480474 2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 21 00:54:41.482199 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480477 2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 21 00:54:41.482199 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480479 2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 21 00:54:41.482199 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480482 2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 21 00:54:41.482199 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480484 2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 21 00:54:41.482199 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480487 2549 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 21 00:54:41.482199 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480490 2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 21 00:54:41.482199 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480494 2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 21 00:54:41.482199 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480497 2549 feature_gate.go:328] unrecognized feature gate: DualReplica May 21 00:54:41.482199 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480499 2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 21 00:54:41.482199 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480502 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 21 00:54:41.482199 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480504 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 21 00:54:41.482678 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480507 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 21 00:54:41.482678 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480510 2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 21 00:54:41.482678 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480514 2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 21 00:54:41.482678 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480517 2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 21 00:54:41.482678 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480520 2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 21 00:54:41.482678 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480523 2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 21 00:54:41.482678 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480526 2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 21 00:54:41.482678 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480528 2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 21 00:54:41.482678 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480531 2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 21 00:54:41.482678 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480534 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 21 00:54:41.482678 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480537 2549 feature_gate.go:328] unrecognized feature gate: Example May 21 00:54:41.482678 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480539 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 21 00:54:41.482678 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480542 2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 21 00:54:41.482678 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480545 2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 21 00:54:41.482678 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480548 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 21 00:54:41.482678 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480550 2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 21 00:54:41.482678 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480553 2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 21 00:54:41.483120 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.480558 2549 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 21 00:54:41.483120 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480654 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 21 00:54:41.483120 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480659 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 21 00:54:41.483120 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480662 2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 21 00:54:41.483120 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480665 2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 21 00:54:41.483120 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480668 2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 21 00:54:41.483120 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480670 2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 21 00:54:41.483120 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480673 2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 21 00:54:41.483120 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480675 2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 21 00:54:41.483120 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480678 2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 21 00:54:41.483120 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480681 2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 21 00:54:41.483120 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480684 2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 21 00:54:41.483120 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480687 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 21 00:54:41.483120 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480690 2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 21 00:54:41.483120 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480692 2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 21 00:54:41.483494 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480695 2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 21 00:54:41.483494 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480698 2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 21 00:54:41.483494 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480701 2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 21 00:54:41.483494 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480705 2549 feature_gate.go:328] unrecognized feature gate: SignatureStores May 21 00:54:41.483494 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480708 2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 21 00:54:41.483494 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480711 2549 feature_gate.go:328] unrecognized feature gate: Example2 May 21 00:54:41.483494 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480714 2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 21 00:54:41.483494 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480716 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 21 00:54:41.483494 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480719 2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 21 00:54:41.483494 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480721 2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 21 00:54:41.483494 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480724 2549 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 21 00:54:41.483494 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480726 2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 21 00:54:41.483494 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480729 2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 21 00:54:41.483494 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480731 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 21 00:54:41.483494 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480734 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 21 00:54:41.483494 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480737 2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 21 00:54:41.483494 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480739 2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 21 00:54:41.483494 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480742 2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 21 00:54:41.483494 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480745 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 21 00:54:41.483494 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480747 2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 21 00:54:41.483977 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480750 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 21 00:54:41.483977 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480752 2549 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 21 00:54:41.483977 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480755 2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 21 00:54:41.483977 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480757 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 21 00:54:41.483977 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480760 2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 21 00:54:41.483977 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480763 2549 feature_gate.go:328] unrecognized feature gate: OVNObservability May 21 00:54:41.483977 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480765 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 21 00:54:41.483977 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480768 2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 21 00:54:41.483977 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480770 2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 21 00:54:41.483977 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480773 2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 21 00:54:41.483977 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480776 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 21 00:54:41.483977 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480778 2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 21 00:54:41.483977 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480781 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 21 00:54:41.483977 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480784 2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 21 00:54:41.483977 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480786 2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 21 00:54:41.483977 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480789 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 21 00:54:41.483977 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480792 2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 21 00:54:41.483977 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480795 2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 21 00:54:41.483977 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480797 2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 21 00:54:41.483977 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480801 2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 21 00:54:41.484476 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480805 2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 21 00:54:41.484476 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480808 2549 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 21 00:54:41.484476 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480811 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 21 00:54:41.484476 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480814 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 21 00:54:41.484476 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480817 2549 feature_gate.go:328] unrecognized feature gate: NewOLM May 21 00:54:41.484476 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480820 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 21 00:54:41.484476 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480822 2549 feature_gate.go:328] unrecognized feature gate: PinnedImages May 21 00:54:41.484476 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480826 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 21 00:54:41.484476 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480829 2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 21 00:54:41.484476 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480831 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 21 00:54:41.484476 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480834 2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 21 00:54:41.484476 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480837 2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 21 00:54:41.484476 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480840 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 21 00:54:41.484476 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480843 2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 21 00:54:41.484476 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480846 2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 21 00:54:41.484476 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480849 2549 feature_gate.go:328] unrecognized feature gate: Example May 21 00:54:41.484476 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480852 2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 21 00:54:41.484476 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480854 2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 21 00:54:41.484476 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480857 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 21 00:54:41.484476 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480859 2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 21 00:54:41.484959 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480862 2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 21 00:54:41.484959 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480864 2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 21 00:54:41.484959 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480867 2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 21 00:54:41.484959 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480870 2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 21 00:54:41.484959 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480872 2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 21 00:54:41.484959 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480875 2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 21 00:54:41.484959 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480877 2549 feature_gate.go:328] unrecognized feature gate: DualReplica May 21 00:54:41.484959 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480880 2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 21 00:54:41.484959 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480882 2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 21 00:54:41.484959 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480885 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 21 00:54:41.484959 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480887 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 21 00:54:41.484959 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:41.480890 2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 21 00:54:41.484959 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.480895 2549 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 21 00:54:41.484959 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.481715 2549 server.go:962] "Client rotation is on, will bootstrap in background" May 21 00:54:41.484959 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.483753 2549 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" May 21 00:54:41.485433 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.484637 2549 server.go:1019] "Starting client certificate rotation" May 21 00:54:41.485433 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.485099 2549 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" May 21 00:54:41.485433 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.485126 2549 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" May 21 00:54:41.507608 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.507590 2549 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 21 00:54:41.509646 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.509629 2549 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 21 00:54:41.527387 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.527275 2549 log.go:25] "Validated CRI v1 runtime API" May 21 00:54:41.532314 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.532298 2549 log.go:25] "Validated CRI v1 image API" May 21 00:54:41.533400 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.533386 2549 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" May 21 00:54:41.536339 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.536320 2549 fs.go:135] Filesystem UUIDs: map[5820c979-79e7-4868-97d7-8a6b2295a059:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 7efef1bd-9fe5-43bf-befb-285144e6ef81:/dev/nvme0n1p3] May 21 00:54:41.536385 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.536340 2549 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] May 21 00:54:41.541854 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.541748 2549 manager.go:217] Machine: {Timestamp:2026-05-21 00:54:41.540458389 +0000 UTC m=+0.437403690 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3072820 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b873f079d0deacb082a155f8bf600 SystemUUID:ec2b873f-079d-0dea-cb08-2a155f8bf600 BootID:80321773-2eb4-4aed-99e0-e14b6a3d632c Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:cc:9f:32:95:c7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:cc:9f:32:95:c7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:46:7c:cd:3a:4c:70 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} May 21 00:54:41.542631 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.542619 2549 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. May 21 00:54:41.542718 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.542700 2549 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} May 21 00:54:41.545270 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.545256 2549 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 21 00:54:41.545460 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.545439 2549 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] May 21 00:54:41.545594 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.545463 2549 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-101.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} May 21 00:54:41.545639 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.545603 2549 topology_manager.go:138] "Creating topology manager with none policy" May 21 00:54:41.545639 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.545612 2549 container_manager_linux.go:306] "Creating device plugin manager" May 21 00:54:41.545639 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.545619 2549 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" May 21 00:54:41.546399 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.546389 2549 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" May 21 00:54:41.547294 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.547284 2549 state_mem.go:36] "Initialized new in-memory state store" May 21 00:54:41.547398 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.547390 2549 server.go:1267] "Using root directory" path="/var/lib/kubelet" May 21 00:54:41.549731 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.549722 2549 kubelet.go:491] "Attempting to sync node with API server" May 21 00:54:41.549781 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.549735 2549 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" May 21 00:54:41.549781 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.549747 2549 file.go:69] "Watching path" path="/etc/kubernetes/manifests" May 21 00:54:41.549781 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.549755 2549 kubelet.go:397] "Adding apiserver pod source" May 21 00:54:41.549781 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.549763 2549 apiserver.go:42] "Waiting for node sync before watching apiserver pods" May 21 00:54:41.550844 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.550833 2549 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 21 00:54:41.550880 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.550851 2549 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 21 00:54:41.553788 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.553771 2549 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1" May 21 00:54:41.557102 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.557071 2549 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" May 21 00:54:41.558160 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.558143 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" May 21 00:54:41.558249 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.558172 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" May 21 00:54:41.558249 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.558215 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" May 21 00:54:41.558356 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.558334 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" May 21 00:54:41.558404 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.558363 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" May 21 00:54:41.558404 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.558387 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" May 21 00:54:41.558404 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.558397 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" May 21 00:54:41.558519 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.558406 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" May 21 00:54:41.558519 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.558418 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" May 21 00:54:41.558519 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.558428 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" May 21 00:54:41.558519 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.558442 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" May 21 00:54:41.558519 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.558455 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" May 21 00:54:41.559483 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.559467 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" May 21 00:54:41.559544 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.559531 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" May 21 00:54:41.562787 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.562772 2549 watchdog_linux.go:99] "Systemd watchdog is not enabled" May 21 00:54:41.562872 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.562808 2549 server.go:1295] "Started kubelet" May 21 00:54:41.562951 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.562911 2549 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 May 21 00:54:41.563005 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.562914 2549 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 May 21 00:54:41.563005 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.562966 2549 server_v1.go:47] "podresources" method="list" useActivePods=true May 21 00:54:41.563629 ip-10-0-138-101 systemd[1]: Started Kubernetes Kubelet. May 21 00:54:41.563993 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:41.563759 2549 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-101.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" May 21 00:54:41.563993 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.563957 2549 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-101.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope May 21 00:54:41.564240 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.564225 2549 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-j8ff7" May 21 00:54:41.564496 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:41.564427 2549 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" May 21 00:54:41.565163 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.565147 2549 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" May 21 00:54:41.566636 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.566624 2549 server.go:317] "Adding debug handlers to kubelet server" May 21 00:54:41.570895 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.570878 2549 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-j8ff7" May 21 00:54:41.571416 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.571397 2549 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" May 21 00:54:41.571914 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:41.571892 2549 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" May 21 00:54:41.571995 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.571981 2549 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" May 21 00:54:41.572081 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.572051 2549 volume_manager.go:295] "The desired_state_of_world populator starts" May 21 00:54:41.572081 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.572079 2549 volume_manager.go:297] "Starting Kubelet Volume Manager" May 21 00:54:41.572185 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.572133 2549 desired_state_of_world_populator.go:150] "Desired state populator starts to run" May 21 00:54:41.572273 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.572261 2549 reconstruct.go:97] "Volume reconstruction finished" May 21 00:54:41.572330 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.572275 2549 reconciler.go:26] "Reconciler: start to sync state" May 21 00:54:41.572893 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:41.570102 2549 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-101.ec2.internal.18b16e8d719052db default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-101.ec2.internal,UID:ip-10-0-138-101.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-101.ec2.internal,},FirstTimestamp:2026-05-21 00:54:41.562784475 +0000 UTC m=+0.459729776,LastTimestamp:2026-05-21 00:54:41.562784475 +0000 UTC m=+0.459729776,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-101.ec2.internal,}" May 21 00:54:41.573000 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.572986 2549 factory.go:55] Registering systemd factory May 21 00:54:41.573074 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.573003 2549 factory.go:223] Registration of the systemd container factory successfully May 21 00:54:41.573145 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:41.573126 2549 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-101.ec2.internal\" not found" May 21 00:54:41.573213 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.573195 2549 factory.go:153] Registering CRI-O factory May 21 00:54:41.573213 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.573211 2549 factory.go:223] Registration of the crio container factory successfully May 21 00:54:41.573344 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.573255 2549 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory May 21 00:54:41.573344 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.573277 2549 factory.go:103] Registering Raw factory May 21 00:54:41.573344 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.573293 2549 manager.go:1196] Started watching for new ooms in manager May 21 00:54:41.573675 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.573660 2549 manager.go:319] Starting recovery of all containers May 21 00:54:41.577511 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.577490 2549 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" May 21 00:54:41.580962 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:41.580940 2549 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-138-101.ec2.internal\" not found" node="ip-10-0-138-101.ec2.internal" May 21 00:54:41.587015 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.586999 2549 manager.go:324] Recovery completed May 21 00:54:41.591441 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.591429 2549 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 21 00:54:41.593251 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.593233 2549 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasSufficientMemory" May 21 00:54:41.593330 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.593266 2549 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasNoDiskPressure" May 21 00:54:41.593330 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.593280 2549 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasSufficientPID" May 21 00:54:41.593864 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.593850 2549 cpu_manager.go:222] "Starting CPU manager" policy="none" May 21 00:54:41.593864 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.593864 2549 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" May 21 00:54:41.593948 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.593880 2549 state_mem.go:36] "Initialized new in-memory state store" May 21 00:54:41.596321 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.596308 2549 policy_none.go:49] "None policy: Start" May 21 00:54:41.596368 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.596325 2549 memory_manager.go:186] "Starting memorymanager" policy="None" May 21 00:54:41.596368 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.596336 2549 state_mem.go:35] "Initializing new in-memory state store" May 21 00:54:41.636646 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.636632 2549 manager.go:341] "Starting Device Plugin manager" May 21 00:54:41.639513 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:41.636688 2549 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" May 21 00:54:41.639513 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.636701 2549 server.go:85] "Starting device plugin registration server" May 21 00:54:41.639513 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.636902 2549 eviction_manager.go:189] "Eviction manager: starting control loop" May 21 00:54:41.639513 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.636914 2549 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" May 21 00:54:41.639513 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.636993 2549 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" May 21 00:54:41.639513 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.637150 2549 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" May 21 00:54:41.639513 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.637163 2549 plugin_manager.go:118] "Starting Kubelet Plugin Manager" May 21 00:54:41.639513 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:41.637668 2549 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" May 21 00:54:41.639513 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:41.637695 2549 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-101.ec2.internal\" not found" May 21 00:54:41.665593 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.665562 2549 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" May 21 00:54:41.666724 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.666707 2549 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" May 21 00:54:41.666776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.666734 2549 status_manager.go:230] "Starting to sync pod status with apiserver" May 21 00:54:41.666776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.666760 2549 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." May 21 00:54:41.666776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.666768 2549 kubelet.go:2451] "Starting kubelet main sync loop" May 21 00:54:41.666880 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:41.666838 2549 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" May 21 00:54:41.668789 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.668774 2549 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 21 00:54:41.737844 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.737829 2549 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 21 00:54:41.738693 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.738679 2549 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasSufficientMemory" May 21 00:54:41.738768 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.738705 2549 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasNoDiskPressure" May 21 00:54:41.738768 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.738717 2549 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasSufficientPID" May 21 00:54:41.738768 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.738740 2549 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-101.ec2.internal" May 21 00:54:41.746218 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.746204 2549 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-101.ec2.internal" May 21 00:54:41.746272 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:41.746225 2549 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-101.ec2.internal\": node \"ip-10-0-138-101.ec2.internal\" not found" May 21 00:54:41.766673 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:41.766650 2549 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-101.ec2.internal\" not found" May 21 00:54:41.767657 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.767635 2549 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-101.ec2.internal"] May 21 00:54:41.767714 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.767697 2549 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 21 00:54:41.768387 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.768373 2549 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasSufficientMemory" May 21 00:54:41.768449 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.768397 2549 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasNoDiskPressure" May 21 00:54:41.768449 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.768408 2549 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasSufficientPID" May 21 00:54:41.769944 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.769932 2549 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 21 00:54:41.770155 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.770134 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal" May 21 00:54:41.770219 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.770169 2549 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 21 00:54:41.770555 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.770543 2549 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasSufficientMemory" May 21 00:54:41.770620 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.770562 2549 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasNoDiskPressure" May 21 00:54:41.770620 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.770571 2549 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasSufficientPID" May 21 00:54:41.770735 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.770721 2549 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasSufficientMemory" May 21 00:54:41.770766 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.770750 2549 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasNoDiskPressure" May 21 00:54:41.770766 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.770762 2549 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasSufficientPID" May 21 00:54:41.772204 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.772186 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-101.ec2.internal" May 21 00:54:41.772269 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.772210 2549 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 21 00:54:41.772816 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.772803 2549 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasSufficientMemory" May 21 00:54:41.772888 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.772831 2549 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasNoDiskPressure" May 21 00:54:41.772888 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.772846 2549 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasSufficientPID" May 21 00:54:41.799668 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:41.799647 2549 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-101.ec2.internal\" not found" node="ip-10-0-138-101.ec2.internal" May 21 00:54:41.803950 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:41.803936 2549 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-101.ec2.internal\" not found" node="ip-10-0-138-101.ec2.internal" May 21 00:54:41.866730 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:41.866709 2549 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-101.ec2.internal\" not found" May 21 00:54:41.873573 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.873557 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f0e5357ad01f42d530056be0164fe34c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal\" (UID: \"f0e5357ad01f42d530056be0164fe34c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal" May 21 00:54:41.873631 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.873603 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0e5357ad01f42d530056be0164fe34c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal\" (UID: \"f0e5357ad01f42d530056be0164fe34c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal" May 21 00:54:41.873666 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.873625 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/97c57dc15067251bbf5d4474732b6483-config\") pod \"kube-apiserver-proxy-ip-10-0-138-101.ec2.internal\" (UID: \"97c57dc15067251bbf5d4474732b6483\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-101.ec2.internal" May 21 00:54:41.967808 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:41.967781 2549 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-101.ec2.internal\" not found" May 21 00:54:41.974048 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.974030 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f0e5357ad01f42d530056be0164fe34c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal\" (UID: \"f0e5357ad01f42d530056be0164fe34c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal" May 21 00:54:41.974117 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.974055 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0e5357ad01f42d530056be0164fe34c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal\" (UID: \"f0e5357ad01f42d530056be0164fe34c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal" May 21 00:54:41.974117 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.974091 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/97c57dc15067251bbf5d4474732b6483-config\") pod \"kube-apiserver-proxy-ip-10-0-138-101.ec2.internal\" (UID: \"97c57dc15067251bbf5d4474732b6483\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-101.ec2.internal" May 21 00:54:41.974117 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.974116 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/97c57dc15067251bbf5d4474732b6483-config\") pod \"kube-apiserver-proxy-ip-10-0-138-101.ec2.internal\" (UID: \"97c57dc15067251bbf5d4474732b6483\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-101.ec2.internal" May 21 00:54:41.974211 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.974127 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0e5357ad01f42d530056be0164fe34c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal\" (UID: \"f0e5357ad01f42d530056be0164fe34c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal" May 21 00:54:41.974211 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:41.974117 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f0e5357ad01f42d530056be0164fe34c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal\" (UID: \"f0e5357ad01f42d530056be0164fe34c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal" May 21 00:54:42.068436 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:42.068406 2549 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-101.ec2.internal\" not found" May 21 00:54:42.102864 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:42.102845 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal" May 21 00:54:42.106431 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:42.106409 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-101.ec2.internal" May 21 00:54:42.169148 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:42.169119 2549 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-101.ec2.internal\" not found" May 21 00:54:42.269632 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:42.269606 2549 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-101.ec2.internal\" not found" May 21 00:54:42.370140 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:42.370083 2549 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-101.ec2.internal\" not found" May 21 00:54:42.470799 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:42.470770 2549 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-101.ec2.internal\" not found" May 21 00:54:42.484243 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:42.484225 2549 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" May 21 00:54:42.484395 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:42.484374 2549 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 21 00:54:42.484435 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:42.484365 2549 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 21 00:54:42.571838 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:42.571801 2549 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" May 21 00:54:42.572002 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:42.571804 2549 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-101.ec2.internal\" not found" May 21 00:54:42.572336 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:42.572313 2549 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-20 00:49:41 +0000 UTC" deadline="2028-02-15 01:25:00.276616016 +0000 UTC" May 21 00:54:42.572407 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:42.572343 2549 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15240h30m17.704284014s" May 21 00:54:42.581010 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:42.580985 2549 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 21 00:54:42.598848 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:42.598824 2549 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2p72f" May 21 00:54:42.606167 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:42.606140 2549 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2p72f" May 21 00:54:42.646234 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:42.646099 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0e5357ad01f42d530056be0164fe34c.slice/crio-ec6eb726b14d4aaf1275e216726535c7f69da55f5610ed7fee2e68d30bd53f80 WatchSource:0}: Error finding container ec6eb726b14d4aaf1275e216726535c7f69da55f5610ed7fee2e68d30bd53f80: Status 404 returned error can't find the container with id ec6eb726b14d4aaf1275e216726535c7f69da55f5610ed7fee2e68d30bd53f80 May 21 00:54:42.649739 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:42.648358 2549 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" May 21 00:54:42.652176 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:42.652160 2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 21 00:54:42.655513 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:42.655494 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97c57dc15067251bbf5d4474732b6483.slice/crio-d1b81ca44a88ffd502e4a0c776d39242904ac89b517b4cf1ce9771bb7f8b441b WatchSource:0}: Error finding container d1b81ca44a88ffd502e4a0c776d39242904ac89b517b4cf1ce9771bb7f8b441b: Status 404 returned error can't find the container with id d1b81ca44a88ffd502e4a0c776d39242904ac89b517b4cf1ce9771bb7f8b441b May 21 00:54:42.655599 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:42.655569 2549 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" May 21 00:54:42.669977 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:42.669941 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-101.ec2.internal" event={"ID":"97c57dc15067251bbf5d4474732b6483","Type":"ContainerStarted","Data":"d1b81ca44a88ffd502e4a0c776d39242904ac89b517b4cf1ce9771bb7f8b441b"} May 21 00:54:42.670930 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:42.670912 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal" event={"ID":"f0e5357ad01f42d530056be0164fe34c","Type":"ContainerStarted","Data":"ec6eb726b14d4aaf1275e216726535c7f69da55f5610ed7fee2e68d30bd53f80"} May 21 00:54:42.672509 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:42.672495 2549 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal" May 21 00:54:42.682621 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:42.682606 2549 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 21 00:54:42.683475 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:42.683464 2549 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-101.ec2.internal" May 21 00:54:42.690726 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:42.690715 2549 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 21 00:54:43.551042 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.551013 2549 apiserver.go:52] "Watching apiserver" May 21 00:54:43.556947 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.556924 2549 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" May 21 00:54:43.557348 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.557327 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-96x9m","openshift-image-registry/node-ca-sfsvt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal","openshift-multus/multus-additional-cni-plugins-spsr8","openshift-multus/network-metrics-daemon-skbch","kube-system/konnectivity-agent-h7nwx","openshift-multus/multus-sh44w","openshift-network-diagnostics/network-check-target-rxzxp","openshift-network-operator/iptables-alerter-qdnd9","openshift-ovn-kubernetes/ovnkube-node-kgxp9","kube-system/kube-apiserver-proxy-ip-10-0-138-101.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9","openshift-cluster-node-tuning-operator/tuned-gvmxz"] May 21 00:54:43.561975 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.561957 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sh44w" May 21 00:54:43.562144 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.562122 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sfsvt" May 21 00:54:43.564310 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.564288 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-spsr8" May 21 00:54:43.564563 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.564544 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" May 21 00:54:43.564827 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.564811 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jqf5q\"" May 21 00:54:43.565468 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.565019 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" May 21 00:54:43.565468 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.565121 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" May 21 00:54:43.565468 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.565157 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qwmpl\"" May 21 00:54:43.565468 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.565251 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" May 21 00:54:43.565468 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.565258 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" May 21 00:54:43.565468 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.565352 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" May 21 00:54:43.565750 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.565533 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" May 21 00:54:43.566200 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.566175 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kbnl5\"" May 21 00:54:43.566421 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.566398 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" May 21 00:54:43.566502 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.566403 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" May 21 00:54:43.568997 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.568764 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:54:43.568997 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:43.568838 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skbch" podUID="5ee0c301-12eb-4c68-9990-6ceafb2c90fa" May 21 00:54:43.568997 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.568869 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h7nwx" May 21 00:54:43.572398 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.571151 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-tf6jn\"" May 21 00:54:43.572398 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.571190 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" May 21 00:54:43.572398 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.571262 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-96x9m" May 21 00:54:43.572398 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.571612 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" May 21 00:54:43.573405 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.573386 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" May 21 00:54:43.573760 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.573724 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bdxl6\"" May 21 00:54:43.574154 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.573758 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" May 21 00:54:43.574516 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.574500 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:54:43.574595 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:43.574576 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxzxp" podUID="3b778703-acf8-4977-a1b1-2f5f9042d2b3" May 21 00:54:43.576705 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.576685 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qdnd9" May 21 00:54:43.579563 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.579127 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.579563 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.579414 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zl5b5\"" May 21 00:54:43.579725 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.579709 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" May 21 00:54:43.579820 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.579799 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" May 21 00:54:43.580083 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.580053 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" May 21 00:54:43.581029 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.581015 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" May 21 00:54:43.581133 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.581053 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9ff27421-0ba9-40f6-98db-150fbcca1a5e-konnectivity-ca\") pod \"konnectivity-agent-h7nwx\" (UID: \"9ff27421-0ba9-40f6-98db-150fbcca1a5e\") " pod="kube-system/konnectivity-agent-h7nwx" May 21 00:54:43.581189 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.581150 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5da3aae9-eaf5-4328-afed-6595832fc2e1-system-cni-dir\") pod \"multus-additional-cni-plugins-spsr8\" (UID: \"5da3aae9-eaf5-4328-afed-6595832fc2e1\") " pod="openshift-multus/multus-additional-cni-plugins-spsr8" May 21 00:54:43.581189 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.581181 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5da3aae9-eaf5-4328-afed-6595832fc2e1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-spsr8\" (UID: \"5da3aae9-eaf5-4328-afed-6595832fc2e1\") " pod="openshift-multus/multus-additional-cni-plugins-spsr8" May 21 00:54:43.581297 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.581214 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cvsw\" (UniqueName: \"kubernetes.io/projected/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-kube-api-access-9cvsw\") pod \"network-metrics-daemon-skbch\" (UID: \"5ee0c301-12eb-4c68-9990-6ceafb2c90fa\") " pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:54:43.581297 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.581238 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5da3aae9-eaf5-4328-afed-6595832fc2e1-cnibin\") pod \"multus-additional-cni-plugins-spsr8\" (UID: \"5da3aae9-eaf5-4328-afed-6595832fc2e1\") " pod="openshift-multus/multus-additional-cni-plugins-spsr8" May 21 00:54:43.581297 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.581285 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-multus-conf-dir\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.581436 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.581311 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/acd173d4-75da-4564-ac8e-84996bb1742d-multus-daemon-config\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.581436 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.581354 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-298bc\" (UniqueName: \"kubernetes.io/projected/0a1071e0-9724-4546-8b54-44bce04c35c2-kube-api-access-298bc\") pod \"node-ca-sfsvt\" (UID: \"0a1071e0-9724-4546-8b54-44bce04c35c2\") " pod="openshift-image-registry/node-ca-sfsvt" May 21 00:54:43.581436 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.581380 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-multus-socket-dir-parent\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.581577 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.581447 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-host-run-netns\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.581577 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.581518 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" May 21 00:54:43.581577 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.581528 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0a1071e0-9724-4546-8b54-44bce04c35c2-serviceca\") pod \"node-ca-sfsvt\" (UID: \"0a1071e0-9724-4546-8b54-44bce04c35c2\") " pod="openshift-image-registry/node-ca-sfsvt" May 21 00:54:43.581577 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.581554 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7fvr\" (UniqueName: \"kubernetes.io/projected/5da3aae9-eaf5-4328-afed-6595832fc2e1-kube-api-access-c7fvr\") pod \"multus-additional-cni-plugins-spsr8\" (UID: \"5da3aae9-eaf5-4328-afed-6595832fc2e1\") " pod="openshift-multus/multus-additional-cni-plugins-spsr8" May 21 00:54:43.581739 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.581601 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-multus-cni-dir\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.581739 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.581626 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jdlvs\"" May 21 00:54:43.581739 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.581627 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-hostroot\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.581739 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.581725 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5da3aae9-eaf5-4328-afed-6595832fc2e1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-spsr8\" (UID: \"5da3aae9-eaf5-4328-afed-6595832fc2e1\") " pod="openshift-multus/multus-additional-cni-plugins-spsr8" May 21 00:54:43.581846 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.581756 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31374768-4e56-404b-8d22-7e2136138fa4-tmp-dir\") pod \"node-resolver-96x9m\" (UID: \"31374768-4e56-404b-8d22-7e2136138fa4\") " pod="openshift-dns/node-resolver-96x9m" May 21 00:54:43.581846 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.581774 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" May 21 00:54:43.581846 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.581785 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m22nq\" (UniqueName: \"kubernetes.io/projected/31374768-4e56-404b-8d22-7e2136138fa4-kube-api-access-m22nq\") pod \"node-resolver-96x9m\" (UID: \"31374768-4e56-404b-8d22-7e2136138fa4\") " pod="openshift-dns/node-resolver-96x9m" May 21 00:54:43.581846 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.581822 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-cnibin\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.581989 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.581882 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/acd173d4-75da-4564-ac8e-84996bb1742d-cni-binary-copy\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.581989 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.581943 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-host-var-lib-kubelet\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.581989 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.581967 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5da3aae9-eaf5-4328-afed-6595832fc2e1-os-release\") pod \"multus-additional-cni-plugins-spsr8\" (UID: \"5da3aae9-eaf5-4328-afed-6595832fc2e1\") " pod="openshift-multus/multus-additional-cni-plugins-spsr8" May 21 00:54:43.582154 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.582011 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5da3aae9-eaf5-4328-afed-6595832fc2e1-cni-binary-copy\") pod \"multus-additional-cni-plugins-spsr8\" (UID: \"5da3aae9-eaf5-4328-afed-6595832fc2e1\") " pod="openshift-multus/multus-additional-cni-plugins-spsr8" May 21 00:54:43.582154 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.582036 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9ff27421-0ba9-40f6-98db-150fbcca1a5e-agent-certs\") pod \"konnectivity-agent-h7nwx\" (UID: \"9ff27421-0ba9-40f6-98db-150fbcca1a5e\") " pod="kube-system/konnectivity-agent-h7nwx" May 21 00:54:43.582154 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.582077 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/31374768-4e56-404b-8d22-7e2136138fa4-hosts-file\") pod \"node-resolver-96x9m\" (UID: \"31374768-4e56-404b-8d22-7e2136138fa4\") " pod="openshift-dns/node-resolver-96x9m" May 21 00:54:43.582154 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.582112 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-system-cni-dir\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.582154 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.582142 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-os-release\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.582393 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.582167 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-host-run-k8s-cni-cncf-io\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.582393 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.582210 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-host-var-lib-cni-multus\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.582393 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.582242 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2brz9\" (UniqueName: \"kubernetes.io/projected/acd173d4-75da-4564-ac8e-84996bb1742d-kube-api-access-2brz9\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.582393 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.582267 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5da3aae9-eaf5-4328-afed-6595832fc2e1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-spsr8\" (UID: \"5da3aae9-eaf5-4328-afed-6595832fc2e1\") " pod="openshift-multus/multus-additional-cni-plugins-spsr8" May 21 00:54:43.582393 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.582292 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-host-run-multus-certs\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.582393 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.582318 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0a1071e0-9724-4546-8b54-44bce04c35c2-host\") pod \"node-ca-sfsvt\" (UID: \"0a1071e0-9724-4546-8b54-44bce04c35c2\") " pod="openshift-image-registry/node-ca-sfsvt" May 21 00:54:43.582393 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.582363 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-metrics-certs\") pod \"network-metrics-daemon-skbch\" (UID: \"5ee0c301-12eb-4c68-9990-6ceafb2c90fa\") " pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:54:43.582393 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.582389 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" May 21 00:54:43.582743 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.582393 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" May 21 00:54:43.582743 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.582392 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" May 21 00:54:43.582743 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.582405 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66pbg\" (UniqueName: \"kubernetes.io/projected/3b778703-acf8-4977-a1b1-2f5f9042d2b3-kube-api-access-66pbg\") pod \"network-check-target-rxzxp\" (UID: \"3b778703-acf8-4977-a1b1-2f5f9042d2b3\") " pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:54:43.582743 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.582468 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-host-var-lib-cni-bin\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.582743 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.582502 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-etc-kubernetes\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.582743 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.582512 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" May 21 00:54:43.583584 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.583566 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" May 21 00:54:43.583951 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.583929 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" May 21 00:54:43.584021 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.583940 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6xsg2\"" May 21 00:54:43.584106 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.584080 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.584380 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.584362 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" May 21 00:54:43.586042 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.586025 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" May 21 00:54:43.586147 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.586052 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" May 21 00:54:43.586147 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.586122 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cp24v\"" May 21 00:54:43.606853 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.606821 2549 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-20 00:49:42 +0000 UTC" deadline="2027-11-25 15:32:41.948456717 +0000 UTC" May 21 00:54:43.606943 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.606854 2549 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13286h37m58.341606602s" May 21 00:54:43.672714 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.672691 2549 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" May 21 00:54:43.682962 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.682928 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-etc-sysconfig\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.683085 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.682970 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-sys\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.683085 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.682995 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-lib-modules\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.683085 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683040 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-multus-conf-dir\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.683225 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683087 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/acd173d4-75da-4564-ac8e-84996bb1742d-multus-daemon-config\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.683225 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683113 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-298bc\" (UniqueName: \"kubernetes.io/projected/0a1071e0-9724-4546-8b54-44bce04c35c2-kube-api-access-298bc\") pod \"node-ca-sfsvt\" (UID: \"0a1071e0-9724-4546-8b54-44bce04c35c2\") " pod="openshift-image-registry/node-ca-sfsvt" May 21 00:54:43.683225 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683121 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-multus-conf-dir\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.683225 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683141 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-host-slash\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.683373 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683233 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-ovnkube-config\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.683373 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683254 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-etc-systemd\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.683373 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683267 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-host\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.683373 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683287 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-host-run-netns\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.683373 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683311 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8adbdb60-a625-47a1-8252-779cd35e6a35-registration-dir\") pod \"aws-ebs-csi-driver-node-vvqd9\" (UID: \"8adbdb60-a625-47a1-8252-779cd35e6a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" May 21 00:54:43.683373 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683335 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-etc-modprobe-d\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.683373 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683358 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-var-lib-kubelet\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.683675 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683377 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-host-run-netns\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.683675 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683382 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-multus-cni-dir\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.683675 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683434 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8adbdb60-a625-47a1-8252-779cd35e6a35-socket-dir\") pod \"aws-ebs-csi-driver-node-vvqd9\" (UID: \"8adbdb60-a625-47a1-8252-779cd35e6a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" May 21 00:54:43.683675 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683461 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8adbdb60-a625-47a1-8252-779cd35e6a35-etc-selinux\") pod \"aws-ebs-csi-driver-node-vvqd9\" (UID: \"8adbdb60-a625-47a1-8252-779cd35e6a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" May 21 00:54:43.683675 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683463 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-multus-cni-dir\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.683675 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683486 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-etc-sysctl-d\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.683675 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683507 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-run\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.683675 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683523 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31374768-4e56-404b-8d22-7e2136138fa4-tmp-dir\") pod \"node-resolver-96x9m\" (UID: \"31374768-4e56-404b-8d22-7e2136138fa4\") " pod="openshift-dns/node-resolver-96x9m" May 21 00:54:43.683675 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683544 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m22nq\" (UniqueName: \"kubernetes.io/projected/31374768-4e56-404b-8d22-7e2136138fa4-kube-api-access-m22nq\") pod \"node-resolver-96x9m\" (UID: \"31374768-4e56-404b-8d22-7e2136138fa4\") " pod="openshift-dns/node-resolver-96x9m" May 21 00:54:43.684014 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683695 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/acd173d4-75da-4564-ac8e-84996bb1742d-multus-daemon-config\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.684014 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683724 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-cnibin\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.684014 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683753 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-host-var-lib-kubelet\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.684014 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683779 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-etc-openvswitch\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.684014 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683812 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-etc-sysctl-conf\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.684014 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683817 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31374768-4e56-404b-8d22-7e2136138fa4-tmp-dir\") pod \"node-resolver-96x9m\" (UID: \"31374768-4e56-404b-8d22-7e2136138fa4\") " pod="openshift-dns/node-resolver-96x9m" May 21 00:54:43.684014 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683810 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-cnibin\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.684014 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683840 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9ff27421-0ba9-40f6-98db-150fbcca1a5e-agent-certs\") pod \"konnectivity-agent-h7nwx\" (UID: \"9ff27421-0ba9-40f6-98db-150fbcca1a5e\") " pod="kube-system/konnectivity-agent-h7nwx" May 21 00:54:43.684014 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683854 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-host-var-lib-kubelet\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.684014 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.683879 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-system-cni-dir\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.684432 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684018 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2brz9\" (UniqueName: \"kubernetes.io/projected/acd173d4-75da-4564-ac8e-84996bb1742d-kube-api-access-2brz9\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.684432 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684049 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.684432 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684018 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-system-cni-dir\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.684432 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684091 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-host-run-multus-certs\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.684432 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684117 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-host-kubelet\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.684432 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684141 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-host-cni-bin\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.684432 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684165 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7d5e55de-c445-48f6-8911-e4f1a11104de-iptables-alerter-script\") pod \"iptables-alerter-qdnd9\" (UID: \"7d5e55de-c445-48f6-8911-e4f1a11104de\") " pod="openshift-network-operator/iptables-alerter-qdnd9" May 21 00:54:43.684432 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684188 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-host-run-multus-certs\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.684432 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684189 2549 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" May 21 00:54:43.684432 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684204 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-host-run-netns\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.684432 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684233 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-run-ovn\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.684432 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684251 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-env-overrides\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.684432 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684301 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwj7k\" (UniqueName: \"kubernetes.io/projected/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-kube-api-access-mwj7k\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.684432 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684324 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9cvsw\" (UniqueName: \"kubernetes.io/projected/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-kube-api-access-9cvsw\") pod \"network-metrics-daemon-skbch\" (UID: \"5ee0c301-12eb-4c68-9990-6ceafb2c90fa\") " pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:54:43.684432 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684378 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5da3aae9-eaf5-4328-afed-6595832fc2e1-cnibin\") pod \"multus-additional-cni-plugins-spsr8\" (UID: \"5da3aae9-eaf5-4328-afed-6595832fc2e1\") " pod="openshift-multus/multus-additional-cni-plugins-spsr8" May 21 00:54:43.684432 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684409 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7d5e55de-c445-48f6-8911-e4f1a11104de-host-slash\") pod \"iptables-alerter-qdnd9\" (UID: \"7d5e55de-c445-48f6-8911-e4f1a11104de\") " pod="openshift-network-operator/iptables-alerter-qdnd9" May 21 00:54:43.684432 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684439 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8adbdb60-a625-47a1-8252-779cd35e6a35-device-dir\") pod \"aws-ebs-csi-driver-node-vvqd9\" (UID: \"8adbdb60-a625-47a1-8252-779cd35e6a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" May 21 00:54:43.685226 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684461 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8adbdb60-a625-47a1-8252-779cd35e6a35-sys-fs\") pod \"aws-ebs-csi-driver-node-vvqd9\" (UID: \"8adbdb60-a625-47a1-8252-779cd35e6a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" May 21 00:54:43.685226 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684469 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5da3aae9-eaf5-4328-afed-6595832fc2e1-cnibin\") pod \"multus-additional-cni-plugins-spsr8\" (UID: \"5da3aae9-eaf5-4328-afed-6595832fc2e1\") " pod="openshift-multus/multus-additional-cni-plugins-spsr8" May 21 00:54:43.685226 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684489 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-multus-socket-dir-parent\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.685226 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684536 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0a1071e0-9724-4546-8b54-44bce04c35c2-serviceca\") pod \"node-ca-sfsvt\" (UID: \"0a1071e0-9724-4546-8b54-44bce04c35c2\") " pod="openshift-image-registry/node-ca-sfsvt" May 21 00:54:43.685226 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684597 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c7fvr\" (UniqueName: \"kubernetes.io/projected/5da3aae9-eaf5-4328-afed-6595832fc2e1-kube-api-access-c7fvr\") pod \"multus-additional-cni-plugins-spsr8\" (UID: \"5da3aae9-eaf5-4328-afed-6595832fc2e1\") " pod="openshift-multus/multus-additional-cni-plugins-spsr8" May 21 00:54:43.685226 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684606 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-multus-socket-dir-parent\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.685226 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684620 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-host-run-ovn-kubernetes\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.685226 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684655 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-host-cni-netd\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.685226 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684678 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8adbdb60-a625-47a1-8252-779cd35e6a35-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vvqd9\" (UID: \"8adbdb60-a625-47a1-8252-779cd35e6a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" May 21 00:54:43.685226 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684701 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-hostroot\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.685226 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684722 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5da3aae9-eaf5-4328-afed-6595832fc2e1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-spsr8\" (UID: \"5da3aae9-eaf5-4328-afed-6595832fc2e1\") " pod="openshift-multus/multus-additional-cni-plugins-spsr8" May 21 00:54:43.685226 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684746 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-run-systemd\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.685226 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684772 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-ovn-node-metrics-cert\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.685226 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684786 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-hostroot\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.685226 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684798 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/acd173d4-75da-4564-ac8e-84996bb1742d-cni-binary-copy\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.685226 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684831 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5da3aae9-eaf5-4328-afed-6595832fc2e1-os-release\") pod \"multus-additional-cni-plugins-spsr8\" (UID: \"5da3aae9-eaf5-4328-afed-6595832fc2e1\") " pod="openshift-multus/multus-additional-cni-plugins-spsr8" May 21 00:54:43.685226 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684853 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5da3aae9-eaf5-4328-afed-6595832fc2e1-cni-binary-copy\") pod \"multus-additional-cni-plugins-spsr8\" (UID: \"5da3aae9-eaf5-4328-afed-6595832fc2e1\") " pod="openshift-multus/multus-additional-cni-plugins-spsr8" May 21 00:54:43.685962 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684870 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-etc-kubernetes\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.685962 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684890 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mb8w\" (UniqueName: \"kubernetes.io/projected/96547212-630f-43e2-9662-5938454d35ec-kube-api-access-6mb8w\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.685962 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684943 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/31374768-4e56-404b-8d22-7e2136138fa4-hosts-file\") pod \"node-resolver-96x9m\" (UID: \"31374768-4e56-404b-8d22-7e2136138fa4\") " pod="openshift-dns/node-resolver-96x9m" May 21 00:54:43.685962 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684974 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5da3aae9-eaf5-4328-afed-6595832fc2e1-os-release\") pod \"multus-additional-cni-plugins-spsr8\" (UID: \"5da3aae9-eaf5-4328-afed-6595832fc2e1\") " pod="openshift-multus/multus-additional-cni-plugins-spsr8" May 21 00:54:43.685962 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684980 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0a1071e0-9724-4546-8b54-44bce04c35c2-serviceca\") pod \"node-ca-sfsvt\" (UID: \"0a1071e0-9724-4546-8b54-44bce04c35c2\") " pod="openshift-image-registry/node-ca-sfsvt" May 21 00:54:43.685962 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684988 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-os-release\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.685962 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.684981 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/31374768-4e56-404b-8d22-7e2136138fa4-hosts-file\") pod \"node-resolver-96x9m\" (UID: \"31374768-4e56-404b-8d22-7e2136138fa4\") " pod="openshift-dns/node-resolver-96x9m" May 21 00:54:43.685962 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685040 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-host-run-k8s-cni-cncf-io\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.685962 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685053 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-os-release\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.685962 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685089 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-host-var-lib-cni-multus\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.685962 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685114 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-host-run-k8s-cni-cncf-io\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.685962 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685117 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5da3aae9-eaf5-4328-afed-6595832fc2e1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-spsr8\" (UID: \"5da3aae9-eaf5-4328-afed-6595832fc2e1\") " pod="openshift-multus/multus-additional-cni-plugins-spsr8" May 21 00:54:43.685962 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685155 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-systemd-units\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.685962 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685184 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-var-lib-openvswitch\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.685962 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685209 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-ovnkube-script-lib\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.685962 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685237 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0a1071e0-9724-4546-8b54-44bce04c35c2-host\") pod \"node-ca-sfsvt\" (UID: \"0a1071e0-9724-4546-8b54-44bce04c35c2\") " pod="openshift-image-registry/node-ca-sfsvt" May 21 00:54:43.685962 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685261 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-run-openvswitch\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.686656 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685284 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/96547212-630f-43e2-9662-5938454d35ec-etc-tuned\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.686656 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685304 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/96547212-630f-43e2-9662-5938454d35ec-tmp\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.686656 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685317 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/acd173d4-75da-4564-ac8e-84996bb1742d-cni-binary-copy\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.686656 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685327 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-log-socket\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.686656 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685352 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-metrics-certs\") pod \"network-metrics-daemon-skbch\" (UID: \"5ee0c301-12eb-4c68-9990-6ceafb2c90fa\") " pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:54:43.686656 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685380 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-66pbg\" (UniqueName: \"kubernetes.io/projected/3b778703-acf8-4977-a1b1-2f5f9042d2b3-kube-api-access-66pbg\") pod \"network-check-target-rxzxp\" (UID: \"3b778703-acf8-4977-a1b1-2f5f9042d2b3\") " pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:54:43.686656 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685388 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0a1071e0-9724-4546-8b54-44bce04c35c2-host\") pod \"node-ca-sfsvt\" (UID: \"0a1071e0-9724-4546-8b54-44bce04c35c2\") " pod="openshift-image-registry/node-ca-sfsvt" May 21 00:54:43.686656 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685158 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-host-var-lib-cni-multus\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.686656 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685405 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-host-var-lib-cni-bin\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.686656 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685439 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-host-var-lib-cni-bin\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.686656 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685445 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5da3aae9-eaf5-4328-afed-6595832fc2e1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-spsr8\" (UID: \"5da3aae9-eaf5-4328-afed-6595832fc2e1\") " pod="openshift-multus/multus-additional-cni-plugins-spsr8" May 21 00:54:43.686656 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685450 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-etc-kubernetes\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.686656 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685479 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/acd173d4-75da-4564-ac8e-84996bb1742d-etc-kubernetes\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.686656 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685458 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5da3aae9-eaf5-4328-afed-6595832fc2e1-cni-binary-copy\") pod \"multus-additional-cni-plugins-spsr8\" (UID: \"5da3aae9-eaf5-4328-afed-6595832fc2e1\") " pod="openshift-multus/multus-additional-cni-plugins-spsr8" May 21 00:54:43.686656 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685508 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9ff27421-0ba9-40f6-98db-150fbcca1a5e-konnectivity-ca\") pod \"konnectivity-agent-h7nwx\" (UID: \"9ff27421-0ba9-40f6-98db-150fbcca1a5e\") " pod="kube-system/konnectivity-agent-h7nwx" May 21 00:54:43.686656 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:43.685566 2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 21 00:54:43.686656 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685598 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5da3aae9-eaf5-4328-afed-6595832fc2e1-system-cni-dir\") pod \"multus-additional-cni-plugins-spsr8\" (UID: \"5da3aae9-eaf5-4328-afed-6595832fc2e1\") " pod="openshift-multus/multus-additional-cni-plugins-spsr8" May 21 00:54:43.686656 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685640 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5da3aae9-eaf5-4328-afed-6595832fc2e1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-spsr8\" (UID: \"5da3aae9-eaf5-4328-afed-6595832fc2e1\") " pod="openshift-multus/multus-additional-cni-plugins-spsr8" May 21 00:54:43.687293 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:43.685688 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-metrics-certs podName:5ee0c301-12eb-4c68-9990-6ceafb2c90fa nodeName:}" failed. No retries permitted until 2026-05-21 00:54:44.185642052 +0000 UTC m=+3.082587364 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-metrics-certs") pod "network-metrics-daemon-skbch" (UID: "5ee0c301-12eb-4c68-9990-6ceafb2c90fa") : object "openshift-multus"/"metrics-daemon-secret" not registered May 21 00:54:43.687293 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685690 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5da3aae9-eaf5-4328-afed-6595832fc2e1-system-cni-dir\") pod \"multus-additional-cni-plugins-spsr8\" (UID: \"5da3aae9-eaf5-4328-afed-6595832fc2e1\") " pod="openshift-multus/multus-additional-cni-plugins-spsr8" May 21 00:54:43.687293 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685720 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5da3aae9-eaf5-4328-afed-6595832fc2e1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-spsr8\" (UID: \"5da3aae9-eaf5-4328-afed-6595832fc2e1\") " pod="openshift-multus/multus-additional-cni-plugins-spsr8" May 21 00:54:43.687293 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685750 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmvfl\" (UniqueName: \"kubernetes.io/projected/7d5e55de-c445-48f6-8911-e4f1a11104de-kube-api-access-jmvfl\") pod \"iptables-alerter-qdnd9\" (UID: \"7d5e55de-c445-48f6-8911-e4f1a11104de\") " pod="openshift-network-operator/iptables-alerter-qdnd9" May 21 00:54:43.687293 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685774 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-node-log\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.687293 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685798 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tjfp\" (UniqueName: \"kubernetes.io/projected/8adbdb60-a625-47a1-8252-779cd35e6a35-kube-api-access-7tjfp\") pod \"aws-ebs-csi-driver-node-vvqd9\" (UID: \"8adbdb60-a625-47a1-8252-779cd35e6a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" May 21 00:54:43.687293 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.685917 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5da3aae9-eaf5-4328-afed-6595832fc2e1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-spsr8\" (UID: \"5da3aae9-eaf5-4328-afed-6595832fc2e1\") " pod="openshift-multus/multus-additional-cni-plugins-spsr8" May 21 00:54:43.687293 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.686035 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9ff27421-0ba9-40f6-98db-150fbcca1a5e-konnectivity-ca\") pod \"konnectivity-agent-h7nwx\" (UID: \"9ff27421-0ba9-40f6-98db-150fbcca1a5e\") " pod="kube-system/konnectivity-agent-h7nwx" May 21 00:54:43.687899 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.687877 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9ff27421-0ba9-40f6-98db-150fbcca1a5e-agent-certs\") pod \"konnectivity-agent-h7nwx\" (UID: \"9ff27421-0ba9-40f6-98db-150fbcca1a5e\") " pod="kube-system/konnectivity-agent-h7nwx" May 21 00:54:43.690073 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:43.689832 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 21 00:54:43.690073 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:43.689852 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 21 00:54:43.690073 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:43.689864 2549 projected.go:194] Error preparing data for projected volume kube-api-access-66pbg for pod openshift-network-diagnostics/network-check-target-rxzxp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 00:54:43.690073 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:43.689919 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b778703-acf8-4977-a1b1-2f5f9042d2b3-kube-api-access-66pbg podName:3b778703-acf8-4977-a1b1-2f5f9042d2b3 nodeName:}" failed. No retries permitted until 2026-05-21 00:54:44.189902458 +0000 UTC m=+3.086847751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-66pbg" (UniqueName: "kubernetes.io/projected/3b778703-acf8-4977-a1b1-2f5f9042d2b3-kube-api-access-66pbg") pod "network-check-target-rxzxp" (UID: "3b778703-acf8-4977-a1b1-2f5f9042d2b3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 00:54:43.690971 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.690920 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-298bc\" (UniqueName: \"kubernetes.io/projected/0a1071e0-9724-4546-8b54-44bce04c35c2-kube-api-access-298bc\") pod \"node-ca-sfsvt\" (UID: \"0a1071e0-9724-4546-8b54-44bce04c35c2\") " pod="openshift-image-registry/node-ca-sfsvt" May 21 00:54:43.691754 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.691264 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m22nq\" (UniqueName: \"kubernetes.io/projected/31374768-4e56-404b-8d22-7e2136138fa4-kube-api-access-m22nq\") pod \"node-resolver-96x9m\" (UID: \"31374768-4e56-404b-8d22-7e2136138fa4\") " pod="openshift-dns/node-resolver-96x9m" May 21 00:54:43.691754 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.691692 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2brz9\" (UniqueName: \"kubernetes.io/projected/acd173d4-75da-4564-ac8e-84996bb1742d-kube-api-access-2brz9\") pod \"multus-sh44w\" (UID: \"acd173d4-75da-4564-ac8e-84996bb1742d\") " pod="openshift-multus/multus-sh44w" May 21 00:54:43.692391 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.692369 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7fvr\" (UniqueName: \"kubernetes.io/projected/5da3aae9-eaf5-4328-afed-6595832fc2e1-kube-api-access-c7fvr\") pod \"multus-additional-cni-plugins-spsr8\" (UID: \"5da3aae9-eaf5-4328-afed-6595832fc2e1\") " pod="openshift-multus/multus-additional-cni-plugins-spsr8" May 21 00:54:43.692602 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.692580 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cvsw\" (UniqueName: \"kubernetes.io/projected/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-kube-api-access-9cvsw\") pod \"network-metrics-daemon-skbch\" (UID: \"5ee0c301-12eb-4c68-9990-6ceafb2c90fa\") " pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:54:43.712621 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.712599 2549 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" May 21 00:54:43.773407 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.773381 2549 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 21 00:54:43.786254 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786212 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.786254 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786254 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-host-kubelet\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.786431 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786267 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.786431 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786280 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-host-cni-bin\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.786431 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786315 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-host-cni-bin\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.786431 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786316 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7d5e55de-c445-48f6-8911-e4f1a11104de-iptables-alerter-script\") pod \"iptables-alerter-qdnd9\" (UID: \"7d5e55de-c445-48f6-8911-e4f1a11104de\") " pod="openshift-network-operator/iptables-alerter-qdnd9" May 21 00:54:43.786431 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786348 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-host-kubelet\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.786431 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786358 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-host-run-netns\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.786431 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786385 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-host-run-netns\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.786431 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786410 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-run-ovn\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.786843 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786434 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-env-overrides\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.786843 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786470 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-run-ovn\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.786843 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786482 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mwj7k\" (UniqueName: \"kubernetes.io/projected/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-kube-api-access-mwj7k\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.786843 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786508 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7d5e55de-c445-48f6-8911-e4f1a11104de-host-slash\") pod \"iptables-alerter-qdnd9\" (UID: \"7d5e55de-c445-48f6-8911-e4f1a11104de\") " pod="openshift-network-operator/iptables-alerter-qdnd9" May 21 00:54:43.786843 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786530 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8adbdb60-a625-47a1-8252-779cd35e6a35-device-dir\") pod \"aws-ebs-csi-driver-node-vvqd9\" (UID: \"8adbdb60-a625-47a1-8252-779cd35e6a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" May 21 00:54:43.786843 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786553 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8adbdb60-a625-47a1-8252-779cd35e6a35-sys-fs\") pod \"aws-ebs-csi-driver-node-vvqd9\" (UID: \"8adbdb60-a625-47a1-8252-779cd35e6a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" May 21 00:54:43.786843 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786582 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-host-run-ovn-kubernetes\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.786843 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786579 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7d5e55de-c445-48f6-8911-e4f1a11104de-host-slash\") pod \"iptables-alerter-qdnd9\" (UID: \"7d5e55de-c445-48f6-8911-e4f1a11104de\") " pod="openshift-network-operator/iptables-alerter-qdnd9" May 21 00:54:43.786843 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786603 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-host-cni-netd\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.786843 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786626 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8adbdb60-a625-47a1-8252-779cd35e6a35-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vvqd9\" (UID: \"8adbdb60-a625-47a1-8252-779cd35e6a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" May 21 00:54:43.786843 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786647 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8adbdb60-a625-47a1-8252-779cd35e6a35-sys-fs\") pod \"aws-ebs-csi-driver-node-vvqd9\" (UID: \"8adbdb60-a625-47a1-8252-779cd35e6a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" May 21 00:54:43.786843 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786651 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-run-systemd\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.786843 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786684 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-run-systemd\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.786843 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786685 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-host-run-ovn-kubernetes\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.786843 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786687 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-ovn-node-metrics-cert\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.786843 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786731 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-etc-kubernetes\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.786843 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786737 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8adbdb60-a625-47a1-8252-779cd35e6a35-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vvqd9\" (UID: \"8adbdb60-a625-47a1-8252-779cd35e6a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" May 21 00:54:43.787792 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786732 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8adbdb60-a625-47a1-8252-779cd35e6a35-device-dir\") pod \"aws-ebs-csi-driver-node-vvqd9\" (UID: \"8adbdb60-a625-47a1-8252-779cd35e6a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" May 21 00:54:43.787792 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786761 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6mb8w\" (UniqueName: \"kubernetes.io/projected/96547212-630f-43e2-9662-5938454d35ec-kube-api-access-6mb8w\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.787792 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786773 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-host-cni-netd\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.787792 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786785 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7d5e55de-c445-48f6-8911-e4f1a11104de-iptables-alerter-script\") pod \"iptables-alerter-qdnd9\" (UID: \"7d5e55de-c445-48f6-8911-e4f1a11104de\") " pod="openshift-network-operator/iptables-alerter-qdnd9" May 21 00:54:43.787792 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786792 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-systemd-units\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.787792 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786801 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-etc-kubernetes\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.787792 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786817 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-var-lib-openvswitch\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.787792 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786841 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-ovnkube-script-lib\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.787792 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786846 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-systemd-units\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.787792 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786863 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-run-openvswitch\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.787792 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786887 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/96547212-630f-43e2-9662-5938454d35ec-etc-tuned\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.787792 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786893 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-var-lib-openvswitch\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.787792 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786910 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/96547212-630f-43e2-9662-5938454d35ec-tmp\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.787792 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786911 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-env-overrides\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.787792 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786931 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-run-openvswitch\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.787792 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786951 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-log-socket\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.787792 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.786994 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-log-socket\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.788674 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787013 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jmvfl\" (UniqueName: \"kubernetes.io/projected/7d5e55de-c445-48f6-8911-e4f1a11104de-kube-api-access-jmvfl\") pod \"iptables-alerter-qdnd9\" (UID: \"7d5e55de-c445-48f6-8911-e4f1a11104de\") " pod="openshift-network-operator/iptables-alerter-qdnd9" May 21 00:54:43.788674 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787042 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-node-log\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.788674 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787090 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7tjfp\" (UniqueName: \"kubernetes.io/projected/8adbdb60-a625-47a1-8252-779cd35e6a35-kube-api-access-7tjfp\") pod \"aws-ebs-csi-driver-node-vvqd9\" (UID: \"8adbdb60-a625-47a1-8252-779cd35e6a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" May 21 00:54:43.788674 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787115 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-etc-sysconfig\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.788674 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787140 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-sys\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.788674 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787162 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-lib-modules\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.788674 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787206 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-host-slash\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.788674 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787203 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-node-log\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.788674 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787232 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-ovnkube-config\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.788674 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787259 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-etc-systemd\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.788674 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787284 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-host\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.788674 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787284 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-sys\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.788674 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787285 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-host-slash\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.788674 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787326 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8adbdb60-a625-47a1-8252-779cd35e6a35-registration-dir\") pod \"aws-ebs-csi-driver-node-vvqd9\" (UID: \"8adbdb60-a625-47a1-8252-779cd35e6a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" May 21 00:54:43.788674 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787359 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-etc-modprobe-d\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.788674 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787377 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-ovnkube-script-lib\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.788674 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787389 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-var-lib-kubelet\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.789461 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787398 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-etc-sysconfig\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.789461 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787418 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8adbdb60-a625-47a1-8252-779cd35e6a35-socket-dir\") pod \"aws-ebs-csi-driver-node-vvqd9\" (UID: \"8adbdb60-a625-47a1-8252-779cd35e6a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" May 21 00:54:43.789461 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787444 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8adbdb60-a625-47a1-8252-779cd35e6a35-etc-selinux\") pod \"aws-ebs-csi-driver-node-vvqd9\" (UID: \"8adbdb60-a625-47a1-8252-779cd35e6a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" May 21 00:54:43.789461 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787448 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-etc-systemd\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.789461 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787475 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-etc-sysctl-d\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.789461 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787485 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-etc-modprobe-d\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.789461 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787503 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-run\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.789461 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787506 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8adbdb60-a625-47a1-8252-779cd35e6a35-etc-selinux\") pod \"aws-ebs-csi-driver-node-vvqd9\" (UID: \"8adbdb60-a625-47a1-8252-779cd35e6a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" May 21 00:54:43.789461 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787362 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-lib-modules\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.789461 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787535 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-etc-openvswitch\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.789461 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787536 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8adbdb60-a625-47a1-8252-779cd35e6a35-registration-dir\") pod \"aws-ebs-csi-driver-node-vvqd9\" (UID: \"8adbdb60-a625-47a1-8252-779cd35e6a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" May 21 00:54:43.789461 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787560 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-etc-sysctl-conf\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.789461 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787571 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-var-lib-kubelet\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.789461 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787620 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-etc-openvswitch\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.789461 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787622 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-host\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.789461 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787644 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8adbdb60-a625-47a1-8252-779cd35e6a35-socket-dir\") pod \"aws-ebs-csi-driver-node-vvqd9\" (UID: \"8adbdb60-a625-47a1-8252-779cd35e6a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" May 21 00:54:43.789461 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787645 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-run\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.789461 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787695 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-etc-sysctl-d\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.790421 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787722 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-ovnkube-config\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.790421 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.787732 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/96547212-630f-43e2-9662-5938454d35ec-etc-sysctl-conf\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.790421 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.789028 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-ovn-node-metrics-cert\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.790421 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.789302 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/96547212-630f-43e2-9662-5938454d35ec-tmp\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.790421 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.789684 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/96547212-630f-43e2-9662-5938454d35ec-etc-tuned\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.794094 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.794075 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwj7k\" (UniqueName: \"kubernetes.io/projected/12a9408a-b799-4b3c-a790-a4d6e6d67cbd-kube-api-access-mwj7k\") pod \"ovnkube-node-kgxp9\" (UID: \"12a9408a-b799-4b3c-a790-a4d6e6d67cbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.794275 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.794256 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mb8w\" (UniqueName: \"kubernetes.io/projected/96547212-630f-43e2-9662-5938454d35ec-kube-api-access-6mb8w\") pod \"tuned-gvmxz\" (UID: \"96547212-630f-43e2-9662-5938454d35ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:43.794621 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.794606 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tjfp\" (UniqueName: \"kubernetes.io/projected/8adbdb60-a625-47a1-8252-779cd35e6a35-kube-api-access-7tjfp\") pod \"aws-ebs-csi-driver-node-vvqd9\" (UID: \"8adbdb60-a625-47a1-8252-779cd35e6a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" May 21 00:54:43.794673 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.794655 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmvfl\" (UniqueName: \"kubernetes.io/projected/7d5e55de-c445-48f6-8911-e4f1a11104de-kube-api-access-jmvfl\") pod \"iptables-alerter-qdnd9\" (UID: \"7d5e55de-c445-48f6-8911-e4f1a11104de\") " pod="openshift-network-operator/iptables-alerter-qdnd9" May 21 00:54:43.875758 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.875651 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sh44w" May 21 00:54:43.883325 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.883294 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sfsvt" May 21 00:54:43.891835 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.891813 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-spsr8" May 21 00:54:43.896462 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.896440 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h7nwx" May 21 00:54:43.902992 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.902973 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-96x9m" May 21 00:54:43.909489 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.909468 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qdnd9" May 21 00:54:43.916087 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.916055 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:54:43.923664 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.923647 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" May 21 00:54:43.928269 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:43.928253 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" May 21 00:54:44.190945 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:44.190862 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-metrics-certs\") pod \"network-metrics-daemon-skbch\" (UID: \"5ee0c301-12eb-4c68-9990-6ceafb2c90fa\") " pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:54:44.190945 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:44.190911 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-66pbg\" (UniqueName: \"kubernetes.io/projected/3b778703-acf8-4977-a1b1-2f5f9042d2b3-kube-api-access-66pbg\") pod \"network-check-target-rxzxp\" (UID: \"3b778703-acf8-4977-a1b1-2f5f9042d2b3\") " pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:54:44.191179 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:44.191023 2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 21 00:54:44.191179 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:44.191040 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 21 00:54:44.191179 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:44.191073 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 21 00:54:44.191179 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:44.191087 2549 projected.go:194] Error preparing data for projected volume kube-api-access-66pbg for pod openshift-network-diagnostics/network-check-target-rxzxp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 00:54:44.191179 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:44.191108 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-metrics-certs podName:5ee0c301-12eb-4c68-9990-6ceafb2c90fa nodeName:}" failed. No retries permitted until 2026-05-21 00:54:45.191089733 +0000 UTC m=+4.088035031 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-metrics-certs") pod "network-metrics-daemon-skbch" (UID: "5ee0c301-12eb-4c68-9990-6ceafb2c90fa") : object "openshift-multus"/"metrics-daemon-secret" not registered May 21 00:54:44.191179 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:44.191135 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b778703-acf8-4977-a1b1-2f5f9042d2b3-kube-api-access-66pbg podName:3b778703-acf8-4977-a1b1-2f5f9042d2b3 nodeName:}" failed. No retries permitted until 2026-05-21 00:54:45.191119793 +0000 UTC m=+4.088065085 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-66pbg" (UniqueName: "kubernetes.io/projected/3b778703-acf8-4977-a1b1-2f5f9042d2b3-kube-api-access-66pbg") pod "network-check-target-rxzxp" (UID: "3b778703-acf8-4977-a1b1-2f5f9042d2b3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 00:54:44.315218 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:44.315188 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ff27421_0ba9_40f6_98db_150fbcca1a5e.slice/crio-771fbefc0afafae7ce6cb0fde6e62198884fd03744f1fd9cb965d8c45ce12d62 WatchSource:0}: Error finding container 771fbefc0afafae7ce6cb0fde6e62198884fd03744f1fd9cb965d8c45ce12d62: Status 404 returned error can't find the container with id 771fbefc0afafae7ce6cb0fde6e62198884fd03744f1fd9cb965d8c45ce12d62 May 21 00:54:44.316830 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:44.316816 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96547212_630f_43e2_9662_5938454d35ec.slice/crio-020b604019eaf9d192d0db786736d61770f1ffb1a0dad8d21d42fcbbaf873533 WatchSource:0}: Error finding container 020b604019eaf9d192d0db786736d61770f1ffb1a0dad8d21d42fcbbaf873533: Status 404 returned error can't find the container with id 020b604019eaf9d192d0db786736d61770f1ffb1a0dad8d21d42fcbbaf873533 May 21 00:54:44.320041 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:44.320007 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31374768_4e56_404b_8d22_7e2136138fa4.slice/crio-dc15089c3b2a773c4e44752a881acb53f5112347fa19ea30d8265c452ed14e56 WatchSource:0}: Error finding container dc15089c3b2a773c4e44752a881acb53f5112347fa19ea30d8265c452ed14e56: Status 404 returned error can't find the container with id dc15089c3b2a773c4e44752a881acb53f5112347fa19ea30d8265c452ed14e56 May 21 00:54:44.320689 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:44.320661 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12a9408a_b799_4b3c_a790_a4d6e6d67cbd.slice/crio-99df5c5aedf0dbe2533a911804a054555648506dd6288a6c6114e081de9b94b9 WatchSource:0}: Error finding container 99df5c5aedf0dbe2533a911804a054555648506dd6288a6c6114e081de9b94b9: Status 404 returned error can't find the container with id 99df5c5aedf0dbe2533a911804a054555648506dd6288a6c6114e081de9b94b9 May 21 00:54:44.321640 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:44.321616 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8adbdb60_a625_47a1_8252_779cd35e6a35.slice/crio-b20e3fb5f3ab0430fe6dd66a643f64b78aeefc53fb49ca4b70bca5272dcd8337 WatchSource:0}: Error finding container b20e3fb5f3ab0430fe6dd66a643f64b78aeefc53fb49ca4b70bca5272dcd8337: Status 404 returned error can't find the container with id b20e3fb5f3ab0430fe6dd66a643f64b78aeefc53fb49ca4b70bca5272dcd8337 May 21 00:54:44.322388 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:44.322366 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacd173d4_75da_4564_ac8e_84996bb1742d.slice/crio-858cc287511473c3eb3be0dc72e6f53bfd2ea28c273cc957671e5cda177a7112 WatchSource:0}: Error finding container 858cc287511473c3eb3be0dc72e6f53bfd2ea28c273cc957671e5cda177a7112: Status 404 returned error can't find the container with id 858cc287511473c3eb3be0dc72e6f53bfd2ea28c273cc957671e5cda177a7112 May 21 00:54:44.323303 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:44.323021 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d5e55de_c445_48f6_8911_e4f1a11104de.slice/crio-53df6f55e404347bbf2c7963a40f1738e447c12a2c029c9cf668a675d0a89809 WatchSource:0}: Error finding container 53df6f55e404347bbf2c7963a40f1738e447c12a2c029c9cf668a675d0a89809: Status 404 returned error can't find the container with id 53df6f55e404347bbf2c7963a40f1738e447c12a2c029c9cf668a675d0a89809 May 21 00:54:44.325350 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:54:44.325144 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5da3aae9_eaf5_4328_afed_6595832fc2e1.slice/crio-f5a09bf18bcb746ddf00d6a51edc65d0be95a8a8925c82508857014afbfb066e WatchSource:0}: Error finding container f5a09bf18bcb746ddf00d6a51edc65d0be95a8a8925c82508857014afbfb066e: Status 404 returned error can't find the container with id f5a09bf18bcb746ddf00d6a51edc65d0be95a8a8925c82508857014afbfb066e May 21 00:54:44.607636 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:44.607602 2549 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-20 00:49:42 +0000 UTC" deadline="2028-02-22 12:52:31.090739924 +0000 UTC" May 21 00:54:44.607636 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:44.607631 2549 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15419h57m46.483110917s" May 21 00:54:44.676939 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:44.676910 2549 generic.go:358] "Generic (PLEG): container finished" podID="f0e5357ad01f42d530056be0164fe34c" containerID="280f2a84c685846736ccc8c6c68fcdd62da3653ff0ee522bbb6224a3e0ceaa53" exitCode=0 May 21 00:54:44.677163 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:44.676993 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal" event={"ID":"f0e5357ad01f42d530056be0164fe34c","Type":"ContainerDied","Data":"280f2a84c685846736ccc8c6c68fcdd62da3653ff0ee522bbb6224a3e0ceaa53"} May 21 00:54:44.678777 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:44.678745 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sfsvt" event={"ID":"0a1071e0-9724-4546-8b54-44bce04c35c2","Type":"ContainerStarted","Data":"d39dd92a3015cd8e36854466fb6c528bcf332ad8f301f866b020e0d0e92d355c"} May 21 00:54:44.680638 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:44.680262 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-spsr8" event={"ID":"5da3aae9-eaf5-4328-afed-6595832fc2e1","Type":"ContainerStarted","Data":"f5a09bf18bcb746ddf00d6a51edc65d0be95a8a8925c82508857014afbfb066e"} May 21 00:54:44.681897 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:44.681872 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sh44w" event={"ID":"acd173d4-75da-4564-ac8e-84996bb1742d","Type":"ContainerStarted","Data":"858cc287511473c3eb3be0dc72e6f53bfd2ea28c273cc957671e5cda177a7112"} May 21 00:54:44.684828 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:44.684790 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" event={"ID":"8adbdb60-a625-47a1-8252-779cd35e6a35","Type":"ContainerStarted","Data":"b20e3fb5f3ab0430fe6dd66a643f64b78aeefc53fb49ca4b70bca5272dcd8337"} May 21 00:54:44.686369 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:44.686343 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" event={"ID":"12a9408a-b799-4b3c-a790-a4d6e6d67cbd","Type":"ContainerStarted","Data":"99df5c5aedf0dbe2533a911804a054555648506dd6288a6c6114e081de9b94b9"} May 21 00:54:44.687549 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:44.687504 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" event={"ID":"96547212-630f-43e2-9662-5938454d35ec","Type":"ContainerStarted","Data":"020b604019eaf9d192d0db786736d61770f1ffb1a0dad8d21d42fcbbaf873533"} May 21 00:54:44.691182 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:44.689986 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h7nwx" event={"ID":"9ff27421-0ba9-40f6-98db-150fbcca1a5e","Type":"ContainerStarted","Data":"771fbefc0afafae7ce6cb0fde6e62198884fd03744f1fd9cb965d8c45ce12d62"} May 21 00:54:44.691667 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:44.691567 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-101.ec2.internal" event={"ID":"97c57dc15067251bbf5d4474732b6483","Type":"ContainerStarted","Data":"0dc530ef5aa5930a1051f46f55a679b4c5e5cf9e07a25317695adf76586badc7"} May 21 00:54:44.693027 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:44.693004 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qdnd9" event={"ID":"7d5e55de-c445-48f6-8911-e4f1a11104de","Type":"ContainerStarted","Data":"53df6f55e404347bbf2c7963a40f1738e447c12a2c029c9cf668a675d0a89809"} May 21 00:54:44.695365 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:44.695341 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-96x9m" event={"ID":"31374768-4e56-404b-8d22-7e2136138fa4","Type":"ContainerStarted","Data":"dc15089c3b2a773c4e44752a881acb53f5112347fa19ea30d8265c452ed14e56"} May 21 00:54:44.708085 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:44.706706 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-101.ec2.internal" podStartSLOduration=2.706691651 podStartE2EDuration="2.706691651s" podCreationTimestamp="2026-05-21 00:54:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 00:54:44.705814907 +0000 UTC m=+3.602760219" watchObservedRunningTime="2026-05-21 00:54:44.706691651 +0000 UTC m=+3.603636963" May 21 00:54:45.200498 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:45.200213 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-metrics-certs\") pod \"network-metrics-daemon-skbch\" (UID: \"5ee0c301-12eb-4c68-9990-6ceafb2c90fa\") " pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:54:45.200498 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:45.200259 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-66pbg\" (UniqueName: \"kubernetes.io/projected/3b778703-acf8-4977-a1b1-2f5f9042d2b3-kube-api-access-66pbg\") pod \"network-check-target-rxzxp\" (UID: \"3b778703-acf8-4977-a1b1-2f5f9042d2b3\") " pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:54:45.200498 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:45.200413 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 21 00:54:45.200498 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:45.200432 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 21 00:54:45.200498 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:45.200444 2549 projected.go:194] Error preparing data for projected volume kube-api-access-66pbg for pod openshift-network-diagnostics/network-check-target-rxzxp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 00:54:45.200498 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:45.200501 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b778703-acf8-4977-a1b1-2f5f9042d2b3-kube-api-access-66pbg podName:3b778703-acf8-4977-a1b1-2f5f9042d2b3 nodeName:}" failed. No retries permitted until 2026-05-21 00:54:47.200481496 +0000 UTC m=+6.097426787 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-66pbg" (UniqueName: "kubernetes.io/projected/3b778703-acf8-4977-a1b1-2f5f9042d2b3-kube-api-access-66pbg") pod "network-check-target-rxzxp" (UID: "3b778703-acf8-4977-a1b1-2f5f9042d2b3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 00:54:45.200930 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:45.200897 2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 21 00:54:45.200993 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:45.200950 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-metrics-certs podName:5ee0c301-12eb-4c68-9990-6ceafb2c90fa nodeName:}" failed. No retries permitted until 2026-05-21 00:54:47.200933588 +0000 UTC m=+6.097878894 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-metrics-certs") pod "network-metrics-daemon-skbch" (UID: "5ee0c301-12eb-4c68-9990-6ceafb2c90fa") : object "openshift-multus"/"metrics-daemon-secret" not registered May 21 00:54:45.670329 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:45.670288 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:54:45.670755 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:45.670425 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skbch" podUID="5ee0c301-12eb-4c68-9990-6ceafb2c90fa" May 21 00:54:45.671824 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:45.671431 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:54:45.671824 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:45.671524 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxzxp" podUID="3b778703-acf8-4977-a1b1-2f5f9042d2b3" May 21 00:54:45.709180 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:45.709129 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal" event={"ID":"f0e5357ad01f42d530056be0164fe34c","Type":"ContainerStarted","Data":"8a072afa8ccef611f7f9a7b5519f3fc3fb6d52c63c156d58b8eb548b00087317"} May 21 00:54:47.218105 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:47.218071 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-metrics-certs\") pod \"network-metrics-daemon-skbch\" (UID: \"5ee0c301-12eb-4c68-9990-6ceafb2c90fa\") " pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:54:47.218677 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:47.218121 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-66pbg\" (UniqueName: \"kubernetes.io/projected/3b778703-acf8-4977-a1b1-2f5f9042d2b3-kube-api-access-66pbg\") pod \"network-check-target-rxzxp\" (UID: \"3b778703-acf8-4977-a1b1-2f5f9042d2b3\") " pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:54:47.218677 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:47.218304 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 21 00:54:47.218677 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:47.218323 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 21 00:54:47.218677 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:47.218335 2549 projected.go:194] Error preparing data for projected volume kube-api-access-66pbg for pod openshift-network-diagnostics/network-check-target-rxzxp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 00:54:47.218677 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:47.218390 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b778703-acf8-4977-a1b1-2f5f9042d2b3-kube-api-access-66pbg podName:3b778703-acf8-4977-a1b1-2f5f9042d2b3 nodeName:}" failed. No retries permitted until 2026-05-21 00:54:51.218370617 +0000 UTC m=+10.115315926 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-66pbg" (UniqueName: "kubernetes.io/projected/3b778703-acf8-4977-a1b1-2f5f9042d2b3-kube-api-access-66pbg") pod "network-check-target-rxzxp" (UID: "3b778703-acf8-4977-a1b1-2f5f9042d2b3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 00:54:47.218677 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:47.218538 2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 21 00:54:47.218677 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:47.218581 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-metrics-certs podName:5ee0c301-12eb-4c68-9990-6ceafb2c90fa nodeName:}" failed. No retries permitted until 2026-05-21 00:54:51.218568181 +0000 UTC m=+10.115513478 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-metrics-certs") pod "network-metrics-daemon-skbch" (UID: "5ee0c301-12eb-4c68-9990-6ceafb2c90fa") : object "openshift-multus"/"metrics-daemon-secret" not registered May 21 00:54:47.667220 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:47.667193 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:54:47.667382 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:47.667335 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skbch" podUID="5ee0c301-12eb-4c68-9990-6ceafb2c90fa" May 21 00:54:47.667382 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:47.667193 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:54:47.667509 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:47.667433 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxzxp" podUID="3b778703-acf8-4977-a1b1-2f5f9042d2b3" May 21 00:54:49.667597 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:49.667555 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:54:49.668196 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:49.667699 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skbch" podUID="5ee0c301-12eb-4c68-9990-6ceafb2c90fa" May 21 00:54:49.668196 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:49.667735 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:54:49.668196 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:49.667836 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxzxp" podUID="3b778703-acf8-4977-a1b1-2f5f9042d2b3" May 21 00:54:51.249342 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:51.249306 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-metrics-certs\") pod \"network-metrics-daemon-skbch\" (UID: \"5ee0c301-12eb-4c68-9990-6ceafb2c90fa\") " pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:54:51.249813 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:51.249357 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-66pbg\" (UniqueName: \"kubernetes.io/projected/3b778703-acf8-4977-a1b1-2f5f9042d2b3-kube-api-access-66pbg\") pod \"network-check-target-rxzxp\" (UID: \"3b778703-acf8-4977-a1b1-2f5f9042d2b3\") " pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:54:51.249813 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:51.249512 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 21 00:54:51.249813 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:51.249530 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 21 00:54:51.249813 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:51.249544 2549 projected.go:194] Error preparing data for projected volume kube-api-access-66pbg for pod openshift-network-diagnostics/network-check-target-rxzxp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 00:54:51.249813 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:51.249601 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b778703-acf8-4977-a1b1-2f5f9042d2b3-kube-api-access-66pbg podName:3b778703-acf8-4977-a1b1-2f5f9042d2b3 nodeName:}" failed. No retries permitted until 2026-05-21 00:54:59.249582874 +0000 UTC m=+18.146528178 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-66pbg" (UniqueName: "kubernetes.io/projected/3b778703-acf8-4977-a1b1-2f5f9042d2b3-kube-api-access-66pbg") pod "network-check-target-rxzxp" (UID: "3b778703-acf8-4977-a1b1-2f5f9042d2b3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 00:54:51.250083 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:51.249994 2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 21 00:54:51.250083 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:51.250039 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-metrics-certs podName:5ee0c301-12eb-4c68-9990-6ceafb2c90fa nodeName:}" failed. No retries permitted until 2026-05-21 00:54:59.250024541 +0000 UTC m=+18.146969832 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-metrics-certs") pod "network-metrics-daemon-skbch" (UID: "5ee0c301-12eb-4c68-9990-6ceafb2c90fa") : object "openshift-multus"/"metrics-daemon-secret" not registered May 21 00:54:51.668050 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:51.667980 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:54:51.668195 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:51.668112 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skbch" podUID="5ee0c301-12eb-4c68-9990-6ceafb2c90fa" May 21 00:54:51.668430 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:51.668409 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:54:51.668531 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:51.668511 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxzxp" podUID="3b778703-acf8-4977-a1b1-2f5f9042d2b3" May 21 00:54:52.063657 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:52.063598 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal" podStartSLOduration=10.063579872 podStartE2EDuration="10.063579872s" podCreationTimestamp="2026-05-21 00:54:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 00:54:45.723511458 +0000 UTC m=+4.620456770" watchObservedRunningTime="2026-05-21 00:54:52.063579872 +0000 UTC m=+10.960525185" May 21 00:54:52.064135 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:52.064116 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-n4j7l"] May 21 00:54:52.070189 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:52.070168 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:54:52.070309 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:52.070253 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4j7l" podUID="8f8b563f-d153-48f5-ab5e-836807f76cad" May 21 00:54:52.155664 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:52.155631 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f8b563f-d153-48f5-ab5e-836807f76cad-original-pull-secret\") pod \"global-pull-secret-syncer-n4j7l\" (UID: \"8f8b563f-d153-48f5-ab5e-836807f76cad\") " pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:54:52.155791 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:52.155685 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8f8b563f-d153-48f5-ab5e-836807f76cad-kubelet-config\") pod \"global-pull-secret-syncer-n4j7l\" (UID: \"8f8b563f-d153-48f5-ab5e-836807f76cad\") " pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:54:52.155791 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:52.155718 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8f8b563f-d153-48f5-ab5e-836807f76cad-dbus\") pod \"global-pull-secret-syncer-n4j7l\" (UID: \"8f8b563f-d153-48f5-ab5e-836807f76cad\") " pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:54:52.256318 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:52.256289 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f8b563f-d153-48f5-ab5e-836807f76cad-original-pull-secret\") pod \"global-pull-secret-syncer-n4j7l\" (UID: \"8f8b563f-d153-48f5-ab5e-836807f76cad\") " pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:54:52.256710 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:52.256340 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8f8b563f-d153-48f5-ab5e-836807f76cad-kubelet-config\") pod \"global-pull-secret-syncer-n4j7l\" (UID: \"8f8b563f-d153-48f5-ab5e-836807f76cad\") " pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:54:52.256710 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:52.256367 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8f8b563f-d153-48f5-ab5e-836807f76cad-dbus\") pod \"global-pull-secret-syncer-n4j7l\" (UID: \"8f8b563f-d153-48f5-ab5e-836807f76cad\") " pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:54:52.256710 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:52.256397 2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 21 00:54:52.256710 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:52.256451 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8f8b563f-d153-48f5-ab5e-836807f76cad-dbus\") pod \"global-pull-secret-syncer-n4j7l\" (UID: \"8f8b563f-d153-48f5-ab5e-836807f76cad\") " pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:54:52.256710 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:52.256457 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f8b563f-d153-48f5-ab5e-836807f76cad-original-pull-secret podName:8f8b563f-d153-48f5-ab5e-836807f76cad nodeName:}" failed. No retries permitted until 2026-05-21 00:54:52.756439068 +0000 UTC m=+11.653384371 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f8b563f-d153-48f5-ab5e-836807f76cad-original-pull-secret") pod "global-pull-secret-syncer-n4j7l" (UID: "8f8b563f-d153-48f5-ab5e-836807f76cad") : object "kube-system"/"original-pull-secret" not registered May 21 00:54:52.256710 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:52.256501 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8f8b563f-d153-48f5-ab5e-836807f76cad-kubelet-config\") pod \"global-pull-secret-syncer-n4j7l\" (UID: \"8f8b563f-d153-48f5-ab5e-836807f76cad\") " pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:54:52.759850 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:52.759813 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f8b563f-d153-48f5-ab5e-836807f76cad-original-pull-secret\") pod \"global-pull-secret-syncer-n4j7l\" (UID: \"8f8b563f-d153-48f5-ab5e-836807f76cad\") " pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:54:52.760027 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:52.759946 2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 21 00:54:52.760027 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:52.760011 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f8b563f-d153-48f5-ab5e-836807f76cad-original-pull-secret podName:8f8b563f-d153-48f5-ab5e-836807f76cad nodeName:}" failed. No retries permitted until 2026-05-21 00:54:53.759993567 +0000 UTC m=+12.656938859 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f8b563f-d153-48f5-ab5e-836807f76cad-original-pull-secret") pod "global-pull-secret-syncer-n4j7l" (UID: "8f8b563f-d153-48f5-ab5e-836807f76cad") : object "kube-system"/"original-pull-secret" not registered May 21 00:54:53.667247 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:53.667212 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:54:53.667691 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:53.667334 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:54:53.667691 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:53.667337 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxzxp" podUID="3b778703-acf8-4977-a1b1-2f5f9042d2b3" May 21 00:54:53.667691 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:53.667509 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4j7l" podUID="8f8b563f-d153-48f5-ab5e-836807f76cad" May 21 00:54:53.667691 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:53.667549 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:54:53.667691 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:53.667625 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skbch" podUID="5ee0c301-12eb-4c68-9990-6ceafb2c90fa" May 21 00:54:53.765375 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:53.765346 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f8b563f-d153-48f5-ab5e-836807f76cad-original-pull-secret\") pod \"global-pull-secret-syncer-n4j7l\" (UID: \"8f8b563f-d153-48f5-ab5e-836807f76cad\") " pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:54:53.765527 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:53.765485 2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 21 00:54:53.765574 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:53.765544 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f8b563f-d153-48f5-ab5e-836807f76cad-original-pull-secret podName:8f8b563f-d153-48f5-ab5e-836807f76cad nodeName:}" failed. No retries permitted until 2026-05-21 00:54:55.765529841 +0000 UTC m=+14.662475135 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f8b563f-d153-48f5-ab5e-836807f76cad-original-pull-secret") pod "global-pull-secret-syncer-n4j7l" (UID: "8f8b563f-d153-48f5-ab5e-836807f76cad") : object "kube-system"/"original-pull-secret" not registered May 21 00:54:55.667300 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:55.667245 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:54:55.667679 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:55.667253 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:54:55.667679 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:55.667373 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4j7l" podUID="8f8b563f-d153-48f5-ab5e-836807f76cad" May 21 00:54:55.667679 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:55.667256 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:54:55.667679 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:55.667453 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxzxp" podUID="3b778703-acf8-4977-a1b1-2f5f9042d2b3" May 21 00:54:55.667679 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:55.667560 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skbch" podUID="5ee0c301-12eb-4c68-9990-6ceafb2c90fa" May 21 00:54:55.780261 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:55.780226 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f8b563f-d153-48f5-ab5e-836807f76cad-original-pull-secret\") pod \"global-pull-secret-syncer-n4j7l\" (UID: \"8f8b563f-d153-48f5-ab5e-836807f76cad\") " pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:54:55.780415 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:55.780394 2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 21 00:54:55.780480 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:55.780468 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f8b563f-d153-48f5-ab5e-836807f76cad-original-pull-secret podName:8f8b563f-d153-48f5-ab5e-836807f76cad nodeName:}" failed. No retries permitted until 2026-05-21 00:54:59.780446373 +0000 UTC m=+18.677391667 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f8b563f-d153-48f5-ab5e-836807f76cad-original-pull-secret") pod "global-pull-secret-syncer-n4j7l" (UID: "8f8b563f-d153-48f5-ab5e-836807f76cad") : object "kube-system"/"original-pull-secret" not registered May 21 00:54:57.667544 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:57.667509 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:54:57.667544 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:57.667536 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:54:57.668137 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:57.667556 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:54:57.668137 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:57.667689 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4j7l" podUID="8f8b563f-d153-48f5-ab5e-836807f76cad" May 21 00:54:57.668137 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:57.667857 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxzxp" podUID="3b778703-acf8-4977-a1b1-2f5f9042d2b3" May 21 00:54:57.668137 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:57.667963 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skbch" podUID="5ee0c301-12eb-4c68-9990-6ceafb2c90fa" May 21 00:54:59.309524 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:59.309490 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-metrics-certs\") pod \"network-metrics-daemon-skbch\" (UID: \"5ee0c301-12eb-4c68-9990-6ceafb2c90fa\") " pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:54:59.309979 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:59.309531 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-66pbg\" (UniqueName: \"kubernetes.io/projected/3b778703-acf8-4977-a1b1-2f5f9042d2b3-kube-api-access-66pbg\") pod \"network-check-target-rxzxp\" (UID: \"3b778703-acf8-4977-a1b1-2f5f9042d2b3\") " pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:54:59.309979 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:59.309664 2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 21 00:54:59.309979 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:59.309728 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 21 00:54:59.309979 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:59.309745 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-metrics-certs podName:5ee0c301-12eb-4c68-9990-6ceafb2c90fa nodeName:}" failed. No retries permitted until 2026-05-21 00:55:15.309714984 +0000 UTC m=+34.206660289 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-metrics-certs") pod "network-metrics-daemon-skbch" (UID: "5ee0c301-12eb-4c68-9990-6ceafb2c90fa") : object "openshift-multus"/"metrics-daemon-secret" not registered May 21 00:54:59.309979 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:59.309746 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 21 00:54:59.309979 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:59.309767 2549 projected.go:194] Error preparing data for projected volume kube-api-access-66pbg for pod openshift-network-diagnostics/network-check-target-rxzxp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 00:54:59.309979 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:59.309804 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b778703-acf8-4977-a1b1-2f5f9042d2b3-kube-api-access-66pbg podName:3b778703-acf8-4977-a1b1-2f5f9042d2b3 nodeName:}" failed. No retries permitted until 2026-05-21 00:55:15.30979305 +0000 UTC m=+34.206738339 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-66pbg" (UniqueName: "kubernetes.io/projected/3b778703-acf8-4977-a1b1-2f5f9042d2b3-kube-api-access-66pbg") pod "network-check-target-rxzxp" (UID: "3b778703-acf8-4977-a1b1-2f5f9042d2b3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 00:54:59.667398 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:59.667323 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:54:59.667549 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:59.667323 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:54:59.667549 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:59.667456 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:54:59.667549 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:59.667451 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skbch" podUID="5ee0c301-12eb-4c68-9990-6ceafb2c90fa" May 21 00:54:59.667549 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:59.667537 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4j7l" podUID="8f8b563f-d153-48f5-ab5e-836807f76cad" May 21 00:54:59.667740 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:59.667624 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxzxp" podUID="3b778703-acf8-4977-a1b1-2f5f9042d2b3" May 21 00:54:59.813941 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:54:59.813911 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f8b563f-d153-48f5-ab5e-836807f76cad-original-pull-secret\") pod \"global-pull-secret-syncer-n4j7l\" (UID: \"8f8b563f-d153-48f5-ab5e-836807f76cad\") " pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:54:59.814123 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:59.814077 2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 21 00:54:59.814188 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:54:59.814140 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f8b563f-d153-48f5-ab5e-836807f76cad-original-pull-secret podName:8f8b563f-d153-48f5-ab5e-836807f76cad nodeName:}" failed. No retries permitted until 2026-05-21 00:55:07.814120762 +0000 UTC m=+26.711066077 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f8b563f-d153-48f5-ab5e-836807f76cad-original-pull-secret") pod "global-pull-secret-syncer-n4j7l" (UID: "8f8b563f-d153-48f5-ab5e-836807f76cad") : object "kube-system"/"original-pull-secret" not registered May 21 00:55:01.667958 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:01.667928 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:55:01.668423 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:01.668028 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4j7l" podUID="8f8b563f-d153-48f5-ab5e-836807f76cad" May 21 00:55:01.668423 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:01.668132 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:55:01.668423 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:01.668248 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skbch" podUID="5ee0c301-12eb-4c68-9990-6ceafb2c90fa" May 21 00:55:01.668423 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:01.668295 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:55:01.668423 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:01.668369 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxzxp" podUID="3b778703-acf8-4977-a1b1-2f5f9042d2b3" May 21 00:55:02.751359 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:02.751126 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sfsvt" event={"ID":"0a1071e0-9724-4546-8b54-44bce04c35c2","Type":"ContainerStarted","Data":"bc44b01b3d69a1c3d3a5b56bc65ae25755d5054553753a56b1a95b1af1b70527"} May 21 00:55:02.753219 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:02.753193 2549 generic.go:358] "Generic (PLEG): container finished" podID="5da3aae9-eaf5-4328-afed-6595832fc2e1" containerID="c15fa3cf84e2d227dd0c01fba43cd9cb8ccc1b7031dd8cba0556ed66aacc8918" exitCode=0 May 21 00:55:02.753342 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:02.753259 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-spsr8" event={"ID":"5da3aae9-eaf5-4328-afed-6595832fc2e1","Type":"ContainerDied","Data":"c15fa3cf84e2d227dd0c01fba43cd9cb8ccc1b7031dd8cba0556ed66aacc8918"} May 21 00:55:02.754695 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:02.754608 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sh44w" event={"ID":"acd173d4-75da-4564-ac8e-84996bb1742d","Type":"ContainerStarted","Data":"19b7f576ff8ae7d6f8c2d4aac6482d39a8cbc5be06aececa41913a182ea84b7b"} May 21 00:55:02.756136 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:02.756021 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" event={"ID":"8adbdb60-a625-47a1-8252-779cd35e6a35","Type":"ContainerStarted","Data":"7d949264392f7ddf30cd81e1a39220c6a25cc13b7169bf83be56e5fa2b22f555"} May 21 00:55:02.757517 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:02.757500 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgxp9_12a9408a-b799-4b3c-a790-a4d6e6d67cbd/ovn-acl-logging/0.log" May 21 00:55:02.757879 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:02.757854 2549 generic.go:358] "Generic (PLEG): container finished" podID="12a9408a-b799-4b3c-a790-a4d6e6d67cbd" containerID="fe9a3df0a0dfd3f98e6ad79f1482c1534b9df8e1389a4583353f26bc139478e2" exitCode=1 May 21 00:55:02.757970 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:02.757923 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" event={"ID":"12a9408a-b799-4b3c-a790-a4d6e6d67cbd","Type":"ContainerDied","Data":"fe9a3df0a0dfd3f98e6ad79f1482c1534b9df8e1389a4583353f26bc139478e2"} May 21 00:55:02.757970 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:02.757947 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" event={"ID":"12a9408a-b799-4b3c-a790-a4d6e6d67cbd","Type":"ContainerStarted","Data":"6bc379776225482869158323d09fc621e231416756a7786eb681258d866d52e8"} May 21 00:55:02.759258 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:02.759236 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" event={"ID":"96547212-630f-43e2-9662-5938454d35ec","Type":"ContainerStarted","Data":"8f7654d26fb1218c5e3935299872dfc4e4ae8ccdc057235a255b6c0221269a79"} May 21 00:55:02.760779 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:02.760749 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h7nwx" event={"ID":"9ff27421-0ba9-40f6-98db-150fbcca1a5e","Type":"ContainerStarted","Data":"102b9396f48b38238819978f39a1055013bc7c52daa491982c41d5d8a934d121"} May 21 00:55:02.762102 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:02.762082 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-96x9m" event={"ID":"31374768-4e56-404b-8d22-7e2136138fa4","Type":"ContainerStarted","Data":"5b9a8206c8573afcbf3a5399e4247b384bfdda213cfd9f9903835de14ed294f6"} May 21 00:55:02.764600 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:02.764552 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sfsvt" podStartSLOduration=3.936021127 podStartE2EDuration="21.764536876s" podCreationTimestamp="2026-05-21 00:54:41 +0000 UTC" firstStartedPulling="2026-05-21 00:54:44.327244049 +0000 UTC m=+3.224189342" lastFinishedPulling="2026-05-21 00:55:02.155759789 +0000 UTC m=+21.052705091" observedRunningTime="2026-05-21 00:55:02.763694134 +0000 UTC m=+21.660639455" watchObservedRunningTime="2026-05-21 00:55:02.764536876 +0000 UTC m=+21.661482188" May 21 00:55:02.777228 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:02.777180 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-h7nwx" podStartSLOduration=7.457486917 podStartE2EDuration="21.777170623s" podCreationTimestamp="2026-05-21 00:54:41 +0000 UTC" firstStartedPulling="2026-05-21 00:54:44.317672716 +0000 UTC m=+3.214618015" lastFinishedPulling="2026-05-21 00:54:58.637356429 +0000 UTC m=+17.534301721" observedRunningTime="2026-05-21 00:55:02.776732682 +0000 UTC m=+21.673677994" watchObservedRunningTime="2026-05-21 00:55:02.777170623 +0000 UTC m=+21.674115931" May 21 00:55:02.788561 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:02.788525 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-96x9m" podStartSLOduration=3.953846847 podStartE2EDuration="21.788514799s" podCreationTimestamp="2026-05-21 00:54:41 +0000 UTC" firstStartedPulling="2026-05-21 00:54:44.321639652 +0000 UTC m=+3.218584956" lastFinishedPulling="2026-05-21 00:55:02.156307604 +0000 UTC m=+21.053252908" observedRunningTime="2026-05-21 00:55:02.788494595 +0000 UTC m=+21.685439905" watchObservedRunningTime="2026-05-21 00:55:02.788514799 +0000 UTC m=+21.685460109" May 21 00:55:02.805436 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:02.805229 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gvmxz" podStartSLOduration=3.965667256 podStartE2EDuration="21.805214421s" podCreationTimestamp="2026-05-21 00:54:41 +0000 UTC" firstStartedPulling="2026-05-21 00:54:44.318773165 +0000 UTC m=+3.215718454" lastFinishedPulling="2026-05-21 00:55:02.158320323 +0000 UTC m=+21.055265619" observedRunningTime="2026-05-21 00:55:02.804783194 +0000 UTC m=+21.701728507" watchObservedRunningTime="2026-05-21 00:55:02.805214421 +0000 UTC m=+21.702159733" May 21 00:55:02.819431 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:02.819389 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-sh44w" podStartSLOduration=3.611766889 podStartE2EDuration="21.819375989s" podCreationTimestamp="2026-05-21 00:54:41 +0000 UTC" firstStartedPulling="2026-05-21 00:54:44.325336786 +0000 UTC m=+3.222282075" lastFinishedPulling="2026-05-21 00:55:02.532945873 +0000 UTC m=+21.429891175" observedRunningTime="2026-05-21 00:55:02.819288856 +0000 UTC m=+21.716234167" watchObservedRunningTime="2026-05-21 00:55:02.819375989 +0000 UTC m=+21.716321301" May 21 00:55:03.667858 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:03.667659 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:55:03.667962 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:03.667668 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:55:03.668012 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:03.667958 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4j7l" podUID="8f8b563f-d153-48f5-ab5e-836807f76cad" May 21 00:55:03.668085 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:03.667672 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:55:03.668085 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:03.668071 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skbch" podUID="5ee0c301-12eb-4c68-9990-6ceafb2c90fa" May 21 00:55:03.668201 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:03.668130 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxzxp" podUID="3b778703-acf8-4977-a1b1-2f5f9042d2b3" May 21 00:55:03.674974 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:03.674900 2549 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" May 21 00:55:03.766411 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:03.766376 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" event={"ID":"8adbdb60-a625-47a1-8252-779cd35e6a35","Type":"ContainerStarted","Data":"f05c406cefd9c32c492bd6430e6e8026b11ed20b7b4fa789b44f8ae456c37538"} May 21 00:55:03.768887 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:03.768867 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgxp9_12a9408a-b799-4b3c-a790-a4d6e6d67cbd/ovn-acl-logging/0.log" May 21 00:55:03.769308 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:03.769282 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" event={"ID":"12a9408a-b799-4b3c-a790-a4d6e6d67cbd","Type":"ContainerStarted","Data":"e654408cd750e556ab239269bdea641526b0cab80e4d2d886bf48dd9745142ad"} May 21 00:55:03.769433 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:03.769316 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" event={"ID":"12a9408a-b799-4b3c-a790-a4d6e6d67cbd","Type":"ContainerStarted","Data":"3aebdd8d4289286d1127c93fb558f614a46406a6e967ce7dfd2a6120434821c9"} May 21 00:55:03.769433 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:03.769329 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" event={"ID":"12a9408a-b799-4b3c-a790-a4d6e6d67cbd","Type":"ContainerStarted","Data":"4fd2115594b7c7b1bab900e8754c635fb16ef7929e46458909c53232b6fff351"} May 21 00:55:03.769433 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:03.769341 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" event={"ID":"12a9408a-b799-4b3c-a790-a4d6e6d67cbd","Type":"ContainerStarted","Data":"7d8b6a02e0611708ba7af8b9e3dd0440f6028845776ba6bb90a1fb239a1531bb"} May 21 00:55:03.770626 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:03.770600 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qdnd9" event={"ID":"7d5e55de-c445-48f6-8911-e4f1a11104de","Type":"ContainerStarted","Data":"64cfd8d1e97e8436480c19efbf2f494c87c43c08902494e1b8456f8b4a63bd8d"} May 21 00:55:03.784086 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:03.784031 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qdnd9" podStartSLOduration=4.954876754 podStartE2EDuration="22.784019244s" podCreationTimestamp="2026-05-21 00:54:41 +0000 UTC" firstStartedPulling="2026-05-21 00:54:44.326478665 +0000 UTC m=+3.223423969" lastFinishedPulling="2026-05-21 00:55:02.155621156 +0000 UTC m=+21.052566459" observedRunningTime="2026-05-21 00:55:03.783706547 +0000 UTC m=+22.680651870" watchObservedRunningTime="2026-05-21 00:55:03.784019244 +0000 UTC m=+22.680964554" May 21 00:55:04.020104 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:04.020013 2549 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-h7nwx" May 21 00:55:04.020643 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:04.020624 2549 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-h7nwx" May 21 00:55:04.649457 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:04.649361 2549 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-21T00:55:03.674912727Z","UUID":"a268dbe5-6d8e-4ad5-9835-0599a0511989","Handler":null,"Name":"","Endpoint":""} May 21 00:55:04.651283 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:04.651262 2549 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 May 21 00:55:04.651379 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:04.651289 2549 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock May 21 00:55:04.774040 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:04.774006 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" event={"ID":"8adbdb60-a625-47a1-8252-779cd35e6a35","Type":"ContainerStarted","Data":"b535bc0b579b346ee263c22575d51a954e10a3d9f644f5f5c96b0cd6db9ba842"} May 21 00:55:04.774532 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:04.774482 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-h7nwx" May 21 00:55:04.774868 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:04.774849 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-h7nwx" May 21 00:55:04.789800 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:04.789760 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vvqd9" podStartSLOduration=3.565679336 podStartE2EDuration="23.78974847s" podCreationTimestamp="2026-05-21 00:54:41 +0000 UTC" firstStartedPulling="2026-05-21 00:54:44.32377861 +0000 UTC m=+3.220723912" lastFinishedPulling="2026-05-21 00:55:04.547847752 +0000 UTC m=+23.444793046" observedRunningTime="2026-05-21 00:55:04.789274628 +0000 UTC m=+23.686219939" watchObservedRunningTime="2026-05-21 00:55:04.78974847 +0000 UTC m=+23.686693781" May 21 00:55:05.667386 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:05.667347 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:55:05.667550 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:05.667347 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:55:05.667550 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:05.667477 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skbch" podUID="5ee0c301-12eb-4c68-9990-6ceafb2c90fa" May 21 00:55:05.667550 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:05.667350 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:55:05.667696 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:05.667553 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxzxp" podUID="3b778703-acf8-4977-a1b1-2f5f9042d2b3" May 21 00:55:05.667696 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:05.667621 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4j7l" podUID="8f8b563f-d153-48f5-ab5e-836807f76cad" May 21 00:55:05.778894 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:05.778870 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgxp9_12a9408a-b799-4b3c-a790-a4d6e6d67cbd/ovn-acl-logging/0.log" May 21 00:55:05.779350 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:05.779224 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" event={"ID":"12a9408a-b799-4b3c-a790-a4d6e6d67cbd","Type":"ContainerStarted","Data":"ba9ba97cf5c1f36979234c3ba134a123721122d2a9ac17a035b879801d5002f5"} May 21 00:55:07.667199 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:07.667163 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:55:07.667901 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:07.667276 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxzxp" podUID="3b778703-acf8-4977-a1b1-2f5f9042d2b3" May 21 00:55:07.667901 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:07.667285 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:55:07.667901 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:07.667309 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:55:07.667901 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:07.667414 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skbch" podUID="5ee0c301-12eb-4c68-9990-6ceafb2c90fa" May 21 00:55:07.667901 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:07.667504 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4j7l" podUID="8f8b563f-d153-48f5-ab5e-836807f76cad" May 21 00:55:07.879310 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:07.879243 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f8b563f-d153-48f5-ab5e-836807f76cad-original-pull-secret\") pod \"global-pull-secret-syncer-n4j7l\" (UID: \"8f8b563f-d153-48f5-ab5e-836807f76cad\") " pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:55:07.879429 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:07.879365 2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 21 00:55:07.879429 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:07.879419 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f8b563f-d153-48f5-ab5e-836807f76cad-original-pull-secret podName:8f8b563f-d153-48f5-ab5e-836807f76cad nodeName:}" failed. No retries permitted until 2026-05-21 00:55:23.879403749 +0000 UTC m=+42.776349043 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f8b563f-d153-48f5-ab5e-836807f76cad-original-pull-secret") pod "global-pull-secret-syncer-n4j7l" (UID: "8f8b563f-d153-48f5-ab5e-836807f76cad") : object "kube-system"/"original-pull-secret" not registered May 21 00:55:08.787211 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:08.786981 2549 generic.go:358] "Generic (PLEG): container finished" podID="5da3aae9-eaf5-4328-afed-6595832fc2e1" containerID="11b1547fa8cfe632ff20e8b41fd078b23b5ef7f027382c9a9aea62c88cbb2796" exitCode=0 May 21 00:55:08.787626 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:08.787097 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-spsr8" event={"ID":"5da3aae9-eaf5-4328-afed-6595832fc2e1","Type":"ContainerDied","Data":"11b1547fa8cfe632ff20e8b41fd078b23b5ef7f027382c9a9aea62c88cbb2796"} May 21 00:55:08.790935 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:08.790917 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgxp9_12a9408a-b799-4b3c-a790-a4d6e6d67cbd/ovn-acl-logging/0.log" May 21 00:55:08.791252 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:08.791227 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" event={"ID":"12a9408a-b799-4b3c-a790-a4d6e6d67cbd","Type":"ContainerStarted","Data":"06949834862cfdf1d286b096a8e87133d3c005f729e98d21db286ba727425827"} May 21 00:55:08.791608 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:08.791569 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:55:08.791608 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:08.791602 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:55:08.791814 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:08.791615 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:55:08.791814 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:08.791695 2549 scope.go:117] "RemoveContainer" containerID="fe9a3df0a0dfd3f98e6ad79f1482c1534b9df8e1389a4583353f26bc139478e2" May 21 00:55:08.806974 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:08.806952 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:55:08.807094 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:08.807032 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:55:09.668486 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:09.668044 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:55:09.668486 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:09.668185 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4j7l" podUID="8f8b563f-d153-48f5-ab5e-836807f76cad" May 21 00:55:09.668486 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:09.668480 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:55:09.668757 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:09.668584 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxzxp" podUID="3b778703-acf8-4977-a1b1-2f5f9042d2b3" May 21 00:55:09.668757 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:09.668637 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:55:09.668757 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:09.668712 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skbch" podUID="5ee0c301-12eb-4c68-9990-6ceafb2c90fa" May 21 00:55:09.796203 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:09.796082 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgxp9_12a9408a-b799-4b3c-a790-a4d6e6d67cbd/ovn-acl-logging/0.log" May 21 00:55:09.796574 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:09.796549 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" event={"ID":"12a9408a-b799-4b3c-a790-a4d6e6d67cbd","Type":"ContainerStarted","Data":"f9d68a25e69a1218c070a8d322823122d94064880f516d9a63b013c0eaa06051"} May 21 00:55:09.798329 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:09.798307 2549 generic.go:358] "Generic (PLEG): container finished" podID="5da3aae9-eaf5-4328-afed-6595832fc2e1" containerID="83de464f82a77b756d85497b838ceee2609c4f236c218d12c33f57fa6de515ef" exitCode=0 May 21 00:55:09.798422 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:09.798340 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-spsr8" event={"ID":"5da3aae9-eaf5-4328-afed-6595832fc2e1","Type":"ContainerDied","Data":"83de464f82a77b756d85497b838ceee2609c4f236c218d12c33f57fa6de515ef"} May 21 00:55:09.821776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:09.821723 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" podStartSLOduration=10.945958966 podStartE2EDuration="28.821712236s" podCreationTimestamp="2026-05-21 00:54:41 +0000 UTC" firstStartedPulling="2026-05-21 00:54:44.323439345 +0000 UTC m=+3.220384633" lastFinishedPulling="2026-05-21 00:55:02.199192608 +0000 UTC m=+21.096137903" observedRunningTime="2026-05-21 00:55:09.82139344 +0000 UTC m=+28.718338753" watchObservedRunningTime="2026-05-21 00:55:09.821712236 +0000 UTC m=+28.718657547" May 21 00:55:09.935992 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:09.935925 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n4j7l"] May 21 00:55:09.936096 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:09.936013 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:55:09.936144 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:09.936128 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4j7l" podUID="8f8b563f-d153-48f5-ab5e-836807f76cad" May 21 00:55:09.939047 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:09.939026 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rxzxp"] May 21 00:55:09.939167 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:09.939119 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:55:09.939229 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:09.939207 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxzxp" podUID="3b778703-acf8-4977-a1b1-2f5f9042d2b3" May 21 00:55:09.939602 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:09.939581 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-skbch"] May 21 00:55:09.939672 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:09.939644 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:55:09.939732 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:09.939715 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skbch" podUID="5ee0c301-12eb-4c68-9990-6ceafb2c90fa" May 21 00:55:11.673025 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:11.672996 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:55:11.673554 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:11.673121 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxzxp" podUID="3b778703-acf8-4977-a1b1-2f5f9042d2b3" May 21 00:55:11.673554 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:11.673190 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:55:11.673554 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:11.673276 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4j7l" podUID="8f8b563f-d153-48f5-ab5e-836807f76cad" May 21 00:55:11.673554 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:11.673321 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:55:11.673554 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:11.673433 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skbch" podUID="5ee0c301-12eb-4c68-9990-6ceafb2c90fa" May 21 00:55:11.804802 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:11.804762 2549 generic.go:358] "Generic (PLEG): container finished" podID="5da3aae9-eaf5-4328-afed-6595832fc2e1" containerID="c912bb6ff580757c7fca14c71e9fbd0997523a13c261cf9b3a65148a97a238b6" exitCode=0 May 21 00:55:11.804941 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:11.804834 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-spsr8" event={"ID":"5da3aae9-eaf5-4328-afed-6595832fc2e1","Type":"ContainerDied","Data":"c912bb6ff580757c7fca14c71e9fbd0997523a13c261cf9b3a65148a97a238b6"} May 21 00:55:13.671252 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:13.671224 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:55:13.671848 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:13.671223 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:55:13.671848 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:13.671335 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4j7l" podUID="8f8b563f-d153-48f5-ab5e-836807f76cad" May 21 00:55:13.671848 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:13.671221 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:55:13.671848 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:13.671424 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-skbch" podUID="5ee0c301-12eb-4c68-9990-6ceafb2c90fa" May 21 00:55:13.671848 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:13.671475 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxzxp" podUID="3b778703-acf8-4977-a1b1-2f5f9042d2b3" May 21 00:55:15.334845 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.334807 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-metrics-certs\") pod \"network-metrics-daemon-skbch\" (UID: \"5ee0c301-12eb-4c68-9990-6ceafb2c90fa\") " pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:55:15.335263 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.334852 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-66pbg\" (UniqueName: \"kubernetes.io/projected/3b778703-acf8-4977-a1b1-2f5f9042d2b3-kube-api-access-66pbg\") pod \"network-check-target-rxzxp\" (UID: \"3b778703-acf8-4977-a1b1-2f5f9042d2b3\") " pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:55:15.335263 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:15.334961 2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 21 00:55:15.335263 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:15.335031 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 21 00:55:15.335263 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:15.335039 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-metrics-certs podName:5ee0c301-12eb-4c68-9990-6ceafb2c90fa nodeName:}" failed. No retries permitted until 2026-05-21 00:55:47.335019295 +0000 UTC m=+66.231964603 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-metrics-certs") pod "network-metrics-daemon-skbch" (UID: "5ee0c301-12eb-4c68-9990-6ceafb2c90fa") : object "openshift-multus"/"metrics-daemon-secret" not registered May 21 00:55:15.335263 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:15.335050 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 21 00:55:15.335263 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:15.335080 2549 projected.go:194] Error preparing data for projected volume kube-api-access-66pbg for pod openshift-network-diagnostics/network-check-target-rxzxp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 00:55:15.335263 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:15.335142 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b778703-acf8-4977-a1b1-2f5f9042d2b3-kube-api-access-66pbg podName:3b778703-acf8-4977-a1b1-2f5f9042d2b3 nodeName:}" failed. No retries permitted until 2026-05-21 00:55:47.335123947 +0000 UTC m=+66.232069240 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-66pbg" (UniqueName: "kubernetes.io/projected/3b778703-acf8-4977-a1b1-2f5f9042d2b3-kube-api-access-66pbg") pod "network-check-target-rxzxp" (UID: "3b778703-acf8-4977-a1b1-2f5f9042d2b3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 00:55:15.437012 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.436982 2549 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeReady" May 21 00:55:15.437189 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.437122 2549 kubelet_node_status.go:550] "Fast updating node status as it just became ready" May 21 00:55:15.477864 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.477836 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5mlhs"] May 21 00:55:15.510145 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.510119 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2lhq8"] May 21 00:55:15.510309 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.510286 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5mlhs" May 21 00:55:15.512714 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.512690 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" May 21 00:55:15.512714 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.512706 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" May 21 00:55:15.512891 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.512761 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8hsl2\"" May 21 00:55:15.524090 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.524042 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5mlhs"] May 21 00:55:15.524090 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.524088 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2lhq8"] May 21 00:55:15.524266 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.524181 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2lhq8" May 21 00:55:15.526523 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.526505 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" May 21 00:55:15.526523 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.526512 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" May 21 00:55:15.526685 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.526560 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8sxzc\"" May 21 00:55:15.526819 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.526796 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" May 21 00:55:15.638209 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.638136 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/776a4785-ccb6-42c4-b6d8-050c42ad5541-config-volume\") pod \"dns-default-5mlhs\" (UID: \"776a4785-ccb6-42c4-b6d8-050c42ad5541\") " pod="openshift-dns/dns-default-5mlhs" May 21 00:55:15.638332 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.638208 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/776a4785-ccb6-42c4-b6d8-050c42ad5541-metrics-tls\") pod \"dns-default-5mlhs\" (UID: \"776a4785-ccb6-42c4-b6d8-050c42ad5541\") " pod="openshift-dns/dns-default-5mlhs" May 21 00:55:15.638332 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.638264 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/776a4785-ccb6-42c4-b6d8-050c42ad5541-tmp-dir\") pod \"dns-default-5mlhs\" (UID: \"776a4785-ccb6-42c4-b6d8-050c42ad5541\") " pod="openshift-dns/dns-default-5mlhs" May 21 00:55:15.638332 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.638283 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnq6n\" (UniqueName: \"kubernetes.io/projected/776a4785-ccb6-42c4-b6d8-050c42ad5541-kube-api-access-wnq6n\") pod \"dns-default-5mlhs\" (UID: \"776a4785-ccb6-42c4-b6d8-050c42ad5541\") " pod="openshift-dns/dns-default-5mlhs" May 21 00:55:15.638332 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.638314 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/877e95c2-b86f-49d3-a207-eb91f732bfff-cert\") pod \"ingress-canary-2lhq8\" (UID: \"877e95c2-b86f-49d3-a207-eb91f732bfff\") " pod="openshift-ingress-canary/ingress-canary-2lhq8" May 21 00:55:15.638509 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.638330 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqntg\" (UniqueName: \"kubernetes.io/projected/877e95c2-b86f-49d3-a207-eb91f732bfff-kube-api-access-kqntg\") pod \"ingress-canary-2lhq8\" (UID: \"877e95c2-b86f-49d3-a207-eb91f732bfff\") " pod="openshift-ingress-canary/ingress-canary-2lhq8" May 21 00:55:15.670388 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.670366 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:55:15.670523 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.670367 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:55:15.670595 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.670367 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:55:15.672999 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.672976 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 21 00:55:15.673183 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.673167 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" May 21 00:55:15.673282 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.673212 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 21 00:55:15.673282 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.673251 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 21 00:55:15.673476 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.673461 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2nm7s\"" May 21 00:55:15.673624 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.673608 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fhwfb\"" May 21 00:55:15.739649 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.739621 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/877e95c2-b86f-49d3-a207-eb91f732bfff-cert\") pod \"ingress-canary-2lhq8\" (UID: \"877e95c2-b86f-49d3-a207-eb91f732bfff\") " pod="openshift-ingress-canary/ingress-canary-2lhq8" May 21 00:55:15.739754 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.739659 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kqntg\" (UniqueName: \"kubernetes.io/projected/877e95c2-b86f-49d3-a207-eb91f732bfff-kube-api-access-kqntg\") pod \"ingress-canary-2lhq8\" (UID: \"877e95c2-b86f-49d3-a207-eb91f732bfff\") " pod="openshift-ingress-canary/ingress-canary-2lhq8" May 21 00:55:15.739754 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.739699 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/776a4785-ccb6-42c4-b6d8-050c42ad5541-config-volume\") pod \"dns-default-5mlhs\" (UID: \"776a4785-ccb6-42c4-b6d8-050c42ad5541\") " pod="openshift-dns/dns-default-5mlhs" May 21 00:55:15.739844 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:15.739787 2549 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 21 00:55:15.739897 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:15.739851 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/877e95c2-b86f-49d3-a207-eb91f732bfff-cert podName:877e95c2-b86f-49d3-a207-eb91f732bfff nodeName:}" failed. No retries permitted until 2026-05-21 00:55:16.239831278 +0000 UTC m=+35.136776572 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/877e95c2-b86f-49d3-a207-eb91f732bfff-cert") pod "ingress-canary-2lhq8" (UID: "877e95c2-b86f-49d3-a207-eb91f732bfff") : secret "canary-serving-cert" not found May 21 00:55:15.739959 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.739898 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/776a4785-ccb6-42c4-b6d8-050c42ad5541-metrics-tls\") pod \"dns-default-5mlhs\" (UID: \"776a4785-ccb6-42c4-b6d8-050c42ad5541\") " pod="openshift-dns/dns-default-5mlhs" May 21 00:55:15.739959 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.739942 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/776a4785-ccb6-42c4-b6d8-050c42ad5541-tmp-dir\") pod \"dns-default-5mlhs\" (UID: \"776a4785-ccb6-42c4-b6d8-050c42ad5541\") " pod="openshift-dns/dns-default-5mlhs" May 21 00:55:15.740055 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.739972 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wnq6n\" (UniqueName: \"kubernetes.io/projected/776a4785-ccb6-42c4-b6d8-050c42ad5541-kube-api-access-wnq6n\") pod \"dns-default-5mlhs\" (UID: \"776a4785-ccb6-42c4-b6d8-050c42ad5541\") " pod="openshift-dns/dns-default-5mlhs" May 21 00:55:15.740132 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:15.740079 2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 21 00:55:15.740181 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:15.740134 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776a4785-ccb6-42c4-b6d8-050c42ad5541-metrics-tls podName:776a4785-ccb6-42c4-b6d8-050c42ad5541 nodeName:}" failed. No retries permitted until 2026-05-21 00:55:16.240116373 +0000 UTC m=+35.137061679 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/776a4785-ccb6-42c4-b6d8-050c42ad5541-metrics-tls") pod "dns-default-5mlhs" (UID: "776a4785-ccb6-42c4-b6d8-050c42ad5541") : secret "dns-default-metrics-tls" not found May 21 00:55:15.740294 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.740253 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/776a4785-ccb6-42c4-b6d8-050c42ad5541-tmp-dir\") pod \"dns-default-5mlhs\" (UID: \"776a4785-ccb6-42c4-b6d8-050c42ad5541\") " pod="openshift-dns/dns-default-5mlhs" May 21 00:55:15.740398 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.740378 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/776a4785-ccb6-42c4-b6d8-050c42ad5541-config-volume\") pod \"dns-default-5mlhs\" (UID: \"776a4785-ccb6-42c4-b6d8-050c42ad5541\") " pod="openshift-dns/dns-default-5mlhs" May 21 00:55:15.751728 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.751571 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqntg\" (UniqueName: \"kubernetes.io/projected/877e95c2-b86f-49d3-a207-eb91f732bfff-kube-api-access-kqntg\") pod \"ingress-canary-2lhq8\" (UID: \"877e95c2-b86f-49d3-a207-eb91f732bfff\") " pod="openshift-ingress-canary/ingress-canary-2lhq8" May 21 00:55:15.758530 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:15.758510 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnq6n\" (UniqueName: \"kubernetes.io/projected/776a4785-ccb6-42c4-b6d8-050c42ad5541-kube-api-access-wnq6n\") pod \"dns-default-5mlhs\" (UID: \"776a4785-ccb6-42c4-b6d8-050c42ad5541\") " pod="openshift-dns/dns-default-5mlhs" May 21 00:55:16.244847 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:16.244808 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/776a4785-ccb6-42c4-b6d8-050c42ad5541-metrics-tls\") pod \"dns-default-5mlhs\" (UID: \"776a4785-ccb6-42c4-b6d8-050c42ad5541\") " pod="openshift-dns/dns-default-5mlhs" May 21 00:55:16.245019 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:16.244863 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/877e95c2-b86f-49d3-a207-eb91f732bfff-cert\") pod \"ingress-canary-2lhq8\" (UID: \"877e95c2-b86f-49d3-a207-eb91f732bfff\") " pod="openshift-ingress-canary/ingress-canary-2lhq8" May 21 00:55:16.245019 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:16.244976 2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 21 00:55:16.245137 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:16.245043 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776a4785-ccb6-42c4-b6d8-050c42ad5541-metrics-tls podName:776a4785-ccb6-42c4-b6d8-050c42ad5541 nodeName:}" failed. No retries permitted until 2026-05-21 00:55:17.245024379 +0000 UTC m=+36.141969672 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/776a4785-ccb6-42c4-b6d8-050c42ad5541-metrics-tls") pod "dns-default-5mlhs" (UID: "776a4785-ccb6-42c4-b6d8-050c42ad5541") : secret "dns-default-metrics-tls" not found May 21 00:55:16.245137 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:16.244979 2549 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 21 00:55:16.245137 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:16.245110 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/877e95c2-b86f-49d3-a207-eb91f732bfff-cert podName:877e95c2-b86f-49d3-a207-eb91f732bfff nodeName:}" failed. No retries permitted until 2026-05-21 00:55:17.24509861 +0000 UTC m=+36.142043911 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/877e95c2-b86f-49d3-a207-eb91f732bfff-cert") pod "ingress-canary-2lhq8" (UID: "877e95c2-b86f-49d3-a207-eb91f732bfff") : secret "canary-serving-cert" not found May 21 00:55:17.252224 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:17.252187 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/877e95c2-b86f-49d3-a207-eb91f732bfff-cert\") pod \"ingress-canary-2lhq8\" (UID: \"877e95c2-b86f-49d3-a207-eb91f732bfff\") " pod="openshift-ingress-canary/ingress-canary-2lhq8" May 21 00:55:17.252636 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:17.252280 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/776a4785-ccb6-42c4-b6d8-050c42ad5541-metrics-tls\") pod \"dns-default-5mlhs\" (UID: \"776a4785-ccb6-42c4-b6d8-050c42ad5541\") " pod="openshift-dns/dns-default-5mlhs" May 21 00:55:17.252636 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:17.252350 2549 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 21 00:55:17.252636 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:17.252385 2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 21 00:55:17.252636 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:17.252423 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/877e95c2-b86f-49d3-a207-eb91f732bfff-cert podName:877e95c2-b86f-49d3-a207-eb91f732bfff nodeName:}" failed. No retries permitted until 2026-05-21 00:55:19.252402206 +0000 UTC m=+38.149347501 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/877e95c2-b86f-49d3-a207-eb91f732bfff-cert") pod "ingress-canary-2lhq8" (UID: "877e95c2-b86f-49d3-a207-eb91f732bfff") : secret "canary-serving-cert" not found May 21 00:55:17.252636 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:17.252445 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776a4785-ccb6-42c4-b6d8-050c42ad5541-metrics-tls podName:776a4785-ccb6-42c4-b6d8-050c42ad5541 nodeName:}" failed. No retries permitted until 2026-05-21 00:55:19.252435286 +0000 UTC m=+38.149380576 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/776a4785-ccb6-42c4-b6d8-050c42ad5541-metrics-tls") pod "dns-default-5mlhs" (UID: "776a4785-ccb6-42c4-b6d8-050c42ad5541") : secret "dns-default-metrics-tls" not found May 21 00:55:17.817732 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:17.817648 2549 generic.go:358] "Generic (PLEG): container finished" podID="5da3aae9-eaf5-4328-afed-6595832fc2e1" containerID="16e951f9a1d0f24e7c1dc48fdbd497977e0542515385539002d2cabd6476707f" exitCode=0 May 21 00:55:17.817854 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:17.817728 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-spsr8" event={"ID":"5da3aae9-eaf5-4328-afed-6595832fc2e1","Type":"ContainerDied","Data":"16e951f9a1d0f24e7c1dc48fdbd497977e0542515385539002d2cabd6476707f"} May 21 00:55:18.822835 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:18.822801 2549 generic.go:358] "Generic (PLEG): container finished" podID="5da3aae9-eaf5-4328-afed-6595832fc2e1" containerID="c8fc7e0924956c66402271eaf31a6547ad7ae486545e9bb8969db1ac38cb9e05" exitCode=0 May 21 00:55:18.823348 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:18.822866 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-spsr8" event={"ID":"5da3aae9-eaf5-4328-afed-6595832fc2e1","Type":"ContainerDied","Data":"c8fc7e0924956c66402271eaf31a6547ad7ae486545e9bb8969db1ac38cb9e05"} May 21 00:55:19.265181 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:19.265145 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/776a4785-ccb6-42c4-b6d8-050c42ad5541-metrics-tls\") pod \"dns-default-5mlhs\" (UID: \"776a4785-ccb6-42c4-b6d8-050c42ad5541\") " pod="openshift-dns/dns-default-5mlhs" May 21 00:55:19.265340 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:19.265209 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/877e95c2-b86f-49d3-a207-eb91f732bfff-cert\") pod \"ingress-canary-2lhq8\" (UID: \"877e95c2-b86f-49d3-a207-eb91f732bfff\") " pod="openshift-ingress-canary/ingress-canary-2lhq8" May 21 00:55:19.265340 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:19.265292 2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 21 00:55:19.265340 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:19.265310 2549 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 21 00:55:19.265461 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:19.265359 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776a4785-ccb6-42c4-b6d8-050c42ad5541-metrics-tls podName:776a4785-ccb6-42c4-b6d8-050c42ad5541 nodeName:}" failed. No retries permitted until 2026-05-21 00:55:23.265342394 +0000 UTC m=+42.162287682 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/776a4785-ccb6-42c4-b6d8-050c42ad5541-metrics-tls") pod "dns-default-5mlhs" (UID: "776a4785-ccb6-42c4-b6d8-050c42ad5541") : secret "dns-default-metrics-tls" not found May 21 00:55:19.265461 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:19.265376 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/877e95c2-b86f-49d3-a207-eb91f732bfff-cert podName:877e95c2-b86f-49d3-a207-eb91f732bfff nodeName:}" failed. No retries permitted until 2026-05-21 00:55:23.265367917 +0000 UTC m=+42.162313206 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/877e95c2-b86f-49d3-a207-eb91f732bfff-cert") pod "ingress-canary-2lhq8" (UID: "877e95c2-b86f-49d3-a207-eb91f732bfff") : secret "canary-serving-cert" not found May 21 00:55:19.827738 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:19.827700 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-spsr8" event={"ID":"5da3aae9-eaf5-4328-afed-6595832fc2e1","Type":"ContainerStarted","Data":"d10c423600f19cce9d0a41571c1fb90d91a45b26521b3ff0203cf1d49bafcf1b"} May 21 00:55:19.851407 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:19.851360 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-spsr8" podStartSLOduration=5.761538922 podStartE2EDuration="38.851345029s" podCreationTimestamp="2026-05-21 00:54:41 +0000 UTC" firstStartedPulling="2026-05-21 00:54:44.326461317 +0000 UTC m=+3.223406626" lastFinishedPulling="2026-05-21 00:55:17.416267436 +0000 UTC m=+36.313212733" observedRunningTime="2026-05-21 00:55:19.850171598 +0000 UTC m=+38.747116908" watchObservedRunningTime="2026-05-21 00:55:19.851345029 +0000 UTC m=+38.748290546" May 21 00:55:23.295475 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:23.295429 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/776a4785-ccb6-42c4-b6d8-050c42ad5541-metrics-tls\") pod \"dns-default-5mlhs\" (UID: \"776a4785-ccb6-42c4-b6d8-050c42ad5541\") " pod="openshift-dns/dns-default-5mlhs" May 21 00:55:23.295875 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:23.295509 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/877e95c2-b86f-49d3-a207-eb91f732bfff-cert\") pod \"ingress-canary-2lhq8\" (UID: \"877e95c2-b86f-49d3-a207-eb91f732bfff\") " pod="openshift-ingress-canary/ingress-canary-2lhq8" May 21 00:55:23.295875 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:23.295578 2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 21 00:55:23.295875 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:23.295641 2549 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 21 00:55:23.295875 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:23.295650 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776a4785-ccb6-42c4-b6d8-050c42ad5541-metrics-tls podName:776a4785-ccb6-42c4-b6d8-050c42ad5541 nodeName:}" failed. No retries permitted until 2026-05-21 00:55:31.295633218 +0000 UTC m=+50.192578524 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/776a4785-ccb6-42c4-b6d8-050c42ad5541-metrics-tls") pod "dns-default-5mlhs" (UID: "776a4785-ccb6-42c4-b6d8-050c42ad5541") : secret "dns-default-metrics-tls" not found May 21 00:55:23.295875 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:23.295697 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/877e95c2-b86f-49d3-a207-eb91f732bfff-cert podName:877e95c2-b86f-49d3-a207-eb91f732bfff nodeName:}" failed. No retries permitted until 2026-05-21 00:55:31.295684975 +0000 UTC m=+50.192630271 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/877e95c2-b86f-49d3-a207-eb91f732bfff-cert") pod "ingress-canary-2lhq8" (UID: "877e95c2-b86f-49d3-a207-eb91f732bfff") : secret "canary-serving-cert" not found May 21 00:55:23.899971 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:23.899918 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f8b563f-d153-48f5-ab5e-836807f76cad-original-pull-secret\") pod \"global-pull-secret-syncer-n4j7l\" (UID: \"8f8b563f-d153-48f5-ab5e-836807f76cad\") " pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:55:23.903145 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:23.903125 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f8b563f-d153-48f5-ab5e-836807f76cad-original-pull-secret\") pod \"global-pull-secret-syncer-n4j7l\" (UID: \"8f8b563f-d153-48f5-ab5e-836807f76cad\") " pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:55:24.095134 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:24.095094 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4j7l" May 21 00:55:24.232002 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:24.231967 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n4j7l"] May 21 00:55:24.235744 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:55:24.235705 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f8b563f_d153_48f5_ab5e_836807f76cad.slice/crio-353a48849595acf8a335dee42f1338be2a9fc7b599f97d0d3c6f9d407fb833a0 WatchSource:0}: Error finding container 353a48849595acf8a335dee42f1338be2a9fc7b599f97d0d3c6f9d407fb833a0: Status 404 returned error can't find the container with id 353a48849595acf8a335dee42f1338be2a9fc7b599f97d0d3c6f9d407fb833a0 May 21 00:55:24.838287 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:24.838248 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n4j7l" event={"ID":"8f8b563f-d153-48f5-ab5e-836807f76cad","Type":"ContainerStarted","Data":"353a48849595acf8a335dee42f1338be2a9fc7b599f97d0d3c6f9d407fb833a0"} May 21 00:55:28.846571 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:28.846535 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n4j7l" event={"ID":"8f8b563f-d153-48f5-ab5e-836807f76cad","Type":"ContainerStarted","Data":"46ae37bbf339188bc6d49cf7bb97aa78ff1cd4acdfbf45a49134a3a7a023bb9a"} May 21 00:55:28.861138 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:28.861096 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-n4j7l" podStartSLOduration=32.644061861 podStartE2EDuration="36.861085419s" podCreationTimestamp="2026-05-21 00:54:52 +0000 UTC" firstStartedPulling="2026-05-21 00:55:24.237367319 +0000 UTC m=+43.134312608" lastFinishedPulling="2026-05-21 00:55:28.454390873 +0000 UTC m=+47.351336166" observedRunningTime="2026-05-21 00:55:28.860943703 +0000 UTC m=+47.757889024" watchObservedRunningTime="2026-05-21 00:55:28.861085419 +0000 UTC m=+47.758030730" May 21 00:55:31.354748 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:31.354708 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/776a4785-ccb6-42c4-b6d8-050c42ad5541-metrics-tls\") pod \"dns-default-5mlhs\" (UID: \"776a4785-ccb6-42c4-b6d8-050c42ad5541\") " pod="openshift-dns/dns-default-5mlhs" May 21 00:55:31.355252 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:31.354759 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/877e95c2-b86f-49d3-a207-eb91f732bfff-cert\") pod \"ingress-canary-2lhq8\" (UID: \"877e95c2-b86f-49d3-a207-eb91f732bfff\") " pod="openshift-ingress-canary/ingress-canary-2lhq8" May 21 00:55:31.355252 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:31.354849 2549 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 21 00:55:31.355252 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:31.354852 2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 21 00:55:31.355252 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:31.354902 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/877e95c2-b86f-49d3-a207-eb91f732bfff-cert podName:877e95c2-b86f-49d3-a207-eb91f732bfff nodeName:}" failed. No retries permitted until 2026-05-21 00:55:47.354888327 +0000 UTC m=+66.251833616 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/877e95c2-b86f-49d3-a207-eb91f732bfff-cert") pod "ingress-canary-2lhq8" (UID: "877e95c2-b86f-49d3-a207-eb91f732bfff") : secret "canary-serving-cert" not found May 21 00:55:31.355252 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:31.354916 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776a4785-ccb6-42c4-b6d8-050c42ad5541-metrics-tls podName:776a4785-ccb6-42c4-b6d8-050c42ad5541 nodeName:}" failed. No retries permitted until 2026-05-21 00:55:47.35490881 +0000 UTC m=+66.251854098 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/776a4785-ccb6-42c4-b6d8-050c42ad5541-metrics-tls") pod "dns-default-5mlhs" (UID: "776a4785-ccb6-42c4-b6d8-050c42ad5541") : secret "dns-default-metrics-tls" not found May 21 00:55:40.809883 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:40.809853 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kgxp9" May 21 00:55:47.366834 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:47.366793 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-metrics-certs\") pod \"network-metrics-daemon-skbch\" (UID: \"5ee0c301-12eb-4c68-9990-6ceafb2c90fa\") " pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:55:47.367364 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:47.366847 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-66pbg\" (UniqueName: \"kubernetes.io/projected/3b778703-acf8-4977-a1b1-2f5f9042d2b3-kube-api-access-66pbg\") pod \"network-check-target-rxzxp\" (UID: \"3b778703-acf8-4977-a1b1-2f5f9042d2b3\") " pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:55:47.367364 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:47.366877 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/776a4785-ccb6-42c4-b6d8-050c42ad5541-metrics-tls\") pod \"dns-default-5mlhs\" (UID: \"776a4785-ccb6-42c4-b6d8-050c42ad5541\") " pod="openshift-dns/dns-default-5mlhs" May 21 00:55:47.367364 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:47.366920 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/877e95c2-b86f-49d3-a207-eb91f732bfff-cert\") pod \"ingress-canary-2lhq8\" (UID: \"877e95c2-b86f-49d3-a207-eb91f732bfff\") " pod="openshift-ingress-canary/ingress-canary-2lhq8" May 21 00:55:47.367364 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:47.367022 2549 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 21 00:55:47.367364 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:47.367021 2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 21 00:55:47.367364 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:47.367114 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/877e95c2-b86f-49d3-a207-eb91f732bfff-cert podName:877e95c2-b86f-49d3-a207-eb91f732bfff nodeName:}" failed. No retries permitted until 2026-05-21 00:56:19.367093219 +0000 UTC m=+98.264038527 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/877e95c2-b86f-49d3-a207-eb91f732bfff-cert") pod "ingress-canary-2lhq8" (UID: "877e95c2-b86f-49d3-a207-eb91f732bfff") : secret "canary-serving-cert" not found May 21 00:55:47.367364 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:47.367149 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776a4785-ccb6-42c4-b6d8-050c42ad5541-metrics-tls podName:776a4785-ccb6-42c4-b6d8-050c42ad5541 nodeName:}" failed. No retries permitted until 2026-05-21 00:56:19.367135084 +0000 UTC m=+98.264080378 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/776a4785-ccb6-42c4-b6d8-050c42ad5541-metrics-tls") pod "dns-default-5mlhs" (UID: "776a4785-ccb6-42c4-b6d8-050c42ad5541") : secret "dns-default-metrics-tls" not found May 21 00:55:47.369714 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:47.369696 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 21 00:55:47.369819 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:47.369729 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 21 00:55:47.377530 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:47.377514 2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found May 21 00:55:47.377607 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:55:47.377582 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-metrics-certs podName:5ee0c301-12eb-4c68-9990-6ceafb2c90fa nodeName:}" failed. No retries permitted until 2026-05-21 00:56:51.377570016 +0000 UTC m=+130.274515309 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-metrics-certs") pod "network-metrics-daemon-skbch" (UID: "5ee0c301-12eb-4c68-9990-6ceafb2c90fa") : secret "metrics-daemon-secret" not found May 21 00:55:47.379335 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:47.379319 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 21 00:55:47.390807 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:47.390787 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-66pbg\" (UniqueName: \"kubernetes.io/projected/3b778703-acf8-4977-a1b1-2f5f9042d2b3-kube-api-access-66pbg\") pod \"network-check-target-rxzxp\" (UID: \"3b778703-acf8-4977-a1b1-2f5f9042d2b3\") " pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:55:47.484130 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:47.484108 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fhwfb\"" May 21 00:55:47.491986 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:47.491964 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:55:47.603706 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:47.603674 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rxzxp"] May 21 00:55:47.606741 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:55:47.606710 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b778703_acf8_4977_a1b1_2f5f9042d2b3.slice/crio-09a405e66809bd499013834ef3631482fd30db1a5756f6a4cb58c18011c28103 WatchSource:0}: Error finding container 09a405e66809bd499013834ef3631482fd30db1a5756f6a4cb58c18011c28103: Status 404 returned error can't find the container with id 09a405e66809bd499013834ef3631482fd30db1a5756f6a4cb58c18011c28103 May 21 00:55:47.879160 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:47.879107 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rxzxp" event={"ID":"3b778703-acf8-4977-a1b1-2f5f9042d2b3","Type":"ContainerStarted","Data":"09a405e66809bd499013834ef3631482fd30db1a5756f6a4cb58c18011c28103"} May 21 00:55:50.886866 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:50.886831 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rxzxp" event={"ID":"3b778703-acf8-4977-a1b1-2f5f9042d2b3","Type":"ContainerStarted","Data":"b8f5ef0bd82bef553d1efaf52d4a0d6a5c6f81d4b9622092b6ccc63db4e8204b"} May 21 00:55:50.887247 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:50.886972 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:55:50.902110 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:55:50.902042 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rxzxp" podStartSLOduration=67.242793501 podStartE2EDuration="1m9.902030557s" podCreationTimestamp="2026-05-21 00:54:41 +0000 UTC" firstStartedPulling="2026-05-21 00:55:47.608536597 +0000 UTC m=+66.505481886" lastFinishedPulling="2026-05-21 00:55:50.267773653 +0000 UTC m=+69.164718942" observedRunningTime="2026-05-21 00:55:50.901310726 +0000 UTC m=+69.798256037" watchObservedRunningTime="2026-05-21 00:55:50.902030557 +0000 UTC m=+69.798975914" May 21 00:56:09.516901 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:09.516793 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-685d6d5876-58bgk"] May 21 00:56:09.520507 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:09.519828 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:09.522321 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:09.522302 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" May 21 00:56:09.522712 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:09.522694 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" May 21 00:56:09.522904 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:09.522889 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" May 21 00:56:09.522983 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:09.522924 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" May 21 00:56:09.522983 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:09.522956 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-rp6fr\"" May 21 00:56:09.523350 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:09.523338 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" May 21 00:56:09.523449 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:09.523435 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" May 21 00:56:09.530292 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:09.530266 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-685d6d5876-58bgk"] May 21 00:56:09.622234 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:09.622213 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6891f821-6e83-4a11-80c9-34bf3144c9ed-service-ca-bundle\") pod \"router-default-685d6d5876-58bgk\" (UID: \"6891f821-6e83-4a11-80c9-34bf3144c9ed\") " pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:09.622325 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:09.622249 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6891f821-6e83-4a11-80c9-34bf3144c9ed-stats-auth\") pod \"router-default-685d6d5876-58bgk\" (UID: \"6891f821-6e83-4a11-80c9-34bf3144c9ed\") " pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:09.622325 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:09.622293 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6891f821-6e83-4a11-80c9-34bf3144c9ed-metrics-certs\") pod \"router-default-685d6d5876-58bgk\" (UID: \"6891f821-6e83-4a11-80c9-34bf3144c9ed\") " pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:09.622325 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:09.622319 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flwbs\" (UniqueName: \"kubernetes.io/projected/6891f821-6e83-4a11-80c9-34bf3144c9ed-kube-api-access-flwbs\") pod \"router-default-685d6d5876-58bgk\" (UID: \"6891f821-6e83-4a11-80c9-34bf3144c9ed\") " pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:09.622427 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:09.622363 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6891f821-6e83-4a11-80c9-34bf3144c9ed-default-certificate\") pod \"router-default-685d6d5876-58bgk\" (UID: \"6891f821-6e83-4a11-80c9-34bf3144c9ed\") " pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:09.723576 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:09.723550 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6891f821-6e83-4a11-80c9-34bf3144c9ed-service-ca-bundle\") pod \"router-default-685d6d5876-58bgk\" (UID: \"6891f821-6e83-4a11-80c9-34bf3144c9ed\") " pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:09.723662 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:09.723587 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6891f821-6e83-4a11-80c9-34bf3144c9ed-stats-auth\") pod \"router-default-685d6d5876-58bgk\" (UID: \"6891f821-6e83-4a11-80c9-34bf3144c9ed\") " pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:09.723662 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:09.723618 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6891f821-6e83-4a11-80c9-34bf3144c9ed-metrics-certs\") pod \"router-default-685d6d5876-58bgk\" (UID: \"6891f821-6e83-4a11-80c9-34bf3144c9ed\") " pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:09.723662 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:09.723644 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-flwbs\" (UniqueName: \"kubernetes.io/projected/6891f821-6e83-4a11-80c9-34bf3144c9ed-kube-api-access-flwbs\") pod \"router-default-685d6d5876-58bgk\" (UID: \"6891f821-6e83-4a11-80c9-34bf3144c9ed\") " pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:09.723776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:09.723676 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6891f821-6e83-4a11-80c9-34bf3144c9ed-default-certificate\") pod \"router-default-685d6d5876-58bgk\" (UID: \"6891f821-6e83-4a11-80c9-34bf3144c9ed\") " pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:09.723776 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:09.723712 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6891f821-6e83-4a11-80c9-34bf3144c9ed-service-ca-bundle podName:6891f821-6e83-4a11-80c9-34bf3144c9ed nodeName:}" failed. No retries permitted until 2026-05-21 00:56:10.223690579 +0000 UTC m=+89.120635886 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6891f821-6e83-4a11-80c9-34bf3144c9ed-service-ca-bundle") pod "router-default-685d6d5876-58bgk" (UID: "6891f821-6e83-4a11-80c9-34bf3144c9ed") : configmap references non-existent config key: service-ca.crt May 21 00:56:09.723776 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:09.723759 2549 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 21 00:56:09.723924 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:09.723817 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6891f821-6e83-4a11-80c9-34bf3144c9ed-metrics-certs podName:6891f821-6e83-4a11-80c9-34bf3144c9ed nodeName:}" failed. No retries permitted until 2026-05-21 00:56:10.223802469 +0000 UTC m=+89.120747771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6891f821-6e83-4a11-80c9-34bf3144c9ed-metrics-certs") pod "router-default-685d6d5876-58bgk" (UID: "6891f821-6e83-4a11-80c9-34bf3144c9ed") : secret "router-metrics-certs-default" not found May 21 00:56:09.725967 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:09.725945 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6891f821-6e83-4a11-80c9-34bf3144c9ed-stats-auth\") pod \"router-default-685d6d5876-58bgk\" (UID: \"6891f821-6e83-4a11-80c9-34bf3144c9ed\") " pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:09.726080 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:09.725979 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6891f821-6e83-4a11-80c9-34bf3144c9ed-default-certificate\") pod \"router-default-685d6d5876-58bgk\" (UID: \"6891f821-6e83-4a11-80c9-34bf3144c9ed\") " pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:09.731568 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:09.731547 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flwbs\" (UniqueName: \"kubernetes.io/projected/6891f821-6e83-4a11-80c9-34bf3144c9ed-kube-api-access-flwbs\") pod \"router-default-685d6d5876-58bgk\" (UID: \"6891f821-6e83-4a11-80c9-34bf3144c9ed\") " pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:10.227633 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:10.227586 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6891f821-6e83-4a11-80c9-34bf3144c9ed-metrics-certs\") pod \"router-default-685d6d5876-58bgk\" (UID: \"6891f821-6e83-4a11-80c9-34bf3144c9ed\") " pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:10.227812 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:10.227665 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6891f821-6e83-4a11-80c9-34bf3144c9ed-service-ca-bundle\") pod \"router-default-685d6d5876-58bgk\" (UID: \"6891f821-6e83-4a11-80c9-34bf3144c9ed\") " pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:10.227812 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:10.227727 2549 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 21 00:56:10.227812 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:10.227804 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6891f821-6e83-4a11-80c9-34bf3144c9ed-service-ca-bundle podName:6891f821-6e83-4a11-80c9-34bf3144c9ed nodeName:}" failed. No retries permitted until 2026-05-21 00:56:11.227786427 +0000 UTC m=+90.124731716 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6891f821-6e83-4a11-80c9-34bf3144c9ed-service-ca-bundle") pod "router-default-685d6d5876-58bgk" (UID: "6891f821-6e83-4a11-80c9-34bf3144c9ed") : configmap references non-existent config key: service-ca.crt May 21 00:56:10.227921 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:10.227828 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6891f821-6e83-4a11-80c9-34bf3144c9ed-metrics-certs podName:6891f821-6e83-4a11-80c9-34bf3144c9ed nodeName:}" failed. No retries permitted until 2026-05-21 00:56:11.227817846 +0000 UTC m=+90.124763134 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6891f821-6e83-4a11-80c9-34bf3144c9ed-metrics-certs") pod "router-default-685d6d5876-58bgk" (UID: "6891f821-6e83-4a11-80c9-34bf3144c9ed") : secret "router-metrics-certs-default" not found May 21 00:56:11.234433 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:11.234397 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6891f821-6e83-4a11-80c9-34bf3144c9ed-metrics-certs\") pod \"router-default-685d6d5876-58bgk\" (UID: \"6891f821-6e83-4a11-80c9-34bf3144c9ed\") " pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:11.234812 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:11.234460 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6891f821-6e83-4a11-80c9-34bf3144c9ed-service-ca-bundle\") pod \"router-default-685d6d5876-58bgk\" (UID: \"6891f821-6e83-4a11-80c9-34bf3144c9ed\") " pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:11.234812 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:11.234544 2549 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 21 00:56:11.234812 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:11.234578 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6891f821-6e83-4a11-80c9-34bf3144c9ed-service-ca-bundle podName:6891f821-6e83-4a11-80c9-34bf3144c9ed nodeName:}" failed. No retries permitted until 2026-05-21 00:56:13.234563424 +0000 UTC m=+92.131508733 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6891f821-6e83-4a11-80c9-34bf3144c9ed-service-ca-bundle") pod "router-default-685d6d5876-58bgk" (UID: "6891f821-6e83-4a11-80c9-34bf3144c9ed") : configmap references non-existent config key: service-ca.crt May 21 00:56:11.234812 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:11.234608 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6891f821-6e83-4a11-80c9-34bf3144c9ed-metrics-certs podName:6891f821-6e83-4a11-80c9-34bf3144c9ed nodeName:}" failed. No retries permitted until 2026-05-21 00:56:13.234593282 +0000 UTC m=+92.131538571 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6891f821-6e83-4a11-80c9-34bf3144c9ed-metrics-certs") pod "router-default-685d6d5876-58bgk" (UID: "6891f821-6e83-4a11-80c9-34bf3144c9ed") : secret "router-metrics-certs-default" not found May 21 00:56:13.248572 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:13.248527 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6891f821-6e83-4a11-80c9-34bf3144c9ed-service-ca-bundle\") pod \"router-default-685d6d5876-58bgk\" (UID: \"6891f821-6e83-4a11-80c9-34bf3144c9ed\") " pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:13.248938 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:13.248590 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6891f821-6e83-4a11-80c9-34bf3144c9ed-metrics-certs\") pod \"router-default-685d6d5876-58bgk\" (UID: \"6891f821-6e83-4a11-80c9-34bf3144c9ed\") " pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:13.248938 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:13.248680 2549 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 21 00:56:13.248938 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:13.248712 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6891f821-6e83-4a11-80c9-34bf3144c9ed-service-ca-bundle podName:6891f821-6e83-4a11-80c9-34bf3144c9ed nodeName:}" failed. No retries permitted until 2026-05-21 00:56:17.248695352 +0000 UTC m=+96.145640641 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6891f821-6e83-4a11-80c9-34bf3144c9ed-service-ca-bundle") pod "router-default-685d6d5876-58bgk" (UID: "6891f821-6e83-4a11-80c9-34bf3144c9ed") : configmap references non-existent config key: service-ca.crt May 21 00:56:13.248938 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:13.248734 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6891f821-6e83-4a11-80c9-34bf3144c9ed-metrics-certs podName:6891f821-6e83-4a11-80c9-34bf3144c9ed nodeName:}" failed. No retries permitted until 2026-05-21 00:56:17.248727057 +0000 UTC m=+96.145672346 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6891f821-6e83-4a11-80c9-34bf3144c9ed-metrics-certs") pod "router-default-685d6d5876-58bgk" (UID: "6891f821-6e83-4a11-80c9-34bf3144c9ed") : secret "router-metrics-certs-default" not found May 21 00:56:15.622121 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:15.622094 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-96x9m_31374768-4e56-404b-8d22-7e2136138fa4/dns-node-resolver/0.log" May 21 00:56:16.621969 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:16.621948 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sfsvt_0a1071e0-9724-4546-8b54-44bce04c35c2/node-ca/0.log" May 21 00:56:17.275613 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:17.275581 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6891f821-6e83-4a11-80c9-34bf3144c9ed-metrics-certs\") pod \"router-default-685d6d5876-58bgk\" (UID: \"6891f821-6e83-4a11-80c9-34bf3144c9ed\") " pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:17.275996 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:17.275640 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6891f821-6e83-4a11-80c9-34bf3144c9ed-service-ca-bundle\") pod \"router-default-685d6d5876-58bgk\" (UID: \"6891f821-6e83-4a11-80c9-34bf3144c9ed\") " pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:17.275996 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:17.275721 2549 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 21 00:56:17.275996 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:17.275759 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6891f821-6e83-4a11-80c9-34bf3144c9ed-service-ca-bundle podName:6891f821-6e83-4a11-80c9-34bf3144c9ed nodeName:}" failed. No retries permitted until 2026-05-21 00:56:25.275745113 +0000 UTC m=+104.172690403 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6891f821-6e83-4a11-80c9-34bf3144c9ed-service-ca-bundle") pod "router-default-685d6d5876-58bgk" (UID: "6891f821-6e83-4a11-80c9-34bf3144c9ed") : configmap references non-existent config key: service-ca.crt May 21 00:56:17.275996 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:17.275784 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6891f821-6e83-4a11-80c9-34bf3144c9ed-metrics-certs podName:6891f821-6e83-4a11-80c9-34bf3144c9ed nodeName:}" failed. No retries permitted until 2026-05-21 00:56:25.275770504 +0000 UTC m=+104.172715797 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6891f821-6e83-4a11-80c9-34bf3144c9ed-metrics-certs") pod "router-default-685d6d5876-58bgk" (UID: "6891f821-6e83-4a11-80c9-34bf3144c9ed") : secret "router-metrics-certs-default" not found May 21 00:56:19.388867 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:19.388832 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/776a4785-ccb6-42c4-b6d8-050c42ad5541-metrics-tls\") pod \"dns-default-5mlhs\" (UID: \"776a4785-ccb6-42c4-b6d8-050c42ad5541\") " pod="openshift-dns/dns-default-5mlhs" May 21 00:56:19.389264 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:19.388892 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/877e95c2-b86f-49d3-a207-eb91f732bfff-cert\") pod \"ingress-canary-2lhq8\" (UID: \"877e95c2-b86f-49d3-a207-eb91f732bfff\") " pod="openshift-ingress-canary/ingress-canary-2lhq8" May 21 00:56:19.389264 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:19.388977 2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 21 00:56:19.389264 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:19.388993 2549 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 21 00:56:19.389264 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:19.389041 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776a4785-ccb6-42c4-b6d8-050c42ad5541-metrics-tls podName:776a4785-ccb6-42c4-b6d8-050c42ad5541 nodeName:}" failed. No retries permitted until 2026-05-21 00:57:23.38902466 +0000 UTC m=+162.285969950 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/776a4785-ccb6-42c4-b6d8-050c42ad5541-metrics-tls") pod "dns-default-5mlhs" (UID: "776a4785-ccb6-42c4-b6d8-050c42ad5541") : secret "dns-default-metrics-tls" not found May 21 00:56:19.389264 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:19.389055 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/877e95c2-b86f-49d3-a207-eb91f732bfff-cert podName:877e95c2-b86f-49d3-a207-eb91f732bfff nodeName:}" failed. No retries permitted until 2026-05-21 00:57:23.389049325 +0000 UTC m=+162.285994614 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/877e95c2-b86f-49d3-a207-eb91f732bfff-cert") pod "ingress-canary-2lhq8" (UID: "877e95c2-b86f-49d3-a207-eb91f732bfff") : secret "canary-serving-cert" not found May 21 00:56:21.891293 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:21.891257 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rxzxp" May 21 00:56:24.322158 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.322127 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-d8ttl"] May 21 00:56:24.324617 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.324599 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-d8ttl" May 21 00:56:24.328563 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.328528 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" May 21 00:56:24.328672 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.328568 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" May 21 00:56:24.329217 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.329200 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-qfs57\"" May 21 00:56:24.331352 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.331335 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-d8ttl"] May 21 00:56:24.424130 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.424104 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-9d999d7b8-pw9kx"] May 21 00:56:24.425696 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.425681 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:24.425871 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.425847 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tk4p\" (UniqueName: \"kubernetes.io/projected/02d56ea6-c685-4f09-b951-40d453ba0a31-kube-api-access-6tk4p\") pod \"migrator-b985f5d89-d8ttl\" (UID: \"02d56ea6-c685-4f09-b951-40d453ba0a31\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-d8ttl" May 21 00:56:24.427968 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.427946 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" May 21 00:56:24.428094 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.428039 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" May 21 00:56:24.428094 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.428082 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7258q\"" May 21 00:56:24.428223 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.428206 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" May 21 00:56:24.432660 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.432639 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" May 21 00:56:24.437944 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.437924 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9d999d7b8-pw9kx"] May 21 00:56:24.526906 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.526867 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6tk4p\" (UniqueName: \"kubernetes.io/projected/02d56ea6-c685-4f09-b951-40d453ba0a31-kube-api-access-6tk4p\") pod \"migrator-b985f5d89-d8ttl\" (UID: \"02d56ea6-c685-4f09-b951-40d453ba0a31\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-d8ttl" May 21 00:56:24.527113 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.526912 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/68d5c953-f922-4671-b7ba-5436cb108e4e-registry-certificates\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:24.527113 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.526965 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/68d5c953-f922-4671-b7ba-5436cb108e4e-image-registry-private-configuration\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:24.527113 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.526995 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-bound-sa-token\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:24.527113 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.527020 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/68d5c953-f922-4671-b7ba-5436cb108e4e-installation-pull-secrets\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:24.527113 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.527094 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7gmt\" (UniqueName: \"kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-kube-api-access-b7gmt\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:24.527337 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.527131 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-registry-tls\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:24.527337 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.527162 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/68d5c953-f922-4671-b7ba-5436cb108e4e-ca-trust-extracted\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:24.527337 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.527204 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/68d5c953-f922-4671-b7ba-5436cb108e4e-trusted-ca\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:24.534219 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.534196 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tk4p\" (UniqueName: \"kubernetes.io/projected/02d56ea6-c685-4f09-b951-40d453ba0a31-kube-api-access-6tk4p\") pod \"migrator-b985f5d89-d8ttl\" (UID: \"02d56ea6-c685-4f09-b951-40d453ba0a31\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-d8ttl" May 21 00:56:24.628562 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.628472 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/68d5c953-f922-4671-b7ba-5436cb108e4e-ca-trust-extracted\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:24.628562 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.628543 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/68d5c953-f922-4671-b7ba-5436cb108e4e-trusted-ca\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:24.628709 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.628584 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/68d5c953-f922-4671-b7ba-5436cb108e4e-registry-certificates\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:24.628709 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.628620 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/68d5c953-f922-4671-b7ba-5436cb108e4e-image-registry-private-configuration\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:24.628709 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.628640 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-bound-sa-token\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:24.628709 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.628665 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/68d5c953-f922-4671-b7ba-5436cb108e4e-installation-pull-secrets\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:24.628709 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.628690 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b7gmt\" (UniqueName: \"kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-kube-api-access-b7gmt\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:24.628932 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.628714 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-registry-tls\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:24.628932 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:24.628812 2549 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 21 00:56:24.628932 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:24.628825 2549 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9d999d7b8-pw9kx: secret "image-registry-tls" not found May 21 00:56:24.628932 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:24.628874 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-registry-tls podName:68d5c953-f922-4671-b7ba-5436cb108e4e nodeName:}" failed. No retries permitted until 2026-05-21 00:56:25.128856203 +0000 UTC m=+104.025801493 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-registry-tls") pod "image-registry-9d999d7b8-pw9kx" (UID: "68d5c953-f922-4671-b7ba-5436cb108e4e") : secret "image-registry-tls" not found May 21 00:56:24.628932 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.628907 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/68d5c953-f922-4671-b7ba-5436cb108e4e-ca-trust-extracted\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:24.629222 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.629196 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/68d5c953-f922-4671-b7ba-5436cb108e4e-registry-certificates\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:24.629668 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.629646 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/68d5c953-f922-4671-b7ba-5436cb108e4e-trusted-ca\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:24.631021 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.631002 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/68d5c953-f922-4671-b7ba-5436cb108e4e-installation-pull-secrets\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:24.631150 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.631135 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/68d5c953-f922-4671-b7ba-5436cb108e4e-image-registry-private-configuration\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:24.637832 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.637811 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-d8ttl" May 21 00:56:24.638704 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.638603 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-bound-sa-token\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:24.638704 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.638631 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7gmt\" (UniqueName: \"kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-kube-api-access-b7gmt\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:24.749113 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.749085 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-d8ttl"] May 21 00:56:24.752115 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:56:24.752092 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02d56ea6_c685_4f09_b951_40d453ba0a31.slice/crio-6450508c370897e6638119bc4f1c311b3bfc4b4c0a2b9c2654bc7e597a350d7f WatchSource:0}: Error finding container 6450508c370897e6638119bc4f1c311b3bfc4b4c0a2b9c2654bc7e597a350d7f: Status 404 returned error can't find the container with id 6450508c370897e6638119bc4f1c311b3bfc4b4c0a2b9c2654bc7e597a350d7f May 21 00:56:24.950126 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:24.950025 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-d8ttl" event={"ID":"02d56ea6-c685-4f09-b951-40d453ba0a31","Type":"ContainerStarted","Data":"6450508c370897e6638119bc4f1c311b3bfc4b4c0a2b9c2654bc7e597a350d7f"} May 21 00:56:25.132663 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:25.132621 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-registry-tls\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:25.132896 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:25.132765 2549 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 21 00:56:25.132896 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:25.132782 2549 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9d999d7b8-pw9kx: secret "image-registry-tls" not found May 21 00:56:25.132896 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:25.132834 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-registry-tls podName:68d5c953-f922-4671-b7ba-5436cb108e4e nodeName:}" failed. No retries permitted until 2026-05-21 00:56:26.13281896 +0000 UTC m=+105.029764249 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-registry-tls") pod "image-registry-9d999d7b8-pw9kx" (UID: "68d5c953-f922-4671-b7ba-5436cb108e4e") : secret "image-registry-tls" not found May 21 00:56:25.334031 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:25.333999 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6891f821-6e83-4a11-80c9-34bf3144c9ed-metrics-certs\") pod \"router-default-685d6d5876-58bgk\" (UID: \"6891f821-6e83-4a11-80c9-34bf3144c9ed\") " pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:25.334396 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:25.334115 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6891f821-6e83-4a11-80c9-34bf3144c9ed-service-ca-bundle\") pod \"router-default-685d6d5876-58bgk\" (UID: \"6891f821-6e83-4a11-80c9-34bf3144c9ed\") " pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:25.334396 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:25.334138 2549 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 21 00:56:25.334396 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:25.334239 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6891f821-6e83-4a11-80c9-34bf3144c9ed-metrics-certs podName:6891f821-6e83-4a11-80c9-34bf3144c9ed nodeName:}" failed. No retries permitted until 2026-05-21 00:56:41.334218881 +0000 UTC m=+120.231164172 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6891f821-6e83-4a11-80c9-34bf3144c9ed-metrics-certs") pod "router-default-685d6d5876-58bgk" (UID: "6891f821-6e83-4a11-80c9-34bf3144c9ed") : secret "router-metrics-certs-default" not found May 21 00:56:25.334396 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:25.334254 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6891f821-6e83-4a11-80c9-34bf3144c9ed-service-ca-bundle podName:6891f821-6e83-4a11-80c9-34bf3144c9ed nodeName:}" failed. No retries permitted until 2026-05-21 00:56:41.334247991 +0000 UTC m=+120.231193279 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6891f821-6e83-4a11-80c9-34bf3144c9ed-service-ca-bundle") pod "router-default-685d6d5876-58bgk" (UID: "6891f821-6e83-4a11-80c9-34bf3144c9ed") : configmap references non-existent config key: service-ca.crt May 21 00:56:26.047075 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:26.047034 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-xvxlc"] May 21 00:56:26.049094 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:26.049073 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-xvxlc" May 21 00:56:26.051295 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:26.051272 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-jsfm7\"" May 21 00:56:26.055677 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:26.055652 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-xvxlc"] May 21 00:56:26.141906 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:26.141876 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-registry-tls\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:26.142118 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:26.142025 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92b4b\" (UniqueName: \"kubernetes.io/projected/88201346-80c9-4e71-8c22-d42722339e1f-kube-api-access-92b4b\") pod \"network-check-source-7f44b6dd87-xvxlc\" (UID: \"88201346-80c9-4e71-8c22-d42722339e1f\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-xvxlc" May 21 00:56:26.142118 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:26.142044 2549 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 21 00:56:26.142118 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:26.142081 2549 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9d999d7b8-pw9kx: secret "image-registry-tls" not found May 21 00:56:26.142285 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:26.142150 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-registry-tls podName:68d5c953-f922-4671-b7ba-5436cb108e4e nodeName:}" failed. No retries permitted until 2026-05-21 00:56:28.14213009 +0000 UTC m=+107.039075387 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-registry-tls") pod "image-registry-9d999d7b8-pw9kx" (UID: "68d5c953-f922-4671-b7ba-5436cb108e4e") : secret "image-registry-tls" not found May 21 00:56:26.245384 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:26.245351 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-92b4b\" (UniqueName: \"kubernetes.io/projected/88201346-80c9-4e71-8c22-d42722339e1f-kube-api-access-92b4b\") pod \"network-check-source-7f44b6dd87-xvxlc\" (UID: \"88201346-80c9-4e71-8c22-d42722339e1f\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-xvxlc" May 21 00:56:26.252392 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:26.252369 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92b4b\" (UniqueName: \"kubernetes.io/projected/88201346-80c9-4e71-8c22-d42722339e1f-kube-api-access-92b4b\") pod \"network-check-source-7f44b6dd87-xvxlc\" (UID: \"88201346-80c9-4e71-8c22-d42722339e1f\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-xvxlc" May 21 00:56:26.359793 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:26.359757 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-xvxlc" May 21 00:56:26.476438 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:26.476410 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-xvxlc"] May 21 00:56:26.479432 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:56:26.479394 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88201346_80c9_4e71_8c22_d42722339e1f.slice/crio-2f41b70e0cd63d3faa7d797f82c95ad62cd2cca31fc9086b3caff11541c54026 WatchSource:0}: Error finding container 2f41b70e0cd63d3faa7d797f82c95ad62cd2cca31fc9086b3caff11541c54026: Status 404 returned error can't find the container with id 2f41b70e0cd63d3faa7d797f82c95ad62cd2cca31fc9086b3caff11541c54026 May 21 00:56:26.956322 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:26.956271 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-d8ttl" event={"ID":"02d56ea6-c685-4f09-b951-40d453ba0a31","Type":"ContainerStarted","Data":"d9aa148ae6a95d05259aca9d107d0e6b494c2891a0cacbd2cc56c1d156810ca8"} May 21 00:56:26.956322 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:26.956324 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-d8ttl" event={"ID":"02d56ea6-c685-4f09-b951-40d453ba0a31","Type":"ContainerStarted","Data":"9e92e280e3ddc2e12309d9bad85017325d123759207b3748136d9b7d33a2eba8"} May 21 00:56:26.957510 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:26.957486 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-xvxlc" event={"ID":"88201346-80c9-4e71-8c22-d42722339e1f","Type":"ContainerStarted","Data":"52c09e01dc2dceba5bec699f1923728d205fbe02b34422cef7060d4ff3d7af5f"} May 21 00:56:26.957623 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:26.957515 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-xvxlc" event={"ID":"88201346-80c9-4e71-8c22-d42722339e1f","Type":"ContainerStarted","Data":"2f41b70e0cd63d3faa7d797f82c95ad62cd2cca31fc9086b3caff11541c54026"} May 21 00:56:26.971113 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:26.971077 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-d8ttl" podStartSLOduration=1.5271511150000001 podStartE2EDuration="2.971047907s" podCreationTimestamp="2026-05-21 00:56:24 +0000 UTC" firstStartedPulling="2026-05-21 00:56:24.753709868 +0000 UTC m=+103.650655157" lastFinishedPulling="2026-05-21 00:56:26.197606648 +0000 UTC m=+105.094551949" observedRunningTime="2026-05-21 00:56:26.969768355 +0000 UTC m=+105.866713667" watchObservedRunningTime="2026-05-21 00:56:26.971047907 +0000 UTC m=+105.867993217" May 21 00:56:26.985633 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:26.985588 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-xvxlc" podStartSLOduration=0.985573876 podStartE2EDuration="985.573876ms" podCreationTimestamp="2026-05-21 00:56:26 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 00:56:26.985321813 +0000 UTC m=+105.882267126" watchObservedRunningTime="2026-05-21 00:56:26.985573876 +0000 UTC m=+105.882519188" May 21 00:56:27.079350 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.079324 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-dpnq5"] May 21 00:56:27.081650 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.081632 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dpnq5" May 21 00:56:27.083831 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.083809 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" May 21 00:56:27.084071 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.084046 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" May 21 00:56:27.084418 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.084399 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cj4wn\"" May 21 00:56:27.084502 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.084478 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" May 21 00:56:27.084562 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.084503 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" May 21 00:56:27.092159 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.092142 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dpnq5"] May 21 00:56:27.111234 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.111203 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-pbjgz"] May 21 00:56:27.113269 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.113254 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-67d667c7d-pbjgz" May 21 00:56:27.115721 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.115706 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" May 21 00:56:27.115793 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.115737 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-t26pk\"" May 21 00:56:27.115858 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.115832 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" May 21 00:56:27.115919 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.115886 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" May 21 00:56:27.115970 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.115936 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" May 21 00:56:27.119551 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.119535 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-pbjgz"] May 21 00:56:27.151675 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.151655 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/151cfde9-ff22-4152-9278-6971c624a383-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dpnq5\" (UID: \"151cfde9-ff22-4152-9278-6971c624a383\") " pod="openshift-insights/insights-runtime-extractor-dpnq5" May 21 00:56:27.151792 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.151690 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/151cfde9-ff22-4152-9278-6971c624a383-data-volume\") pod \"insights-runtime-extractor-dpnq5\" (UID: \"151cfde9-ff22-4152-9278-6971c624a383\") " pod="openshift-insights/insights-runtime-extractor-dpnq5" May 21 00:56:27.151792 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.151709 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/151cfde9-ff22-4152-9278-6971c624a383-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dpnq5\" (UID: \"151cfde9-ff22-4152-9278-6971c624a383\") " pod="openshift-insights/insights-runtime-extractor-dpnq5" May 21 00:56:27.151792 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.151781 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/151cfde9-ff22-4152-9278-6971c624a383-crio-socket\") pod \"insights-runtime-extractor-dpnq5\" (UID: \"151cfde9-ff22-4152-9278-6971c624a383\") " pod="openshift-insights/insights-runtime-extractor-dpnq5" May 21 00:56:27.151945 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.151806 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk9bc\" (UniqueName: \"kubernetes.io/projected/151cfde9-ff22-4152-9278-6971c624a383-kube-api-access-jk9bc\") pod \"insights-runtime-extractor-dpnq5\" (UID: \"151cfde9-ff22-4152-9278-6971c624a383\") " pod="openshift-insights/insights-runtime-extractor-dpnq5" May 21 00:56:27.252637 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.252611 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/151cfde9-ff22-4152-9278-6971c624a383-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dpnq5\" (UID: \"151cfde9-ff22-4152-9278-6971c624a383\") " pod="openshift-insights/insights-runtime-extractor-dpnq5" May 21 00:56:27.252746 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.252643 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/151cfde9-ff22-4152-9278-6971c624a383-crio-socket\") pod \"insights-runtime-extractor-dpnq5\" (UID: \"151cfde9-ff22-4152-9278-6971c624a383\") " pod="openshift-insights/insights-runtime-extractor-dpnq5" May 21 00:56:27.252746 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.252661 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jk9bc\" (UniqueName: \"kubernetes.io/projected/151cfde9-ff22-4152-9278-6971c624a383-kube-api-access-jk9bc\") pod \"insights-runtime-extractor-dpnq5\" (UID: \"151cfde9-ff22-4152-9278-6971c624a383\") " pod="openshift-insights/insights-runtime-extractor-dpnq5" May 21 00:56:27.252746 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.252739 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/151cfde9-ff22-4152-9278-6971c624a383-crio-socket\") pod \"insights-runtime-extractor-dpnq5\" (UID: \"151cfde9-ff22-4152-9278-6971c624a383\") " pod="openshift-insights/insights-runtime-extractor-dpnq5" May 21 00:56:27.252895 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:27.252759 2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found May 21 00:56:27.252895 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:27.252826 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/151cfde9-ff22-4152-9278-6971c624a383-insights-runtime-extractor-tls podName:151cfde9-ff22-4152-9278-6971c624a383 nodeName:}" failed. No retries permitted until 2026-05-21 00:56:27.752808421 +0000 UTC m=+106.649753722 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/151cfde9-ff22-4152-9278-6971c624a383-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dpnq5" (UID: "151cfde9-ff22-4152-9278-6971c624a383") : secret "insights-runtime-extractor-tls" not found May 21 00:56:27.252895 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.252848 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/99084a94-3345-4362-86d2-f9936139c30f-signing-key\") pod \"service-ca-67d667c7d-pbjgz\" (UID: \"99084a94-3345-4362-86d2-f9936139c30f\") " pod="openshift-service-ca/service-ca-67d667c7d-pbjgz" May 21 00:56:27.252895 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.252879 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/151cfde9-ff22-4152-9278-6971c624a383-data-volume\") pod \"insights-runtime-extractor-dpnq5\" (UID: \"151cfde9-ff22-4152-9278-6971c624a383\") " pod="openshift-insights/insights-runtime-extractor-dpnq5" May 21 00:56:27.253127 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.252966 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdqb2\" (UniqueName: \"kubernetes.io/projected/99084a94-3345-4362-86d2-f9936139c30f-kube-api-access-pdqb2\") pod \"service-ca-67d667c7d-pbjgz\" (UID: \"99084a94-3345-4362-86d2-f9936139c30f\") " pod="openshift-service-ca/service-ca-67d667c7d-pbjgz" May 21 00:56:27.253127 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.253020 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/99084a94-3345-4362-86d2-f9936139c30f-signing-cabundle\") pod \"service-ca-67d667c7d-pbjgz\" (UID: \"99084a94-3345-4362-86d2-f9936139c30f\") " pod="openshift-service-ca/service-ca-67d667c7d-pbjgz" May 21 00:56:27.253127 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.253042 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/151cfde9-ff22-4152-9278-6971c624a383-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dpnq5\" (UID: \"151cfde9-ff22-4152-9278-6971c624a383\") " pod="openshift-insights/insights-runtime-extractor-dpnq5" May 21 00:56:27.253276 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.253173 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/151cfde9-ff22-4152-9278-6971c624a383-data-volume\") pod \"insights-runtime-extractor-dpnq5\" (UID: \"151cfde9-ff22-4152-9278-6971c624a383\") " pod="openshift-insights/insights-runtime-extractor-dpnq5" May 21 00:56:27.253450 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.253435 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/151cfde9-ff22-4152-9278-6971c624a383-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dpnq5\" (UID: \"151cfde9-ff22-4152-9278-6971c624a383\") " pod="openshift-insights/insights-runtime-extractor-dpnq5" May 21 00:56:27.259838 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.259821 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk9bc\" (UniqueName: \"kubernetes.io/projected/151cfde9-ff22-4152-9278-6971c624a383-kube-api-access-jk9bc\") pod \"insights-runtime-extractor-dpnq5\" (UID: \"151cfde9-ff22-4152-9278-6971c624a383\") " pod="openshift-insights/insights-runtime-extractor-dpnq5" May 21 00:56:27.353994 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.353974 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/99084a94-3345-4362-86d2-f9936139c30f-signing-key\") pod \"service-ca-67d667c7d-pbjgz\" (UID: \"99084a94-3345-4362-86d2-f9936139c30f\") " pod="openshift-service-ca/service-ca-67d667c7d-pbjgz" May 21 00:56:27.354127 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.354003 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pdqb2\" (UniqueName: \"kubernetes.io/projected/99084a94-3345-4362-86d2-f9936139c30f-kube-api-access-pdqb2\") pod \"service-ca-67d667c7d-pbjgz\" (UID: \"99084a94-3345-4362-86d2-f9936139c30f\") " pod="openshift-service-ca/service-ca-67d667c7d-pbjgz" May 21 00:56:27.354127 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.354046 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/99084a94-3345-4362-86d2-f9936139c30f-signing-cabundle\") pod \"service-ca-67d667c7d-pbjgz\" (UID: \"99084a94-3345-4362-86d2-f9936139c30f\") " pod="openshift-service-ca/service-ca-67d667c7d-pbjgz" May 21 00:56:27.354773 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.354756 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/99084a94-3345-4362-86d2-f9936139c30f-signing-cabundle\") pod \"service-ca-67d667c7d-pbjgz\" (UID: \"99084a94-3345-4362-86d2-f9936139c30f\") " pod="openshift-service-ca/service-ca-67d667c7d-pbjgz" May 21 00:56:27.356319 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.356299 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/99084a94-3345-4362-86d2-f9936139c30f-signing-key\") pod \"service-ca-67d667c7d-pbjgz\" (UID: \"99084a94-3345-4362-86d2-f9936139c30f\") " pod="openshift-service-ca/service-ca-67d667c7d-pbjgz" May 21 00:56:27.360976 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.360958 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdqb2\" (UniqueName: \"kubernetes.io/projected/99084a94-3345-4362-86d2-f9936139c30f-kube-api-access-pdqb2\") pod \"service-ca-67d667c7d-pbjgz\" (UID: \"99084a94-3345-4362-86d2-f9936139c30f\") " pod="openshift-service-ca/service-ca-67d667c7d-pbjgz" May 21 00:56:27.422135 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.422115 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-67d667c7d-pbjgz" May 21 00:56:27.540762 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.540685 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-pbjgz"] May 21 00:56:27.544474 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:56:27.544446 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99084a94_3345_4362_86d2_f9936139c30f.slice/crio-e39d33c8925344aff2d6e644e8e1bf1d2a49a9a8f11e74ac7cb837f469b1f568 WatchSource:0}: Error finding container e39d33c8925344aff2d6e644e8e1bf1d2a49a9a8f11e74ac7cb837f469b1f568: Status 404 returned error can't find the container with id e39d33c8925344aff2d6e644e8e1bf1d2a49a9a8f11e74ac7cb837f469b1f568 May 21 00:56:27.758447 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.758408 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/151cfde9-ff22-4152-9278-6971c624a383-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dpnq5\" (UID: \"151cfde9-ff22-4152-9278-6971c624a383\") " pod="openshift-insights/insights-runtime-extractor-dpnq5" May 21 00:56:27.758620 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:27.758570 2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found May 21 00:56:27.758677 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:27.758648 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/151cfde9-ff22-4152-9278-6971c624a383-insights-runtime-extractor-tls podName:151cfde9-ff22-4152-9278-6971c624a383 nodeName:}" failed. No retries permitted until 2026-05-21 00:56:28.758627439 +0000 UTC m=+107.655572747 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/151cfde9-ff22-4152-9278-6971c624a383-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dpnq5" (UID: "151cfde9-ff22-4152-9278-6971c624a383") : secret "insights-runtime-extractor-tls" not found May 21 00:56:27.961714 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:27.961384 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-67d667c7d-pbjgz" event={"ID":"99084a94-3345-4362-86d2-f9936139c30f","Type":"ContainerStarted","Data":"e39d33c8925344aff2d6e644e8e1bf1d2a49a9a8f11e74ac7cb837f469b1f568"} May 21 00:56:28.161708 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:28.161675 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-registry-tls\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:28.161887 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:28.161833 2549 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 21 00:56:28.161887 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:28.161849 2549 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9d999d7b8-pw9kx: secret "image-registry-tls" not found May 21 00:56:28.161971 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:28.161904 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-registry-tls podName:68d5c953-f922-4671-b7ba-5436cb108e4e nodeName:}" failed. No retries permitted until 2026-05-21 00:56:32.16188695 +0000 UTC m=+111.058832245 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-registry-tls") pod "image-registry-9d999d7b8-pw9kx" (UID: "68d5c953-f922-4671-b7ba-5436cb108e4e") : secret "image-registry-tls" not found May 21 00:56:28.767695 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:28.767651 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/151cfde9-ff22-4152-9278-6971c624a383-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dpnq5\" (UID: \"151cfde9-ff22-4152-9278-6971c624a383\") " pod="openshift-insights/insights-runtime-extractor-dpnq5" May 21 00:56:28.768162 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:28.767836 2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found May 21 00:56:28.768162 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:28.767914 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/151cfde9-ff22-4152-9278-6971c624a383-insights-runtime-extractor-tls podName:151cfde9-ff22-4152-9278-6971c624a383 nodeName:}" failed. No retries permitted until 2026-05-21 00:56:30.767893176 +0000 UTC m=+109.664838468 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/151cfde9-ff22-4152-9278-6971c624a383-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dpnq5" (UID: "151cfde9-ff22-4152-9278-6971c624a383") : secret "insights-runtime-extractor-tls" not found May 21 00:56:29.967553 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:29.967512 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-67d667c7d-pbjgz" event={"ID":"99084a94-3345-4362-86d2-f9936139c30f","Type":"ContainerStarted","Data":"6760af4fca92ca9fe0fc2082fe7041dc3d640bf45c90c39e5b9f7dbe6662ad64"} May 21 00:56:29.982590 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:29.982545 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-67d667c7d-pbjgz" podStartSLOduration=1.338292954 podStartE2EDuration="2.982530765s" podCreationTimestamp="2026-05-21 00:56:27 +0000 UTC" firstStartedPulling="2026-05-21 00:56:27.546836816 +0000 UTC m=+106.443782109" lastFinishedPulling="2026-05-21 00:56:29.191074617 +0000 UTC m=+108.088019920" observedRunningTime="2026-05-21 00:56:29.980897014 +0000 UTC m=+108.877842325" watchObservedRunningTime="2026-05-21 00:56:29.982530765 +0000 UTC m=+108.879476099" May 21 00:56:30.787834 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:30.787796 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/151cfde9-ff22-4152-9278-6971c624a383-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dpnq5\" (UID: \"151cfde9-ff22-4152-9278-6971c624a383\") " pod="openshift-insights/insights-runtime-extractor-dpnq5" May 21 00:56:30.788020 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:30.787952 2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found May 21 00:56:30.788020 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:30.788015 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/151cfde9-ff22-4152-9278-6971c624a383-insights-runtime-extractor-tls podName:151cfde9-ff22-4152-9278-6971c624a383 nodeName:}" failed. No retries permitted until 2026-05-21 00:56:34.787997819 +0000 UTC m=+113.684943116 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/151cfde9-ff22-4152-9278-6971c624a383-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dpnq5" (UID: "151cfde9-ff22-4152-9278-6971c624a383") : secret "insights-runtime-extractor-tls" not found May 21 00:56:32.200611 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:32.200570 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-registry-tls\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:32.200998 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:32.200686 2549 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 21 00:56:32.200998 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:32.200697 2549 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9d999d7b8-pw9kx: secret "image-registry-tls" not found May 21 00:56:32.200998 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:32.200747 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-registry-tls podName:68d5c953-f922-4671-b7ba-5436cb108e4e nodeName:}" failed. No retries permitted until 2026-05-21 00:56:40.200732206 +0000 UTC m=+119.097677498 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-registry-tls") pod "image-registry-9d999d7b8-pw9kx" (UID: "68d5c953-f922-4671-b7ba-5436cb108e4e") : secret "image-registry-tls" not found May 21 00:56:34.821993 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:34.821950 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/151cfde9-ff22-4152-9278-6971c624a383-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dpnq5\" (UID: \"151cfde9-ff22-4152-9278-6971c624a383\") " pod="openshift-insights/insights-runtime-extractor-dpnq5" May 21 00:56:34.822390 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:34.822118 2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found May 21 00:56:34.822390 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:34.822189 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/151cfde9-ff22-4152-9278-6971c624a383-insights-runtime-extractor-tls podName:151cfde9-ff22-4152-9278-6971c624a383 nodeName:}" failed. No retries permitted until 2026-05-21 00:56:42.822172934 +0000 UTC m=+121.719118222 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/151cfde9-ff22-4152-9278-6971c624a383-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dpnq5" (UID: "151cfde9-ff22-4152-9278-6971c624a383") : secret "insights-runtime-extractor-tls" not found May 21 00:56:40.266406 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:40.266368 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-registry-tls\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:40.268758 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:40.268731 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-registry-tls\") pod \"image-registry-9d999d7b8-pw9kx\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:40.334501 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:40.334473 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:40.454737 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:40.454700 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9d999d7b8-pw9kx"] May 21 00:56:40.457610 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:56:40.457573 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68d5c953_f922_4671_b7ba_5436cb108e4e.slice/crio-e6b75e7dc8ade0736ae0b96fd3023eb125e4383668efd9c983c58fc3b5c02dd9 WatchSource:0}: Error finding container e6b75e7dc8ade0736ae0b96fd3023eb125e4383668efd9c983c58fc3b5c02dd9: Status 404 returned error can't find the container with id e6b75e7dc8ade0736ae0b96fd3023eb125e4383668efd9c983c58fc3b5c02dd9 May 21 00:56:40.996312 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:40.996275 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" event={"ID":"68d5c953-f922-4671-b7ba-5436cb108e4e","Type":"ContainerStarted","Data":"06272803ed87a1b8cb2f7c34ffe9b12f90bcd79a4c46a2f3c3f1df197014d89c"} May 21 00:56:40.996312 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:40.996313 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" event={"ID":"68d5c953-f922-4671-b7ba-5436cb108e4e","Type":"ContainerStarted","Data":"e6b75e7dc8ade0736ae0b96fd3023eb125e4383668efd9c983c58fc3b5c02dd9"} May 21 00:56:40.996523 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:40.996411 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:56:41.012807 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:41.012765 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" podStartSLOduration=17.012752359 podStartE2EDuration="17.012752359s" podCreationTimestamp="2026-05-21 00:56:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 00:56:41.012216193 +0000 UTC m=+119.909161505" watchObservedRunningTime="2026-05-21 00:56:41.012752359 +0000 UTC m=+119.909697669" May 21 00:56:41.377386 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:41.377305 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6891f821-6e83-4a11-80c9-34bf3144c9ed-service-ca-bundle\") pod \"router-default-685d6d5876-58bgk\" (UID: \"6891f821-6e83-4a11-80c9-34bf3144c9ed\") " pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:41.377386 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:41.377351 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6891f821-6e83-4a11-80c9-34bf3144c9ed-metrics-certs\") pod \"router-default-685d6d5876-58bgk\" (UID: \"6891f821-6e83-4a11-80c9-34bf3144c9ed\") " pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:41.377875 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:41.377855 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6891f821-6e83-4a11-80c9-34bf3144c9ed-service-ca-bundle\") pod \"router-default-685d6d5876-58bgk\" (UID: \"6891f821-6e83-4a11-80c9-34bf3144c9ed\") " pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:41.379780 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:41.379754 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6891f821-6e83-4a11-80c9-34bf3144c9ed-metrics-certs\") pod \"router-default-685d6d5876-58bgk\" (UID: \"6891f821-6e83-4a11-80c9-34bf3144c9ed\") " pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:41.632054 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:41.631987 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-rp6fr\"" May 21 00:56:41.639893 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:41.639873 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:41.759492 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:41.759467 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-685d6d5876-58bgk"] May 21 00:56:41.762211 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:56:41.762185 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6891f821_6e83_4a11_80c9_34bf3144c9ed.slice/crio-b60df84dd7eed5dc9ab2517a9afcf594327bff6a56229290e7f8cf88ffc7cb2e WatchSource:0}: Error finding container b60df84dd7eed5dc9ab2517a9afcf594327bff6a56229290e7f8cf88ffc7cb2e: Status 404 returned error can't find the container with id b60df84dd7eed5dc9ab2517a9afcf594327bff6a56229290e7f8cf88ffc7cb2e May 21 00:56:41.999487 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:41.999454 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-685d6d5876-58bgk" event={"ID":"6891f821-6e83-4a11-80c9-34bf3144c9ed","Type":"ContainerStarted","Data":"f5c4e9046a4e283ba758b115a0bcaaea683c5528bbda74969542c9536255415b"} May 21 00:56:41.999487 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:41.999489 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-685d6d5876-58bgk" event={"ID":"6891f821-6e83-4a11-80c9-34bf3144c9ed","Type":"ContainerStarted","Data":"b60df84dd7eed5dc9ab2517a9afcf594327bff6a56229290e7f8cf88ffc7cb2e"} May 21 00:56:42.017635 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:42.017594 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/router-default-685d6d5876-58bgk" podStartSLOduration=33.017581289 podStartE2EDuration="33.017581289s" podCreationTimestamp="2026-05-21 00:56:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 00:56:42.015883361 +0000 UTC m=+120.912828672" watchObservedRunningTime="2026-05-21 00:56:42.017581289 +0000 UTC m=+120.914526599" May 21 00:56:42.640674 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:42.640642 2549 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:42.643076 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:42.643038 2549 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:42.888276 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:42.888240 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/151cfde9-ff22-4152-9278-6971c624a383-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dpnq5\" (UID: \"151cfde9-ff22-4152-9278-6971c624a383\") " pod="openshift-insights/insights-runtime-extractor-dpnq5" May 21 00:56:42.891582 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:42.891519 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/151cfde9-ff22-4152-9278-6971c624a383-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dpnq5\" (UID: \"151cfde9-ff22-4152-9278-6971c624a383\") " pod="openshift-insights/insights-runtime-extractor-dpnq5" May 21 00:56:42.994380 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:42.994353 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cj4wn\"" May 21 00:56:43.002272 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:43.002248 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:43.002378 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:43.002310 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dpnq5" May 21 00:56:43.003651 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:43.003630 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-685d6d5876-58bgk" May 21 00:56:43.126182 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:43.126155 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dpnq5"] May 21 00:56:43.129047 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:56:43.129023 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod151cfde9_ff22_4152_9278_6971c624a383.slice/crio-29a57ecc367b581158bf56d7464f20e2b06256d50349154625da1908613c03c9 WatchSource:0}: Error finding container 29a57ecc367b581158bf56d7464f20e2b06256d50349154625da1908613c03c9: Status 404 returned error can't find the container with id 29a57ecc367b581158bf56d7464f20e2b06256d50349154625da1908613c03c9 May 21 00:56:44.006179 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:44.006146 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dpnq5" event={"ID":"151cfde9-ff22-4152-9278-6971c624a383","Type":"ContainerStarted","Data":"bdcd5893a961c87599134e2b61e82e2022a5d1249435a4aa2db530ba6d95faf6"} May 21 00:56:44.006179 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:44.006180 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dpnq5" event={"ID":"151cfde9-ff22-4152-9278-6971c624a383","Type":"ContainerStarted","Data":"bb478ae2d4ac4ec965ef5c46f0fc3c689edc5699ab0ecdf73ba81369d5bc6504"} May 21 00:56:44.006581 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:44.006190 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dpnq5" event={"ID":"151cfde9-ff22-4152-9278-6971c624a383","Type":"ContainerStarted","Data":"29a57ecc367b581158bf56d7464f20e2b06256d50349154625da1908613c03c9"} May 21 00:56:46.012538 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:46.012506 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dpnq5" event={"ID":"151cfde9-ff22-4152-9278-6971c624a383","Type":"ContainerStarted","Data":"23cac9b611003ebad33d225e88ef4e4df295839cb207db7dcb8395461787bef0"} May 21 00:56:46.028720 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:46.028675 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-dpnq5" podStartSLOduration=16.69381948 podStartE2EDuration="19.028662515s" podCreationTimestamp="2026-05-21 00:56:27 +0000 UTC" firstStartedPulling="2026-05-21 00:56:43.177607612 +0000 UTC m=+122.074552901" lastFinishedPulling="2026-05-21 00:56:45.512450644 +0000 UTC m=+124.409395936" observedRunningTime="2026-05-21 00:56:46.027874991 +0000 UTC m=+124.924820301" watchObservedRunningTime="2026-05-21 00:56:46.028662515 +0000 UTC m=+124.925607826" May 21 00:56:48.155580 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:48.155542 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-cqdsk"] May 21 00:56:48.158433 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:48.158415 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-cqdsk" May 21 00:56:48.161516 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:48.161491 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" May 21 00:56:48.161516 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:48.161506 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" May 21 00:56:48.161715 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:48.161513 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-5lg2k\"" May 21 00:56:48.165685 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:48.165333 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-cqdsk"] May 21 00:56:48.192202 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:48.192176 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-9d999d7b8-pw9kx"] May 21 00:56:48.325119 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:48.325083 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/86999352-f51c-47ff-9bc5-a62d9939a6ba-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-cqdsk\" (UID: \"86999352-f51c-47ff-9bc5-a62d9939a6ba\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-cqdsk" May 21 00:56:48.325262 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:48.325148 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/86999352-f51c-47ff-9bc5-a62d9939a6ba-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-cqdsk\" (UID: \"86999352-f51c-47ff-9bc5-a62d9939a6ba\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-cqdsk" May 21 00:56:48.425893 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:48.425835 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/86999352-f51c-47ff-9bc5-a62d9939a6ba-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-cqdsk\" (UID: \"86999352-f51c-47ff-9bc5-a62d9939a6ba\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-cqdsk" May 21 00:56:48.425893 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:48.425881 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/86999352-f51c-47ff-9bc5-a62d9939a6ba-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-cqdsk\" (UID: \"86999352-f51c-47ff-9bc5-a62d9939a6ba\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-cqdsk" May 21 00:56:48.426509 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:48.426490 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/86999352-f51c-47ff-9bc5-a62d9939a6ba-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-cqdsk\" (UID: \"86999352-f51c-47ff-9bc5-a62d9939a6ba\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-cqdsk" May 21 00:56:48.428263 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:48.428245 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/86999352-f51c-47ff-9bc5-a62d9939a6ba-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-cqdsk\" (UID: \"86999352-f51c-47ff-9bc5-a62d9939a6ba\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-cqdsk" May 21 00:56:48.468643 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:48.468622 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-cqdsk" May 21 00:56:48.582291 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:48.582264 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-cqdsk"] May 21 00:56:48.585435 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:56:48.585404 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86999352_f51c_47ff_9bc5_a62d9939a6ba.slice/crio-4f431bf3920f9f34939c5c6d902df2998bfe6e7b202aeae85341f1adc89be0f2 WatchSource:0}: Error finding container 4f431bf3920f9f34939c5c6d902df2998bfe6e7b202aeae85341f1adc89be0f2: Status 404 returned error can't find the container with id 4f431bf3920f9f34939c5c6d902df2998bfe6e7b202aeae85341f1adc89be0f2 May 21 00:56:49.020167 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:49.020131 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-cqdsk" event={"ID":"86999352-f51c-47ff-9bc5-a62d9939a6ba","Type":"ContainerStarted","Data":"4f431bf3920f9f34939c5c6d902df2998bfe6e7b202aeae85341f1adc89be0f2"} May 21 00:56:50.024035 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:50.024005 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-cqdsk" event={"ID":"86999352-f51c-47ff-9bc5-a62d9939a6ba","Type":"ContainerStarted","Data":"4cf16a1e5b3ee115372a01248e75d1e8ad344b886c1d3206bff87a1dbd2d3181"} May 21 00:56:50.040674 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:50.040632 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-cqdsk" podStartSLOduration=0.997721608 podStartE2EDuration="2.040620624s" podCreationTimestamp="2026-05-21 00:56:48 +0000 UTC" firstStartedPulling="2026-05-21 00:56:48.587548795 +0000 UTC m=+127.484494084" lastFinishedPulling="2026-05-21 00:56:49.630447811 +0000 UTC m=+128.527393100" observedRunningTime="2026-05-21 00:56:50.040127433 +0000 UTC m=+128.937072745" watchObservedRunningTime="2026-05-21 00:56:50.040620624 +0000 UTC m=+128.937565935" May 21 00:56:51.447644 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:51.447603 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-metrics-certs\") pod \"network-metrics-daemon-skbch\" (UID: \"5ee0c301-12eb-4c68-9990-6ceafb2c90fa\") " pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:56:51.449924 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:51.449905 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ee0c301-12eb-4c68-9990-6ceafb2c90fa-metrics-certs\") pod \"network-metrics-daemon-skbch\" (UID: \"5ee0c301-12eb-4c68-9990-6ceafb2c90fa\") " pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:56:51.691148 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:51.691121 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2nm7s\"" May 21 00:56:51.699031 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:51.698988 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-skbch" May 21 00:56:51.828022 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:51.827989 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-skbch"] May 21 00:56:51.830601 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:56:51.830574 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ee0c301_12eb_4c68_9990_6ceafb2c90fa.slice/crio-b45dc16b7758757262245cfbe535f22f6252cd3492de28c8a501e7818120a3d0 WatchSource:0}: Error finding container b45dc16b7758757262245cfbe535f22f6252cd3492de28c8a501e7818120a3d0: Status 404 returned error can't find the container with id b45dc16b7758757262245cfbe535f22f6252cd3492de28c8a501e7818120a3d0 May 21 00:56:52.030568 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:52.030462 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-skbch" event={"ID":"5ee0c301-12eb-4c68-9990-6ceafb2c90fa","Type":"ContainerStarted","Data":"b45dc16b7758757262245cfbe535f22f6252cd3492de28c8a501e7818120a3d0"} May 21 00:56:54.036656 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.036623 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-skbch" event={"ID":"5ee0c301-12eb-4c68-9990-6ceafb2c90fa","Type":"ContainerStarted","Data":"2d362fbe7b992689626e6fc7a98e3dccd6cf33e554608cb89701e4aa0f6d31bf"} May 21 00:56:54.036656 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.036660 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-skbch" event={"ID":"5ee0c301-12eb-4c68-9990-6ceafb2c90fa","Type":"ContainerStarted","Data":"dbb360dda1098e9eaa8d32f37a40712bfe21d57dabc3fe8f6577ee84d92ac379"} May 21 00:56:54.050807 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.050773 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-skbch" podStartSLOduration=131.720605665 podStartE2EDuration="2m13.050759467s" podCreationTimestamp="2026-05-21 00:54:41 +0000 UTC" firstStartedPulling="2026-05-21 00:56:51.83256267 +0000 UTC m=+130.729507962" lastFinishedPulling="2026-05-21 00:56:53.162716468 +0000 UTC m=+132.059661764" observedRunningTime="2026-05-21 00:56:54.049912557 +0000 UTC m=+132.946857867" watchObservedRunningTime="2026-05-21 00:56:54.050759467 +0000 UTC m=+132.947704772" May 21 00:56:54.602958 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.602921 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw"] May 21 00:56:54.605604 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.605583 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw" May 21 00:56:54.608863 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.608119 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" May 21 00:56:54.608863 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.608343 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" May 21 00:56:54.608863 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.608553 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lwhqz"] May 21 00:56:54.608863 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.608614 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" May 21 00:56:54.609154 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.609086 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" May 21 00:56:54.609635 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.609310 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" May 21 00:56:54.609635 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.609497 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" May 21 00:56:54.609635 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.609567 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-qg5fd\"" May 21 00:56:54.610865 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.610844 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.613226 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.613206 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" May 21 00:56:54.613912 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.613891 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" May 21 00:56:54.615767 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.615680 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw"] May 21 00:56:54.616347 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.616049 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" May 21 00:56:54.616347 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.616165 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-7stl2\"" May 21 00:56:54.777130 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.777098 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d20f6a20-17df-4d0a-ad91-379c6a7625b9-node-exporter-wtmp\") pod \"node-exporter-lwhqz\" (UID: \"d20f6a20-17df-4d0a-ad91-379c6a7625b9\") " pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.777324 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.777169 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ac23c315-d0c1-42b9-bb31-de85ad519f57-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-54fd6bc5c4-m5nnw\" (UID: \"ac23c315-d0c1-42b9-bb31-de85ad519f57\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw" May 21 00:56:54.777324 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.777199 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ac23c315-d0c1-42b9-bb31-de85ad519f57-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-m5nnw\" (UID: \"ac23c315-d0c1-42b9-bb31-de85ad519f57\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw" May 21 00:56:54.777324 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.777223 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d20f6a20-17df-4d0a-ad91-379c6a7625b9-sys\") pod \"node-exporter-lwhqz\" (UID: \"d20f6a20-17df-4d0a-ad91-379c6a7625b9\") " pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.777324 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.777249 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clbzh\" (UniqueName: \"kubernetes.io/projected/d20f6a20-17df-4d0a-ad91-379c6a7625b9-kube-api-access-clbzh\") pod \"node-exporter-lwhqz\" (UID: \"d20f6a20-17df-4d0a-ad91-379c6a7625b9\") " pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.777324 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.777297 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzgrh\" (UniqueName: \"kubernetes.io/projected/ac23c315-d0c1-42b9-bb31-de85ad519f57-kube-api-access-gzgrh\") pod \"kube-state-metrics-54fd6bc5c4-m5nnw\" (UID: \"ac23c315-d0c1-42b9-bb31-de85ad519f57\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw" May 21 00:56:54.777497 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.777423 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ac23c315-d0c1-42b9-bb31-de85ad519f57-volume-directive-shadow\") pod \"kube-state-metrics-54fd6bc5c4-m5nnw\" (UID: \"ac23c315-d0c1-42b9-bb31-de85ad519f57\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw" May 21 00:56:54.777497 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.777466 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d20f6a20-17df-4d0a-ad91-379c6a7625b9-root\") pod \"node-exporter-lwhqz\" (UID: \"d20f6a20-17df-4d0a-ad91-379c6a7625b9\") " pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.777497 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.777491 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d20f6a20-17df-4d0a-ad91-379c6a7625b9-node-exporter-textfile\") pod \"node-exporter-lwhqz\" (UID: \"d20f6a20-17df-4d0a-ad91-379c6a7625b9\") " pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.777585 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.777519 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d20f6a20-17df-4d0a-ad91-379c6a7625b9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lwhqz\" (UID: \"d20f6a20-17df-4d0a-ad91-379c6a7625b9\") " pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.777585 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.777556 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d20f6a20-17df-4d0a-ad91-379c6a7625b9-metrics-client-ca\") pod \"node-exporter-lwhqz\" (UID: \"d20f6a20-17df-4d0a-ad91-379c6a7625b9\") " pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.777657 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.777580 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac23c315-d0c1-42b9-bb31-de85ad519f57-metrics-client-ca\") pod \"kube-state-metrics-54fd6bc5c4-m5nnw\" (UID: \"ac23c315-d0c1-42b9-bb31-de85ad519f57\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw" May 21 00:56:54.777657 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.777610 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ac23c315-d0c1-42b9-bb31-de85ad519f57-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-54fd6bc5c4-m5nnw\" (UID: \"ac23c315-d0c1-42b9-bb31-de85ad519f57\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw" May 21 00:56:54.777657 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.777639 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d20f6a20-17df-4d0a-ad91-379c6a7625b9-node-exporter-tls\") pod \"node-exporter-lwhqz\" (UID: \"d20f6a20-17df-4d0a-ad91-379c6a7625b9\") " pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.777781 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.777667 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d20f6a20-17df-4d0a-ad91-379c6a7625b9-node-exporter-accelerators-collector-config\") pod \"node-exporter-lwhqz\" (UID: \"d20f6a20-17df-4d0a-ad91-379c6a7625b9\") " pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.878572 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.878482 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-clbzh\" (UniqueName: \"kubernetes.io/projected/d20f6a20-17df-4d0a-ad91-379c6a7625b9-kube-api-access-clbzh\") pod \"node-exporter-lwhqz\" (UID: \"d20f6a20-17df-4d0a-ad91-379c6a7625b9\") " pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.878572 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.878534 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gzgrh\" (UniqueName: \"kubernetes.io/projected/ac23c315-d0c1-42b9-bb31-de85ad519f57-kube-api-access-gzgrh\") pod \"kube-state-metrics-54fd6bc5c4-m5nnw\" (UID: \"ac23c315-d0c1-42b9-bb31-de85ad519f57\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw" May 21 00:56:54.878796 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.878582 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ac23c315-d0c1-42b9-bb31-de85ad519f57-volume-directive-shadow\") pod \"kube-state-metrics-54fd6bc5c4-m5nnw\" (UID: \"ac23c315-d0c1-42b9-bb31-de85ad519f57\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw" May 21 00:56:54.878796 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.878617 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d20f6a20-17df-4d0a-ad91-379c6a7625b9-root\") pod \"node-exporter-lwhqz\" (UID: \"d20f6a20-17df-4d0a-ad91-379c6a7625b9\") " pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.878796 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.878641 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d20f6a20-17df-4d0a-ad91-379c6a7625b9-node-exporter-textfile\") pod \"node-exporter-lwhqz\" (UID: \"d20f6a20-17df-4d0a-ad91-379c6a7625b9\") " pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.878796 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.878710 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d20f6a20-17df-4d0a-ad91-379c6a7625b9-root\") pod \"node-exporter-lwhqz\" (UID: \"d20f6a20-17df-4d0a-ad91-379c6a7625b9\") " pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.878796 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.878753 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d20f6a20-17df-4d0a-ad91-379c6a7625b9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lwhqz\" (UID: \"d20f6a20-17df-4d0a-ad91-379c6a7625b9\") " pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.878991 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.878810 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d20f6a20-17df-4d0a-ad91-379c6a7625b9-metrics-client-ca\") pod \"node-exporter-lwhqz\" (UID: \"d20f6a20-17df-4d0a-ad91-379c6a7625b9\") " pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.879917 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.878843 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac23c315-d0c1-42b9-bb31-de85ad519f57-metrics-client-ca\") pod \"kube-state-metrics-54fd6bc5c4-m5nnw\" (UID: \"ac23c315-d0c1-42b9-bb31-de85ad519f57\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw" May 21 00:56:54.879917 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.879414 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d20f6a20-17df-4d0a-ad91-379c6a7625b9-node-exporter-textfile\") pod \"node-exporter-lwhqz\" (UID: \"d20f6a20-17df-4d0a-ad91-379c6a7625b9\") " pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.879917 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.879254 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ac23c315-d0c1-42b9-bb31-de85ad519f57-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-54fd6bc5c4-m5nnw\" (UID: \"ac23c315-d0c1-42b9-bb31-de85ad519f57\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw" May 21 00:56:54.879917 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.879467 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ac23c315-d0c1-42b9-bb31-de85ad519f57-volume-directive-shadow\") pod \"kube-state-metrics-54fd6bc5c4-m5nnw\" (UID: \"ac23c315-d0c1-42b9-bb31-de85ad519f57\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw" May 21 00:56:54.879917 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.879486 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d20f6a20-17df-4d0a-ad91-379c6a7625b9-node-exporter-tls\") pod \"node-exporter-lwhqz\" (UID: \"d20f6a20-17df-4d0a-ad91-379c6a7625b9\") " pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.879917 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.879545 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d20f6a20-17df-4d0a-ad91-379c6a7625b9-node-exporter-accelerators-collector-config\") pod \"node-exporter-lwhqz\" (UID: \"d20f6a20-17df-4d0a-ad91-379c6a7625b9\") " pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.879917 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.879593 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d20f6a20-17df-4d0a-ad91-379c6a7625b9-node-exporter-wtmp\") pod \"node-exporter-lwhqz\" (UID: \"d20f6a20-17df-4d0a-ad91-379c6a7625b9\") " pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.879917 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.879680 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ac23c315-d0c1-42b9-bb31-de85ad519f57-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-54fd6bc5c4-m5nnw\" (UID: \"ac23c315-d0c1-42b9-bb31-de85ad519f57\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw" May 21 00:56:54.879917 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.879715 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ac23c315-d0c1-42b9-bb31-de85ad519f57-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-m5nnw\" (UID: \"ac23c315-d0c1-42b9-bb31-de85ad519f57\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw" May 21 00:56:54.879917 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.879750 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d20f6a20-17df-4d0a-ad91-379c6a7625b9-sys\") pod \"node-exporter-lwhqz\" (UID: \"d20f6a20-17df-4d0a-ad91-379c6a7625b9\") " pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.879917 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.879823 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d20f6a20-17df-4d0a-ad91-379c6a7625b9-sys\") pod \"node-exporter-lwhqz\" (UID: \"d20f6a20-17df-4d0a-ad91-379c6a7625b9\") " pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.880500 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.880202 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac23c315-d0c1-42b9-bb31-de85ad519f57-metrics-client-ca\") pod \"kube-state-metrics-54fd6bc5c4-m5nnw\" (UID: \"ac23c315-d0c1-42b9-bb31-de85ad519f57\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw" May 21 00:56:54.881756 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:54.880583 2549 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found May 21 00:56:54.881756 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:54.880690 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac23c315-d0c1-42b9-bb31-de85ad519f57-kube-state-metrics-tls podName:ac23c315-d0c1-42b9-bb31-de85ad519f57 nodeName:}" failed. No retries permitted until 2026-05-21 00:56:55.380671249 +0000 UTC m=+134.277616537 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/ac23c315-d0c1-42b9-bb31-de85ad519f57-kube-state-metrics-tls") pod "kube-state-metrics-54fd6bc5c4-m5nnw" (UID: "ac23c315-d0c1-42b9-bb31-de85ad519f57") : secret "kube-state-metrics-tls" not found May 21 00:56:54.881756 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.880824 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d20f6a20-17df-4d0a-ad91-379c6a7625b9-node-exporter-wtmp\") pod \"node-exporter-lwhqz\" (UID: \"d20f6a20-17df-4d0a-ad91-379c6a7625b9\") " pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.881756 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.880879 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d20f6a20-17df-4d0a-ad91-379c6a7625b9-node-exporter-accelerators-collector-config\") pod \"node-exporter-lwhqz\" (UID: \"d20f6a20-17df-4d0a-ad91-379c6a7625b9\") " pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.882010 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.881930 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ac23c315-d0c1-42b9-bb31-de85ad519f57-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-54fd6bc5c4-m5nnw\" (UID: \"ac23c315-d0c1-42b9-bb31-de85ad519f57\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw" May 21 00:56:54.887230 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.882234 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d20f6a20-17df-4d0a-ad91-379c6a7625b9-metrics-client-ca\") pod \"node-exporter-lwhqz\" (UID: \"d20f6a20-17df-4d0a-ad91-379c6a7625b9\") " pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.887230 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.883197 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ac23c315-d0c1-42b9-bb31-de85ad519f57-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-54fd6bc5c4-m5nnw\" (UID: \"ac23c315-d0c1-42b9-bb31-de85ad519f57\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw" May 21 00:56:54.887735 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.887701 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d20f6a20-17df-4d0a-ad91-379c6a7625b9-node-exporter-tls\") pod \"node-exporter-lwhqz\" (UID: \"d20f6a20-17df-4d0a-ad91-379c6a7625b9\") " pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.888352 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.888274 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clbzh\" (UniqueName: \"kubernetes.io/projected/d20f6a20-17df-4d0a-ad91-379c6a7625b9-kube-api-access-clbzh\") pod \"node-exporter-lwhqz\" (UID: \"d20f6a20-17df-4d0a-ad91-379c6a7625b9\") " pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.889234 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.889211 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d20f6a20-17df-4d0a-ad91-379c6a7625b9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lwhqz\" (UID: \"d20f6a20-17df-4d0a-ad91-379c6a7625b9\") " pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.889625 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.889605 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzgrh\" (UniqueName: \"kubernetes.io/projected/ac23c315-d0c1-42b9-bb31-de85ad519f57-kube-api-access-gzgrh\") pod \"kube-state-metrics-54fd6bc5c4-m5nnw\" (UID: \"ac23c315-d0c1-42b9-bb31-de85ad519f57\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw" May 21 00:56:54.928585 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:54.928555 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lwhqz" May 21 00:56:54.937721 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:56:54.937693 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd20f6a20_17df_4d0a_ad91_379c6a7625b9.slice/crio-c33e598298ae032c1ac543750b365a0ff46aa611e60b51dd1233d2faa7e64382 WatchSource:0}: Error finding container c33e598298ae032c1ac543750b365a0ff46aa611e60b51dd1233d2faa7e64382: Status 404 returned error can't find the container with id c33e598298ae032c1ac543750b365a0ff46aa611e60b51dd1233d2faa7e64382 May 21 00:56:55.040799 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.040768 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lwhqz" event={"ID":"d20f6a20-17df-4d0a-ad91-379c6a7625b9","Type":"ContainerStarted","Data":"c33e598298ae032c1ac543750b365a0ff46aa611e60b51dd1233d2faa7e64382"} May 21 00:56:55.384588 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.384552 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ac23c315-d0c1-42b9-bb31-de85ad519f57-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-m5nnw\" (UID: \"ac23c315-d0c1-42b9-bb31-de85ad519f57\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw" May 21 00:56:55.386810 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.386788 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ac23c315-d0c1-42b9-bb31-de85ad519f57-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-m5nnw\" (UID: \"ac23c315-d0c1-42b9-bb31-de85ad519f57\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw" May 21 00:56:55.519587 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.519551 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw" May 21 00:56:55.649517 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.649393 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw"] May 21 00:56:55.652715 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:56:55.652682 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac23c315_d0c1_42b9_bb31_de85ad519f57.slice/crio-2753673117ad8f94ea6e384c4e9b23bb7ed3f77df4d08c2d2f6ac39051500798 WatchSource:0}: Error finding container 2753673117ad8f94ea6e384c4e9b23bb7ed3f77df4d08c2d2f6ac39051500798: Status 404 returned error can't find the container with id 2753673117ad8f94ea6e384c4e9b23bb7ed3f77df4d08c2d2f6ac39051500798 May 21 00:56:55.664350 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.664325 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 21 00:56:55.667853 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.667828 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.670172 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.670152 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" May 21 00:56:55.670275 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.670220 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-jbtml\"" May 21 00:56:55.670908 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.670475 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" May 21 00:56:55.670908 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.670701 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" May 21 00:56:55.671052 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.670991 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" May 21 00:56:55.671052 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.670996 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" May 21 00:56:55.671384 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.671180 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" May 21 00:56:55.671384 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.671246 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" May 21 00:56:55.671384 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.671271 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" May 21 00:56:55.672124 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.672094 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" May 21 00:56:55.682778 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.682756 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 21 00:56:55.686343 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.686317 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/194459e3-e611-4eb3-adfe-61bec9c5f2c9-config-out\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.686486 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.686388 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.686486 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.686450 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.686606 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.686529 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/194459e3-e611-4eb3-adfe-61bec9c5f2c9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.686606 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.686554 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.686685 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.686605 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/194459e3-e611-4eb3-adfe-61bec9c5f2c9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.686685 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.686631 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-config-volume\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.686784 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.686711 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-web-config\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.686784 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.686768 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/194459e3-e611-4eb3-adfe-61bec9c5f2c9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.686896 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.686819 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz72r\" (UniqueName: \"kubernetes.io/projected/194459e3-e611-4eb3-adfe-61bec9c5f2c9-kube-api-access-nz72r\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.686896 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.686854 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/194459e3-e611-4eb3-adfe-61bec9c5f2c9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.686971 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.686910 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.687008 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.686975 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.787712 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.787682 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.787894 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.787744 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.787894 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.787783 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/194459e3-e611-4eb3-adfe-61bec9c5f2c9-config-out\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.787894 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.787825 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.787894 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.787863 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.788134 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.787901 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/194459e3-e611-4eb3-adfe-61bec9c5f2c9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.788134 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.787926 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.788134 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.787955 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/194459e3-e611-4eb3-adfe-61bec9c5f2c9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.788134 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.787982 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-config-volume\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.788134 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.788009 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-web-config\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.788134 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.788044 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/194459e3-e611-4eb3-adfe-61bec9c5f2c9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.788134 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.788090 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nz72r\" (UniqueName: \"kubernetes.io/projected/194459e3-e611-4eb3-adfe-61bec9c5f2c9-kube-api-access-nz72r\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.788134 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.788132 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/194459e3-e611-4eb3-adfe-61bec9c5f2c9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.788517 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:55.788378 2549 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found May 21 00:56:55.788517 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:56:55.788449 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-secret-alertmanager-main-tls podName:194459e3-e611-4eb3-adfe-61bec9c5f2c9 nodeName:}" failed. No retries permitted until 2026-05-21 00:56:56.288428118 +0000 UTC m=+135.185373425 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "194459e3-e611-4eb3-adfe-61bec9c5f2c9") : secret "alertmanager-main-tls" not found May 21 00:56:55.789760 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.789197 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/194459e3-e611-4eb3-adfe-61bec9c5f2c9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.789760 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.789221 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/194459e3-e611-4eb3-adfe-61bec9c5f2c9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.789760 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.789622 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/194459e3-e611-4eb3-adfe-61bec9c5f2c9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.790826 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.790781 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.790926 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.790859 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.790926 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.790895 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/194459e3-e611-4eb3-adfe-61bec9c5f2c9-config-out\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.791822 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.791790 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.792460 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.792425 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/194459e3-e611-4eb3-adfe-61bec9c5f2c9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.792680 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.792657 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-config-volume\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.793161 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.793140 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-web-config\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.793362 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.793347 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:55.797992 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:55.797970 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz72r\" (UniqueName: \"kubernetes.io/projected/194459e3-e611-4eb3-adfe-61bec9c5f2c9-kube-api-access-nz72r\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:56.047611 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:56.047578 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lwhqz" event={"ID":"d20f6a20-17df-4d0a-ad91-379c6a7625b9","Type":"ContainerStarted","Data":"330dec21409924038192f89efb0c2b4a2ed684b1a94f974b136d372f7e18d978"} May 21 00:56:56.048649 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:56.048626 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw" event={"ID":"ac23c315-d0c1-42b9-bb31-de85ad519f57","Type":"ContainerStarted","Data":"2753673117ad8f94ea6e384c4e9b23bb7ed3f77df4d08c2d2f6ac39051500798"} May 21 00:56:56.292709 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:56.292671 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:56.295597 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:56.295561 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:56.579400 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:56.579322 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" May 21 00:56:56.817686 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:56.817661 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 21 00:56:56.822274 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:56:56.822247 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod194459e3_e611_4eb3_adfe_61bec9c5f2c9.slice/crio-36c1add786097a558910b98623273a233f974a37c226e6a851ba29338f8ca799 WatchSource:0}: Error finding container 36c1add786097a558910b98623273a233f974a37c226e6a851ba29338f8ca799: Status 404 returned error can't find the container with id 36c1add786097a558910b98623273a233f974a37c226e6a851ba29338f8ca799 May 21 00:56:57.052324 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.052290 2549 generic.go:358] "Generic (PLEG): container finished" podID="d20f6a20-17df-4d0a-ad91-379c6a7625b9" containerID="330dec21409924038192f89efb0c2b4a2ed684b1a94f974b136d372f7e18d978" exitCode=0 May 21 00:56:57.052739 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.052359 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lwhqz" event={"ID":"d20f6a20-17df-4d0a-ad91-379c6a7625b9","Type":"ContainerDied","Data":"330dec21409924038192f89efb0c2b4a2ed684b1a94f974b136d372f7e18d978"} May 21 00:56:57.054384 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.054343 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw" event={"ID":"ac23c315-d0c1-42b9-bb31-de85ad519f57","Type":"ContainerStarted","Data":"51568290eb3123619c8a6c2ac41be1cd4b4e8d01a714777501e600707fca19ae"} May 21 00:56:57.054384 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.054379 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw" event={"ID":"ac23c315-d0c1-42b9-bb31-de85ad519f57","Type":"ContainerStarted","Data":"387dd5de400a261a365197f4011fec7efbf3270f852f3ef67b43a56fd97a725e"} May 21 00:56:57.054553 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.054393 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw" event={"ID":"ac23c315-d0c1-42b9-bb31-de85ad519f57","Type":"ContainerStarted","Data":"5bc2e7ba3f2c03952f07de14f2d3ec9d4ba2700bbd5b83853fd5cac195c20f89"} May 21 00:56:57.055442 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.055420 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"194459e3-e611-4eb3-adfe-61bec9c5f2c9","Type":"ContainerStarted","Data":"36c1add786097a558910b98623273a233f974a37c226e6a851ba29338f8ca799"} May 21 00:56:57.083882 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.083800 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-m5nnw" podStartSLOduration=2.025418551 podStartE2EDuration="3.083788353s" podCreationTimestamp="2026-05-21 00:56:54 +0000 UTC" firstStartedPulling="2026-05-21 00:56:55.657873918 +0000 UTC m=+134.554819220" lastFinishedPulling="2026-05-21 00:56:56.71624373 +0000 UTC m=+135.613189022" observedRunningTime="2026-05-21 00:56:57.082190857 +0000 UTC m=+135.979136168" watchObservedRunningTime="2026-05-21 00:56:57.083788353 +0000 UTC m=+135.980733663" May 21 00:56:57.570078 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.570028 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g"] May 21 00:56:57.573467 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.573440 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:56:57.575851 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.575826 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" May 21 00:56:57.575851 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.575832 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-1hirqe37t8k2a\"" May 21 00:56:57.576041 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.575926 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" May 21 00:56:57.576312 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.576173 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" May 21 00:56:57.576312 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.576206 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" May 21 00:56:57.576312 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.576236 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-69pqp\"" May 21 00:56:57.576312 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.576239 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" May 21 00:56:57.582826 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.582639 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g"] May 21 00:56:57.604939 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.604913 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2d540d00-7ee2-42d1-b163-8344925e6439-secret-thanos-querier-tls\") pod \"thanos-querier-6ff586c7c5-5ps4g\" (UID: \"2d540d00-7ee2-42d1-b163-8344925e6439\") " pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:56:57.605074 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.604948 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d540d00-7ee2-42d1-b163-8344925e6439-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6ff586c7c5-5ps4g\" (UID: \"2d540d00-7ee2-42d1-b163-8344925e6439\") " pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:56:57.605074 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.604978 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2d540d00-7ee2-42d1-b163-8344925e6439-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6ff586c7c5-5ps4g\" (UID: \"2d540d00-7ee2-42d1-b163-8344925e6439\") " pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:56:57.605074 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.605054 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d540d00-7ee2-42d1-b163-8344925e6439-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6ff586c7c5-5ps4g\" (UID: \"2d540d00-7ee2-42d1-b163-8344925e6439\") " pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:56:57.605212 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.605103 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d540d00-7ee2-42d1-b163-8344925e6439-metrics-client-ca\") pod \"thanos-querier-6ff586c7c5-5ps4g\" (UID: \"2d540d00-7ee2-42d1-b163-8344925e6439\") " pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:56:57.605212 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.605119 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2d540d00-7ee2-42d1-b163-8344925e6439-secret-grpc-tls\") pod \"thanos-querier-6ff586c7c5-5ps4g\" (UID: \"2d540d00-7ee2-42d1-b163-8344925e6439\") " pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:56:57.605212 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.605133 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgjbj\" (UniqueName: \"kubernetes.io/projected/2d540d00-7ee2-42d1-b163-8344925e6439-kube-api-access-kgjbj\") pod \"thanos-querier-6ff586c7c5-5ps4g\" (UID: \"2d540d00-7ee2-42d1-b163-8344925e6439\") " pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:56:57.605212 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.605166 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2d540d00-7ee2-42d1-b163-8344925e6439-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6ff586c7c5-5ps4g\" (UID: \"2d540d00-7ee2-42d1-b163-8344925e6439\") " pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:56:57.706352 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.706326 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2d540d00-7ee2-42d1-b163-8344925e6439-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6ff586c7c5-5ps4g\" (UID: \"2d540d00-7ee2-42d1-b163-8344925e6439\") " pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:56:57.706488 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.706372 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2d540d00-7ee2-42d1-b163-8344925e6439-secret-thanos-querier-tls\") pod \"thanos-querier-6ff586c7c5-5ps4g\" (UID: \"2d540d00-7ee2-42d1-b163-8344925e6439\") " pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:56:57.706488 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.706390 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d540d00-7ee2-42d1-b163-8344925e6439-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6ff586c7c5-5ps4g\" (UID: \"2d540d00-7ee2-42d1-b163-8344925e6439\") " pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:56:57.706567 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.706510 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2d540d00-7ee2-42d1-b163-8344925e6439-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6ff586c7c5-5ps4g\" (UID: \"2d540d00-7ee2-42d1-b163-8344925e6439\") " pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:56:57.706621 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.706598 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d540d00-7ee2-42d1-b163-8344925e6439-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6ff586c7c5-5ps4g\" (UID: \"2d540d00-7ee2-42d1-b163-8344925e6439\") " pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:56:57.706670 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.706636 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d540d00-7ee2-42d1-b163-8344925e6439-metrics-client-ca\") pod \"thanos-querier-6ff586c7c5-5ps4g\" (UID: \"2d540d00-7ee2-42d1-b163-8344925e6439\") " pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:56:57.706670 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.706663 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2d540d00-7ee2-42d1-b163-8344925e6439-secret-grpc-tls\") pod \"thanos-querier-6ff586c7c5-5ps4g\" (UID: \"2d540d00-7ee2-42d1-b163-8344925e6439\") " pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:56:57.706745 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.706688 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kgjbj\" (UniqueName: \"kubernetes.io/projected/2d540d00-7ee2-42d1-b163-8344925e6439-kube-api-access-kgjbj\") pod \"thanos-querier-6ff586c7c5-5ps4g\" (UID: \"2d540d00-7ee2-42d1-b163-8344925e6439\") " pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:56:57.707556 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.707508 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d540d00-7ee2-42d1-b163-8344925e6439-metrics-client-ca\") pod \"thanos-querier-6ff586c7c5-5ps4g\" (UID: \"2d540d00-7ee2-42d1-b163-8344925e6439\") " pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:56:57.709451 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.709426 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2d540d00-7ee2-42d1-b163-8344925e6439-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6ff586c7c5-5ps4g\" (UID: \"2d540d00-7ee2-42d1-b163-8344925e6439\") " pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:56:57.709552 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.709460 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2d540d00-7ee2-42d1-b163-8344925e6439-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6ff586c7c5-5ps4g\" (UID: \"2d540d00-7ee2-42d1-b163-8344925e6439\") " pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:56:57.709552 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.709476 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2d540d00-7ee2-42d1-b163-8344925e6439-secret-thanos-querier-tls\") pod \"thanos-querier-6ff586c7c5-5ps4g\" (UID: \"2d540d00-7ee2-42d1-b163-8344925e6439\") " pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:56:57.709653 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.709632 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d540d00-7ee2-42d1-b163-8344925e6439-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6ff586c7c5-5ps4g\" (UID: \"2d540d00-7ee2-42d1-b163-8344925e6439\") " pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:56:57.709934 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.709910 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2d540d00-7ee2-42d1-b163-8344925e6439-secret-grpc-tls\") pod \"thanos-querier-6ff586c7c5-5ps4g\" (UID: \"2d540d00-7ee2-42d1-b163-8344925e6439\") " pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:56:57.709995 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.709921 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d540d00-7ee2-42d1-b163-8344925e6439-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6ff586c7c5-5ps4g\" (UID: \"2d540d00-7ee2-42d1-b163-8344925e6439\") " pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:56:57.713467 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.713447 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgjbj\" (UniqueName: \"kubernetes.io/projected/2d540d00-7ee2-42d1-b163-8344925e6439-kube-api-access-kgjbj\") pod \"thanos-querier-6ff586c7c5-5ps4g\" (UID: \"2d540d00-7ee2-42d1-b163-8344925e6439\") " pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:56:57.886673 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:57.886590 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:56:58.061668 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:58.061633 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lwhqz" event={"ID":"d20f6a20-17df-4d0a-ad91-379c6a7625b9","Type":"ContainerStarted","Data":"976f413140c0ae493e9c16e6436c95ade75ed72cfdd0a850cf2f3bb89d253725"} May 21 00:56:58.061964 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:58.061672 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lwhqz" event={"ID":"d20f6a20-17df-4d0a-ad91-379c6a7625b9","Type":"ContainerStarted","Data":"f5d55043283505884e97b16d48d70b7e7fb16d21807cd4724ae5af1441bfe42d"} May 21 00:56:58.078502 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:58.078043 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lwhqz" podStartSLOduration=3.082093562 podStartE2EDuration="4.07802467s" podCreationTimestamp="2026-05-21 00:56:54 +0000 UTC" firstStartedPulling="2026-05-21 00:56:54.939468715 +0000 UTC m=+133.836414005" lastFinishedPulling="2026-05-21 00:56:55.935399825 +0000 UTC m=+134.832345113" observedRunningTime="2026-05-21 00:56:58.077392132 +0000 UTC m=+136.974337444" watchObservedRunningTime="2026-05-21 00:56:58.07802467 +0000 UTC m=+136.974969982" May 21 00:56:58.150182 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:58.150046 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g"] May 21 00:56:58.198160 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:58.198110 2549 patch_prober.go:28] interesting pod/image-registry-9d999d7b8-pw9kx container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} May 21 00:56:58.198305 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:58.198178 2549 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" podUID="68d5c953-f922-4671-b7ba-5436cb108e4e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" May 21 00:56:58.207086 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:56:58.207038 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d540d00_7ee2_42d1_b163_8344925e6439.slice/crio-d98d49a052f71300c93fdf9a7c9aead96e4f5c88202e86ad11842dc22a5bfb08 WatchSource:0}: Error finding container d98d49a052f71300c93fdf9a7c9aead96e4f5c88202e86ad11842dc22a5bfb08: Status 404 returned error can't find the container with id d98d49a052f71300c93fdf9a7c9aead96e4f5c88202e86ad11842dc22a5bfb08 May 21 00:56:59.002362 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.002321 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-bc594fcb8-5w7qf"] May 21 00:56:59.004531 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.004512 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" May 21 00:56:59.006715 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.006671 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" May 21 00:56:59.006840 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.006820 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" May 21 00:56:59.007635 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.007614 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" May 21 00:56:59.007745 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.007732 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" May 21 00:56:59.007799 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.007745 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-4pltz\"" May 21 00:56:59.007876 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.007847 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-96br7568om6ej\"" May 21 00:56:59.012671 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.012638 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-bc594fcb8-5w7qf"] May 21 00:56:59.016986 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.016956 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f70c7d64-1725-43ff-8841-59691b5bf34d-metrics-server-audit-profiles\") pod \"metrics-server-bc594fcb8-5w7qf\" (UID: \"f70c7d64-1725-43ff-8841-59691b5bf34d\") " pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" May 21 00:56:59.017158 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.017006 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26mcf\" (UniqueName: \"kubernetes.io/projected/f70c7d64-1725-43ff-8841-59691b5bf34d-kube-api-access-26mcf\") pod \"metrics-server-bc594fcb8-5w7qf\" (UID: \"f70c7d64-1725-43ff-8841-59691b5bf34d\") " pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" May 21 00:56:59.017158 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.017137 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f70c7d64-1725-43ff-8841-59691b5bf34d-secret-metrics-server-client-certs\") pod \"metrics-server-bc594fcb8-5w7qf\" (UID: \"f70c7d64-1725-43ff-8841-59691b5bf34d\") " pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" May 21 00:56:59.017267 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.017200 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f70c7d64-1725-43ff-8841-59691b5bf34d-client-ca-bundle\") pod \"metrics-server-bc594fcb8-5w7qf\" (UID: \"f70c7d64-1725-43ff-8841-59691b5bf34d\") " pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" May 21 00:56:59.017267 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.017248 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f70c7d64-1725-43ff-8841-59691b5bf34d-audit-log\") pod \"metrics-server-bc594fcb8-5w7qf\" (UID: \"f70c7d64-1725-43ff-8841-59691b5bf34d\") " pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" May 21 00:56:59.017376 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.017324 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f70c7d64-1725-43ff-8841-59691b5bf34d-secret-metrics-server-tls\") pod \"metrics-server-bc594fcb8-5w7qf\" (UID: \"f70c7d64-1725-43ff-8841-59691b5bf34d\") " pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" May 21 00:56:59.017376 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.017349 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f70c7d64-1725-43ff-8841-59691b5bf34d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-bc594fcb8-5w7qf\" (UID: \"f70c7d64-1725-43ff-8841-59691b5bf34d\") " pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" May 21 00:56:59.066261 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.066232 2549 generic.go:358] "Generic (PLEG): container finished" podID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerID="bf736ffa394dbd7440ca95cf34a40a6c802890d3afb128e13029bc87e7ee9c10" exitCode=0 May 21 00:56:59.066658 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.066330 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"194459e3-e611-4eb3-adfe-61bec9c5f2c9","Type":"ContainerDied","Data":"bf736ffa394dbd7440ca95cf34a40a6c802890d3afb128e13029bc87e7ee9c10"} May 21 00:56:59.067516 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.067496 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" event={"ID":"2d540d00-7ee2-42d1-b163-8344925e6439","Type":"ContainerStarted","Data":"d98d49a052f71300c93fdf9a7c9aead96e4f5c88202e86ad11842dc22a5bfb08"} May 21 00:56:59.118496 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.118466 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f70c7d64-1725-43ff-8841-59691b5bf34d-secret-metrics-server-tls\") pod \"metrics-server-bc594fcb8-5w7qf\" (UID: \"f70c7d64-1725-43ff-8841-59691b5bf34d\") " pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" May 21 00:56:59.118496 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.118505 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f70c7d64-1725-43ff-8841-59691b5bf34d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-bc594fcb8-5w7qf\" (UID: \"f70c7d64-1725-43ff-8841-59691b5bf34d\") " pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" May 21 00:56:59.118709 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.118657 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f70c7d64-1725-43ff-8841-59691b5bf34d-metrics-server-audit-profiles\") pod \"metrics-server-bc594fcb8-5w7qf\" (UID: \"f70c7d64-1725-43ff-8841-59691b5bf34d\") " pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" May 21 00:56:59.118709 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.118702 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-26mcf\" (UniqueName: \"kubernetes.io/projected/f70c7d64-1725-43ff-8841-59691b5bf34d-kube-api-access-26mcf\") pod \"metrics-server-bc594fcb8-5w7qf\" (UID: \"f70c7d64-1725-43ff-8841-59691b5bf34d\") " pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" May 21 00:56:59.118821 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.118780 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f70c7d64-1725-43ff-8841-59691b5bf34d-secret-metrics-server-client-certs\") pod \"metrics-server-bc594fcb8-5w7qf\" (UID: \"f70c7d64-1725-43ff-8841-59691b5bf34d\") " pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" May 21 00:56:59.118885 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.118871 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f70c7d64-1725-43ff-8841-59691b5bf34d-client-ca-bundle\") pod \"metrics-server-bc594fcb8-5w7qf\" (UID: \"f70c7d64-1725-43ff-8841-59691b5bf34d\") " pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" May 21 00:56:59.119437 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.119301 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f70c7d64-1725-43ff-8841-59691b5bf34d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-bc594fcb8-5w7qf\" (UID: \"f70c7d64-1725-43ff-8841-59691b5bf34d\") " pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" May 21 00:56:59.119555 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.119508 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f70c7d64-1725-43ff-8841-59691b5bf34d-audit-log\") pod \"metrics-server-bc594fcb8-5w7qf\" (UID: \"f70c7d64-1725-43ff-8841-59691b5bf34d\") " pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" May 21 00:56:59.119946 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.119920 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f70c7d64-1725-43ff-8841-59691b5bf34d-audit-log\") pod \"metrics-server-bc594fcb8-5w7qf\" (UID: \"f70c7d64-1725-43ff-8841-59691b5bf34d\") " pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" May 21 00:56:59.120334 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.120085 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f70c7d64-1725-43ff-8841-59691b5bf34d-metrics-server-audit-profiles\") pod \"metrics-server-bc594fcb8-5w7qf\" (UID: \"f70c7d64-1725-43ff-8841-59691b5bf34d\") " pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" May 21 00:56:59.121552 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.121515 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f70c7d64-1725-43ff-8841-59691b5bf34d-secret-metrics-server-tls\") pod \"metrics-server-bc594fcb8-5w7qf\" (UID: \"f70c7d64-1725-43ff-8841-59691b5bf34d\") " pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" May 21 00:56:59.121788 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.121767 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f70c7d64-1725-43ff-8841-59691b5bf34d-secret-metrics-server-client-certs\") pod \"metrics-server-bc594fcb8-5w7qf\" (UID: \"f70c7d64-1725-43ff-8841-59691b5bf34d\") " pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" May 21 00:56:59.121964 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.121942 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f70c7d64-1725-43ff-8841-59691b5bf34d-client-ca-bundle\") pod \"metrics-server-bc594fcb8-5w7qf\" (UID: \"f70c7d64-1725-43ff-8841-59691b5bf34d\") " pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" May 21 00:56:59.127244 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.127222 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26mcf\" (UniqueName: \"kubernetes.io/projected/f70c7d64-1725-43ff-8841-59691b5bf34d-kube-api-access-26mcf\") pod \"metrics-server-bc594fcb8-5w7qf\" (UID: \"f70c7d64-1725-43ff-8841-59691b5bf34d\") " pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" May 21 00:56:59.316570 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.316490 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" May 21 00:56:59.468650 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:56:59.468612 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-bc594fcb8-5w7qf"] May 21 00:56:59.472444 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:56:59.472409 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf70c7d64_1725_43ff_8841_59691b5bf34d.slice/crio-bff4e870ce07d974ded691a9b211782f075d6ae355679cac6f3972a9345ff13a WatchSource:0}: Error finding container bff4e870ce07d974ded691a9b211782f075d6ae355679cac6f3972a9345ff13a: Status 404 returned error can't find the container with id bff4e870ce07d974ded691a9b211782f075d6ae355679cac6f3972a9345ff13a May 21 00:57:00.071702 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.071659 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" event={"ID":"f70c7d64-1725-43ff-8841-59691b5bf34d","Type":"ContainerStarted","Data":"bff4e870ce07d974ded691a9b211782f075d6ae355679cac6f3972a9345ff13a"} May 21 00:57:00.784402 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.784367 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 21 00:57:00.787480 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.787458 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.789956 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.789936 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" May 21 00:57:00.790358 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.790338 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" May 21 00:57:00.790458 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.790374 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" May 21 00:57:00.791107 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.790861 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-fgg2s\"" May 21 00:57:00.791107 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.790945 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" May 21 00:57:00.791107 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.790967 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" May 21 00:57:00.791323 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.791210 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" May 21 00:57:00.791323 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.791244 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" May 21 00:57:00.791323 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.791292 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" May 21 00:57:00.791689 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.791668 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3qf6fgkonbhbg\"" May 21 00:57:00.791689 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.791682 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" May 21 00:57:00.791835 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.791773 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" May 21 00:57:00.794529 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.794509 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" May 21 00:57:00.796047 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.796025 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" May 21 00:57:00.799630 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.799611 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 21 00:57:00.835457 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.835426 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.835542 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.835484 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.835542 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.835508 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-config\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.835542 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.835540 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wcsn\" (UniqueName: \"kubernetes.io/projected/880694c0-306c-452e-bdfe-854e18b4f7bc-kube-api-access-4wcsn\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.835691 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.835566 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.835691 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.835587 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.835691 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.835614 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.835691 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.835644 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.835691 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.835672 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.835937 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.835719 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.835937 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.835784 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.835937 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.835815 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/880694c0-306c-452e-bdfe-854e18b4f7bc-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.835937 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.835861 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.835937 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.835885 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.835937 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.835910 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.836186 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.835939 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/880694c0-306c-452e-bdfe-854e18b4f7bc-config-out\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.836186 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.835964 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-web-config\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.836186 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.836011 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/880694c0-306c-452e-bdfe-854e18b4f7bc-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.937352 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.937322 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.937498 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.937358 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/880694c0-306c-452e-bdfe-854e18b4f7bc-config-out\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.937498 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.937377 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-web-config\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.937580 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.937491 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/880694c0-306c-452e-bdfe-854e18b4f7bc-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.937580 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.937544 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.937676 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.937608 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.937676 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.937651 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-config\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.937787 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.937686 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4wcsn\" (UniqueName: \"kubernetes.io/projected/880694c0-306c-452e-bdfe-854e18b4f7bc-kube-api-access-4wcsn\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.937787 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.937736 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.937787 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.937766 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.937930 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.937800 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.937930 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.937861 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.937930 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.937905 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.938089 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.937935 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.938089 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.937984 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.938089 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.938011 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/880694c0-306c-452e-bdfe-854e18b4f7bc-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.938089 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.938041 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.938296 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.938091 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.938349 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.938324 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.940483 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.940457 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-web-config\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.940929 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.940899 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.941215 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.941192 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.941306 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.941280 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.942447 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.941515 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/880694c0-306c-452e-bdfe-854e18b4f7bc-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.942447 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.941743 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.942447 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.941882 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/880694c0-306c-452e-bdfe-854e18b4f7bc-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.942447 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.942141 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.942753 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.942725 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.942961 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.942867 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.943540 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.943488 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.943907 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.943859 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/880694c0-306c-452e-bdfe-854e18b4f7bc-config-out\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.944131 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.944110 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-config\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.944238 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.944218 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.944496 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.944458 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.945400 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.945359 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:00.945509 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:00.945403 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wcsn\" (UniqueName: \"kubernetes.io/projected/880694c0-306c-452e-bdfe-854e18b4f7bc-kube-api-access-4wcsn\") pod \"prometheus-k8s-0\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:01.077897 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:01.077815 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" event={"ID":"2d540d00-7ee2-42d1-b163-8344925e6439","Type":"ContainerStarted","Data":"81309e9a23f2820f965da8619cd8e27f0b461619c0b508c80d193d9ac053387d"} May 21 00:57:01.077897 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:01.077873 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" event={"ID":"2d540d00-7ee2-42d1-b163-8344925e6439","Type":"ContainerStarted","Data":"02a6322f5da33697036b62de1b95155a93c2afaa24fe4e08a62d98a7f44607ca"} May 21 00:57:01.077897 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:01.077887 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" event={"ID":"2d540d00-7ee2-42d1-b163-8344925e6439","Type":"ContainerStarted","Data":"01c6e2a12418df96e98be1520e2a5877235aa6545eb2fa0820b84cd59c3fcdae"} May 21 00:57:01.100686 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:01.100661 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:01.508048 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:01.507906 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 21 00:57:01.513571 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:57:01.513539 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod880694c0_306c_452e_bdfe_854e18b4f7bc.slice/crio-624118699cda12edfd61545347195d039d44cbdf91d00708c61dea103b65920e WatchSource:0}: Error finding container 624118699cda12edfd61545347195d039d44cbdf91d00708c61dea103b65920e: Status 404 returned error can't find the container with id 624118699cda12edfd61545347195d039d44cbdf91d00708c61dea103b65920e May 21 00:57:02.084336 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:02.084291 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"194459e3-e611-4eb3-adfe-61bec9c5f2c9","Type":"ContainerStarted","Data":"dc2ff51afa78d404e0291458d7ecce540d15c42f45cbe8bbd68f79ff3073a0fe"} May 21 00:57:02.084795 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:02.084341 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"194459e3-e611-4eb3-adfe-61bec9c5f2c9","Type":"ContainerStarted","Data":"3cb0811f7885aa8054d84e85d4522ee45a8c5d88cd623bcfa3f979cb07706cd8"} May 21 00:57:02.084795 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:02.084356 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"194459e3-e611-4eb3-adfe-61bec9c5f2c9","Type":"ContainerStarted","Data":"b7272691c929d90770ad24a57b1e097c6b225826fd2afd95ab84cc9deb0207e8"} May 21 00:57:02.084795 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:02.084371 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"194459e3-e611-4eb3-adfe-61bec9c5f2c9","Type":"ContainerStarted","Data":"20a39cd610e44b44cf25127fc9ad356e338e08d2514a4ec86df968bb55aced70"} May 21 00:57:02.084795 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:02.084383 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"194459e3-e611-4eb3-adfe-61bec9c5f2c9","Type":"ContainerStarted","Data":"a7f78dca6487d7aa396c7efceadbca02dd05e03d1f674afce5f0e010d32e7319"} May 21 00:57:02.084795 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:02.084396 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"194459e3-e611-4eb3-adfe-61bec9c5f2c9","Type":"ContainerStarted","Data":"eb4ad103f0d181e6319143adf1d9d4c6e1b684d528a0225d1ee686107c3edfd2"} May 21 00:57:02.087101 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:02.087051 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" event={"ID":"2d540d00-7ee2-42d1-b163-8344925e6439","Type":"ContainerStarted","Data":"b6c9a3905547f6e25f121100711b91e364cd2c173535f780c4ed5a858d005c69"} May 21 00:57:02.087212 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:02.087105 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" event={"ID":"2d540d00-7ee2-42d1-b163-8344925e6439","Type":"ContainerStarted","Data":"1cc2298fb10882b766eb19b06598f865291b6ada90bbac0123742eab2acdc3d0"} May 21 00:57:02.087212 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:02.087119 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" event={"ID":"2d540d00-7ee2-42d1-b163-8344925e6439","Type":"ContainerStarted","Data":"766bbe578c82b08410da6b6d48170331022173db6144cf46810b19c6eab0b50a"} May 21 00:57:02.087326 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:02.087310 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:57:02.088706 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:02.088683 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" event={"ID":"f70c7d64-1725-43ff-8841-59691b5bf34d","Type":"ContainerStarted","Data":"feb2bfb713f620839f914960625ab165e71801acce4beabb82f31cc01b391110"} May 21 00:57:02.089781 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:02.089758 2549 generic.go:358] "Generic (PLEG): container finished" podID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerID="11f835b1bc28aaa5a878f188a3dfeebf0bfd8b7ce44e6af3ed388750d38d32fe" exitCode=0 May 21 00:57:02.089877 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:02.089806 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880694c0-306c-452e-bdfe-854e18b4f7bc","Type":"ContainerDied","Data":"11f835b1bc28aaa5a878f188a3dfeebf0bfd8b7ce44e6af3ed388750d38d32fe"} May 21 00:57:02.089877 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:02.089826 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880694c0-306c-452e-bdfe-854e18b4f7bc","Type":"ContainerStarted","Data":"624118699cda12edfd61545347195d039d44cbdf91d00708c61dea103b65920e"} May 21 00:57:02.109183 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:02.109020 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.5700060110000003 podStartE2EDuration="7.109005153s" podCreationTimestamp="2026-05-21 00:56:55 +0000 UTC" firstStartedPulling="2026-05-21 00:56:56.823938062 +0000 UTC m=+135.720883355" lastFinishedPulling="2026-05-21 00:57:01.362937207 +0000 UTC m=+140.259882497" observedRunningTime="2026-05-21 00:57:02.10745545 +0000 UTC m=+141.004400758" watchObservedRunningTime="2026-05-21 00:57:02.109005153 +0000 UTC m=+141.005950465" May 21 00:57:02.132076 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:02.132008 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" podStartSLOduration=1.9301726540000002 podStartE2EDuration="5.131993771s" podCreationTimestamp="2026-05-21 00:56:57 +0000 UTC" firstStartedPulling="2026-05-21 00:56:58.209113846 +0000 UTC m=+137.106059135" lastFinishedPulling="2026-05-21 00:57:01.410934952 +0000 UTC m=+140.307880252" observedRunningTime="2026-05-21 00:57:02.129572935 +0000 UTC m=+141.026518245" watchObservedRunningTime="2026-05-21 00:57:02.131993771 +0000 UTC m=+141.028939083" May 21 00:57:02.144671 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:02.144632 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" podStartSLOduration=2.256268148 podStartE2EDuration="4.144620444s" podCreationTimestamp="2026-05-21 00:56:58 +0000 UTC" firstStartedPulling="2026-05-21 00:56:59.47458896 +0000 UTC m=+138.371534249" lastFinishedPulling="2026-05-21 00:57:01.362941257 +0000 UTC m=+140.259886545" observedRunningTime="2026-05-21 00:57:02.143784035 +0000 UTC m=+141.040729348" watchObservedRunningTime="2026-05-21 00:57:02.144620444 +0000 UTC m=+141.041565755" May 21 00:57:06.112163 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:06.112122 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880694c0-306c-452e-bdfe-854e18b4f7bc","Type":"ContainerStarted","Data":"11c419f9c728201a34d0421122be6c427c6de37f48c85ad55d841a5732e10128"} May 21 00:57:06.112497 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:06.112168 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880694c0-306c-452e-bdfe-854e18b4f7bc","Type":"ContainerStarted","Data":"cf21519a24a3c5eff6025331e7995abc101a69c2eb0ea03c1ddd53b20572de84"} May 21 00:57:06.112497 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:06.112182 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880694c0-306c-452e-bdfe-854e18b4f7bc","Type":"ContainerStarted","Data":"c0ea5f184f696d6e218a393b6bad93c3a47932343a6f387768b03d6724cd8aff"} May 21 00:57:06.112497 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:06.112195 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880694c0-306c-452e-bdfe-854e18b4f7bc","Type":"ContainerStarted","Data":"c57e36c51bf23d8653609cccd55d866d0253ea34d64617cfe99de35295e0bf4f"} May 21 00:57:06.112497 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:06.112206 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880694c0-306c-452e-bdfe-854e18b4f7bc","Type":"ContainerStarted","Data":"7fad54173c9cf8d73bb3b2ddcdc65958ece553b10dc89aaa20b8fdb0c6f0690a"} May 21 00:57:06.112497 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:06.112218 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880694c0-306c-452e-bdfe-854e18b4f7bc","Type":"ContainerStarted","Data":"e5e25146683be19210d6c9d99a216062464b0b144ce801d09cd561e0c1a67d2d"} May 21 00:57:06.136289 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:06.136246 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.948316487 podStartE2EDuration="6.136228357s" podCreationTimestamp="2026-05-21 00:57:00 +0000 UTC" firstStartedPulling="2026-05-21 00:57:02.090845749 +0000 UTC m=+140.987791038" lastFinishedPulling="2026-05-21 00:57:05.278757619 +0000 UTC m=+144.175702908" observedRunningTime="2026-05-21 00:57:06.135227075 +0000 UTC m=+145.032172387" watchObservedRunningTime="2026-05-21 00:57:06.136228357 +0000 UTC m=+145.033173669" May 21 00:57:08.099758 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:08.099728 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6ff586c7c5-5ps4g" May 21 00:57:08.196044 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:08.196010 2549 patch_prober.go:28] interesting pod/image-registry-9d999d7b8-pw9kx container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} May 21 00:57:08.196221 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:08.196096 2549 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" podUID="68d5c953-f922-4671-b7ba-5436cb108e4e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" May 21 00:57:11.101517 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:11.101476 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" May 21 00:57:13.210211 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:13.210140 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" podUID="68d5c953-f922-4671-b7ba-5436cb108e4e" containerName="registry" containerID="cri-o://06272803ed87a1b8cb2f7c34ffe9b12f90bcd79a4c46a2f3c3f1df197014d89c" gracePeriod=30 May 21 00:57:13.441591 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:13.441567 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:57:13.548438 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:13.548409 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-bound-sa-token\") pod \"68d5c953-f922-4671-b7ba-5436cb108e4e\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " May 21 00:57:13.548600 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:13.548463 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/68d5c953-f922-4671-b7ba-5436cb108e4e-registry-certificates\") pod \"68d5c953-f922-4671-b7ba-5436cb108e4e\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " May 21 00:57:13.548600 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:13.548514 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7gmt\" (UniqueName: \"kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-kube-api-access-b7gmt\") pod \"68d5c953-f922-4671-b7ba-5436cb108e4e\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " May 21 00:57:13.548600 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:13.548557 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-registry-tls\") pod \"68d5c953-f922-4671-b7ba-5436cb108e4e\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " May 21 00:57:13.548600 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:13.548587 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/68d5c953-f922-4671-b7ba-5436cb108e4e-trusted-ca\") pod \"68d5c953-f922-4671-b7ba-5436cb108e4e\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " May 21 00:57:13.548801 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:13.548627 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/68d5c953-f922-4671-b7ba-5436cb108e4e-installation-pull-secrets\") pod \"68d5c953-f922-4671-b7ba-5436cb108e4e\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " May 21 00:57:13.548801 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:13.548770 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/68d5c953-f922-4671-b7ba-5436cb108e4e-image-registry-private-configuration\") pod \"68d5c953-f922-4671-b7ba-5436cb108e4e\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " May 21 00:57:13.548895 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:13.548811 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/68d5c953-f922-4671-b7ba-5436cb108e4e-ca-trust-extracted\") pod \"68d5c953-f922-4671-b7ba-5436cb108e4e\" (UID: \"68d5c953-f922-4671-b7ba-5436cb108e4e\") " May 21 00:57:13.549252 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:13.548981 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68d5c953-f922-4671-b7ba-5436cb108e4e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "68d5c953-f922-4671-b7ba-5436cb108e4e" (UID: "68d5c953-f922-4671-b7ba-5436cb108e4e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 21 00:57:13.549252 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:13.549193 2549 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/68d5c953-f922-4671-b7ba-5436cb108e4e-registry-certificates\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:57:13.549252 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:13.549213 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68d5c953-f922-4671-b7ba-5436cb108e4e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "68d5c953-f922-4671-b7ba-5436cb108e4e" (UID: "68d5c953-f922-4671-b7ba-5436cb108e4e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 21 00:57:13.550844 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:13.550794 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "68d5c953-f922-4671-b7ba-5436cb108e4e" (UID: "68d5c953-f922-4671-b7ba-5436cb108e4e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 21 00:57:13.550943 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:13.550917 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68d5c953-f922-4671-b7ba-5436cb108e4e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "68d5c953-f922-4671-b7ba-5436cb108e4e" (UID: "68d5c953-f922-4671-b7ba-5436cb108e4e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 00:57:13.551166 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:13.551143 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-kube-api-access-b7gmt" (OuterVolumeSpecName: "kube-api-access-b7gmt") pod "68d5c953-f922-4671-b7ba-5436cb108e4e" (UID: "68d5c953-f922-4671-b7ba-5436cb108e4e"). InnerVolumeSpecName "kube-api-access-b7gmt". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 21 00:57:13.551397 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:13.551376 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "68d5c953-f922-4671-b7ba-5436cb108e4e" (UID: "68d5c953-f922-4671-b7ba-5436cb108e4e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 21 00:57:13.551451 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:13.551386 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68d5c953-f922-4671-b7ba-5436cb108e4e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "68d5c953-f922-4671-b7ba-5436cb108e4e" (UID: "68d5c953-f922-4671-b7ba-5436cb108e4e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 00:57:13.557356 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:13.557336 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68d5c953-f922-4671-b7ba-5436cb108e4e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "68d5c953-f922-4671-b7ba-5436cb108e4e" (UID: "68d5c953-f922-4671-b7ba-5436cb108e4e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 21 00:57:13.650217 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:13.650193 2549 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-registry-tls\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:57:13.650217 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:13.650215 2549 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/68d5c953-f922-4671-b7ba-5436cb108e4e-trusted-ca\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:57:13.650349 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:13.650229 2549 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/68d5c953-f922-4671-b7ba-5436cb108e4e-installation-pull-secrets\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:57:13.650349 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:13.650243 2549 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/68d5c953-f922-4671-b7ba-5436cb108e4e-image-registry-private-configuration\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:57:13.650349 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:13.650257 2549 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/68d5c953-f922-4671-b7ba-5436cb108e4e-ca-trust-extracted\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:57:13.650349 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:13.650270 2549 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-bound-sa-token\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:57:13.650349 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:13.650284 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7gmt\" (UniqueName: \"kubernetes.io/projected/68d5c953-f922-4671-b7ba-5436cb108e4e-kube-api-access-b7gmt\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:57:14.135264 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:14.135233 2549 generic.go:358] "Generic (PLEG): container finished" podID="68d5c953-f922-4671-b7ba-5436cb108e4e" containerID="06272803ed87a1b8cb2f7c34ffe9b12f90bcd79a4c46a2f3c3f1df197014d89c" exitCode=0 May 21 00:57:14.135405 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:14.135289 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" event={"ID":"68d5c953-f922-4671-b7ba-5436cb108e4e","Type":"ContainerDied","Data":"06272803ed87a1b8cb2f7c34ffe9b12f90bcd79a4c46a2f3c3f1df197014d89c"} May 21 00:57:14.135405 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:14.135312 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" event={"ID":"68d5c953-f922-4671-b7ba-5436cb108e4e","Type":"ContainerDied","Data":"e6b75e7dc8ade0736ae0b96fd3023eb125e4383668efd9c983c58fc3b5c02dd9"} May 21 00:57:14.135405 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:14.135326 2549 scope.go:117] "RemoveContainer" containerID="06272803ed87a1b8cb2f7c34ffe9b12f90bcd79a4c46a2f3c3f1df197014d89c" May 21 00:57:14.135405 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:14.135291 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9d999d7b8-pw9kx" May 21 00:57:14.143014 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:14.142997 2549 scope.go:117] "RemoveContainer" containerID="06272803ed87a1b8cb2f7c34ffe9b12f90bcd79a4c46a2f3c3f1df197014d89c" May 21 00:57:14.143289 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:57:14.143267 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06272803ed87a1b8cb2f7c34ffe9b12f90bcd79a4c46a2f3c3f1df197014d89c\": container with ID starting with 06272803ed87a1b8cb2f7c34ffe9b12f90bcd79a4c46a2f3c3f1df197014d89c not found: ID does not exist" containerID="06272803ed87a1b8cb2f7c34ffe9b12f90bcd79a4c46a2f3c3f1df197014d89c" May 21 00:57:14.143333 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:14.143298 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06272803ed87a1b8cb2f7c34ffe9b12f90bcd79a4c46a2f3c3f1df197014d89c"} err="failed to get container status \"06272803ed87a1b8cb2f7c34ffe9b12f90bcd79a4c46a2f3c3f1df197014d89c\": rpc error: code = NotFound desc = could not find container \"06272803ed87a1b8cb2f7c34ffe9b12f90bcd79a4c46a2f3c3f1df197014d89c\": container with ID starting with 06272803ed87a1b8cb2f7c34ffe9b12f90bcd79a4c46a2f3c3f1df197014d89c not found: ID does not exist" May 21 00:57:14.150513 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:14.150492 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-9d999d7b8-pw9kx"] May 21 00:57:14.156136 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:14.156114 2549 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-9d999d7b8-pw9kx"] May 21 00:57:15.670971 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:15.670941 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68d5c953-f922-4671-b7ba-5436cb108e4e" path="/var/lib/kubelet/pods/68d5c953-f922-4671-b7ba-5436cb108e4e/volumes" May 21 00:57:18.521357 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:57:18.521316 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5mlhs" podUID="776a4785-ccb6-42c4-b6d8-050c42ad5541" May 21 00:57:18.534433 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:57:18.534410 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-2lhq8" podUID="877e95c2-b86f-49d3-a207-eb91f732bfff" May 21 00:57:19.150987 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:19.150959 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5mlhs" May 21 00:57:19.317534 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:19.317503 2549 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" May 21 00:57:19.317534 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:19.317544 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" May 21 00:57:23.431451 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:23.431417 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/776a4785-ccb6-42c4-b6d8-050c42ad5541-metrics-tls\") pod \"dns-default-5mlhs\" (UID: \"776a4785-ccb6-42c4-b6d8-050c42ad5541\") " pod="openshift-dns/dns-default-5mlhs" May 21 00:57:23.431859 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:23.431484 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/877e95c2-b86f-49d3-a207-eb91f732bfff-cert\") pod \"ingress-canary-2lhq8\" (UID: \"877e95c2-b86f-49d3-a207-eb91f732bfff\") " pod="openshift-ingress-canary/ingress-canary-2lhq8" May 21 00:57:23.433821 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:23.433790 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/776a4785-ccb6-42c4-b6d8-050c42ad5541-metrics-tls\") pod \"dns-default-5mlhs\" (UID: \"776a4785-ccb6-42c4-b6d8-050c42ad5541\") " pod="openshift-dns/dns-default-5mlhs" May 21 00:57:23.433939 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:23.433855 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/877e95c2-b86f-49d3-a207-eb91f732bfff-cert\") pod \"ingress-canary-2lhq8\" (UID: \"877e95c2-b86f-49d3-a207-eb91f732bfff\") " pod="openshift-ingress-canary/ingress-canary-2lhq8" May 21 00:57:23.653862 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:23.653828 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8hsl2\"" May 21 00:57:23.662752 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:23.662723 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5mlhs" May 21 00:57:23.781560 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:23.781535 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5mlhs"] May 21 00:57:24.165334 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:24.165297 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5mlhs" event={"ID":"776a4785-ccb6-42c4-b6d8-050c42ad5541","Type":"ContainerStarted","Data":"1959046c41dee414ab45c33a0ba4ae155b9a86140339b6c6e508225323a3db55"} May 21 00:57:26.172854 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:26.172769 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5mlhs" event={"ID":"776a4785-ccb6-42c4-b6d8-050c42ad5541","Type":"ContainerStarted","Data":"1ab61b40d5c2f7e3cfb29d531ebd4633aa4a030e849e0dc43c1db54c033564e5"} May 21 00:57:26.172854 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:26.172806 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5mlhs" event={"ID":"776a4785-ccb6-42c4-b6d8-050c42ad5541","Type":"ContainerStarted","Data":"c0dfaef95773c638031322c0149c3baf15c517af0d39593aace4684eab753975"} May 21 00:57:26.173375 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:26.172899 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5mlhs" May 21 00:57:26.192006 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:26.191959 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-5mlhs" podStartSLOduration=129.832220813 podStartE2EDuration="2m11.19194658s" podCreationTimestamp="2026-05-21 00:55:15 +0000 UTC" firstStartedPulling="2026-05-21 00:57:23.787658748 +0000 UTC m=+162.684604036" lastFinishedPulling="2026-05-21 00:57:25.147384513 +0000 UTC m=+164.044329803" observedRunningTime="2026-05-21 00:57:26.190171796 +0000 UTC m=+165.087117108" watchObservedRunningTime="2026-05-21 00:57:26.19194658 +0000 UTC m=+165.088891890" May 21 00:57:31.669328 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:31.669296 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2lhq8" May 21 00:57:31.671632 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:31.671608 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8sxzc\"" May 21 00:57:31.680284 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:31.680259 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2lhq8" May 21 00:57:31.799541 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:31.799508 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2lhq8"] May 21 00:57:31.802307 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:57:31.802281 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod877e95c2_b86f_49d3_a207_eb91f732bfff.slice/crio-ae19e39f3d347b58cb76c79d8d88eeaa03fabc85e4d97d641fe11415cc6a0c39 WatchSource:0}: Error finding container ae19e39f3d347b58cb76c79d8d88eeaa03fabc85e4d97d641fe11415cc6a0c39: Status 404 returned error can't find the container with id ae19e39f3d347b58cb76c79d8d88eeaa03fabc85e4d97d641fe11415cc6a0c39 May 21 00:57:32.195920 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:32.195878 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2lhq8" event={"ID":"877e95c2-b86f-49d3-a207-eb91f732bfff","Type":"ContainerStarted","Data":"ae19e39f3d347b58cb76c79d8d88eeaa03fabc85e4d97d641fe11415cc6a0c39"} May 21 00:57:34.203684 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:34.203649 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2lhq8" event={"ID":"877e95c2-b86f-49d3-a207-eb91f732bfff","Type":"ContainerStarted","Data":"39484be515c78f6b154b2d1fbafe95786159fff843ac586aeb7bf6276ee615cb"} May 21 00:57:34.217476 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:34.217419 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2lhq8" podStartSLOduration=137.759864875 podStartE2EDuration="2m19.217401225s" podCreationTimestamp="2026-05-21 00:55:15 +0000 UTC" firstStartedPulling="2026-05-21 00:57:31.804145266 +0000 UTC m=+170.701090554" lastFinishedPulling="2026-05-21 00:57:33.261681613 +0000 UTC m=+172.158626904" observedRunningTime="2026-05-21 00:57:34.217149333 +0000 UTC m=+173.114094641" watchObservedRunningTime="2026-05-21 00:57:34.217401225 +0000 UTC m=+173.114346537" May 21 00:57:36.177809 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:36.177780 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5mlhs" May 21 00:57:39.323026 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:39.322987 2549 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" May 21 00:57:39.326964 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:57:39.326941 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-bc594fcb8-5w7qf" May 21 00:58:01.101390 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:01.101360 2549 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:01.116593 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:01.116569 2549 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:01.296524 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:01.296499 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:14.856157 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:14.856124 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 21 00:58:14.856622 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:14.856538 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerName="alertmanager" containerID="cri-o://eb4ad103f0d181e6319143adf1d9d4c6e1b684d528a0225d1ee686107c3edfd2" gracePeriod=120 May 21 00:58:14.856704 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:14.856676 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerName="kube-rbac-proxy" containerID="cri-o://b7272691c929d90770ad24a57b1e097c6b225826fd2afd95ab84cc9deb0207e8" gracePeriod=120 May 21 00:58:14.856990 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:14.856639 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerName="kube-rbac-proxy-metric" containerID="cri-o://3cb0811f7885aa8054d84e85d4522ee45a8c5d88cd623bcfa3f979cb07706cd8" gracePeriod=120 May 21 00:58:14.857112 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:14.856658 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerName="prom-label-proxy" containerID="cri-o://dc2ff51afa78d404e0291458d7ecce540d15c42f45cbe8bbd68f79ff3073a0fe" gracePeriod=120 May 21 00:58:14.857112 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:14.856718 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerName="config-reloader" containerID="cri-o://a7f78dca6487d7aa396c7efceadbca02dd05e03d1f674afce5f0e010d32e7319" gracePeriod=120 May 21 00:58:14.857577 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:14.856742 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerName="kube-rbac-proxy-web" containerID="cri-o://20a39cd610e44b44cf25127fc9ad356e338e08d2514a4ec86df968bb55aced70" gracePeriod=120 May 21 00:58:15.323722 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:15.323687 2549 generic.go:358] "Generic (PLEG): container finished" podID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerID="dc2ff51afa78d404e0291458d7ecce540d15c42f45cbe8bbd68f79ff3073a0fe" exitCode=0 May 21 00:58:15.323722 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:15.323719 2549 generic.go:358] "Generic (PLEG): container finished" podID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerID="b7272691c929d90770ad24a57b1e097c6b225826fd2afd95ab84cc9deb0207e8" exitCode=0 May 21 00:58:15.323722 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:15.323727 2549 generic.go:358] "Generic (PLEG): container finished" podID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerID="a7f78dca6487d7aa396c7efceadbca02dd05e03d1f674afce5f0e010d32e7319" exitCode=0 May 21 00:58:15.323949 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:15.323735 2549 generic.go:358] "Generic (PLEG): container finished" podID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerID="eb4ad103f0d181e6319143adf1d9d4c6e1b684d528a0225d1ee686107c3edfd2" exitCode=0 May 21 00:58:15.323949 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:15.323759 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"194459e3-e611-4eb3-adfe-61bec9c5f2c9","Type":"ContainerDied","Data":"dc2ff51afa78d404e0291458d7ecce540d15c42f45cbe8bbd68f79ff3073a0fe"} May 21 00:58:15.323949 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:15.323797 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"194459e3-e611-4eb3-adfe-61bec9c5f2c9","Type":"ContainerDied","Data":"b7272691c929d90770ad24a57b1e097c6b225826fd2afd95ab84cc9deb0207e8"} May 21 00:58:15.323949 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:15.323808 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"194459e3-e611-4eb3-adfe-61bec9c5f2c9","Type":"ContainerDied","Data":"a7f78dca6487d7aa396c7efceadbca02dd05e03d1f674afce5f0e010d32e7319"} May 21 00:58:15.323949 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:15.323819 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"194459e3-e611-4eb3-adfe-61bec9c5f2c9","Type":"ContainerDied","Data":"eb4ad103f0d181e6319143adf1d9d4c6e1b684d528a0225d1ee686107c3edfd2"} May 21 00:58:16.090874 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.090853 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.134103 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.134077 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-config-volume\") pod \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " May 21 00:58:16.134247 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.134114 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/194459e3-e611-4eb3-adfe-61bec9c5f2c9-alertmanager-main-db\") pod \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " May 21 00:58:16.134247 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.134136 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-cluster-tls-config\") pod \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " May 21 00:58:16.134247 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.134159 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " May 21 00:58:16.134247 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.134176 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nz72r\" (UniqueName: \"kubernetes.io/projected/194459e3-e611-4eb3-adfe-61bec9c5f2c9-kube-api-access-nz72r\") pod \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " May 21 00:58:16.134247 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.134212 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/194459e3-e611-4eb3-adfe-61bec9c5f2c9-tls-assets\") pod \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " May 21 00:58:16.134247 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.134237 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/194459e3-e611-4eb3-adfe-61bec9c5f2c9-metrics-client-ca\") pod \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " May 21 00:58:16.134563 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.134265 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/194459e3-e611-4eb3-adfe-61bec9c5f2c9-alertmanager-trusted-ca-bundle\") pod \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " May 21 00:58:16.134563 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.134305 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-secret-alertmanager-main-tls\") pod \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " May 21 00:58:16.134563 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.134330 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-web-config\") pod \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " May 21 00:58:16.134563 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.134368 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/194459e3-e611-4eb3-adfe-61bec9c5f2c9-config-out\") pod \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " May 21 00:58:16.134563 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.134410 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-secret-alertmanager-kube-rbac-proxy\") pod \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " May 21 00:58:16.134563 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.134437 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-secret-alertmanager-kube-rbac-proxy-web\") pod \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\" (UID: \"194459e3-e611-4eb3-adfe-61bec9c5f2c9\") " May 21 00:58:16.134563 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.134500 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/194459e3-e611-4eb3-adfe-61bec9c5f2c9-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "194459e3-e611-4eb3-adfe-61bec9c5f2c9" (UID: "194459e3-e611-4eb3-adfe-61bec9c5f2c9"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 21 00:58:16.134957 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.134748 2549 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/194459e3-e611-4eb3-adfe-61bec9c5f2c9-alertmanager-main-db\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:16.134957 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.134770 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/194459e3-e611-4eb3-adfe-61bec9c5f2c9-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "194459e3-e611-4eb3-adfe-61bec9c5f2c9" (UID: "194459e3-e611-4eb3-adfe-61bec9c5f2c9"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 21 00:58:16.135281 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.135239 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/194459e3-e611-4eb3-adfe-61bec9c5f2c9-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "194459e3-e611-4eb3-adfe-61bec9c5f2c9" (UID: "194459e3-e611-4eb3-adfe-61bec9c5f2c9"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 21 00:58:16.138940 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.138875 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "194459e3-e611-4eb3-adfe-61bec9c5f2c9" (UID: "194459e3-e611-4eb3-adfe-61bec9c5f2c9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 00:58:16.139129 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.139001 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/194459e3-e611-4eb3-adfe-61bec9c5f2c9-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "194459e3-e611-4eb3-adfe-61bec9c5f2c9" (UID: "194459e3-e611-4eb3-adfe-61bec9c5f2c9"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 21 00:58:16.139129 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.139042 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "194459e3-e611-4eb3-adfe-61bec9c5f2c9" (UID: "194459e3-e611-4eb3-adfe-61bec9c5f2c9"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 00:58:16.139454 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.139421 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/194459e3-e611-4eb3-adfe-61bec9c5f2c9-config-out" (OuterVolumeSpecName: "config-out") pod "194459e3-e611-4eb3-adfe-61bec9c5f2c9" (UID: "194459e3-e611-4eb3-adfe-61bec9c5f2c9"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 21 00:58:16.139550 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.139530 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-config-volume" (OuterVolumeSpecName: "config-volume") pod "194459e3-e611-4eb3-adfe-61bec9c5f2c9" (UID: "194459e3-e611-4eb3-adfe-61bec9c5f2c9"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 00:58:16.139671 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.139635 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "194459e3-e611-4eb3-adfe-61bec9c5f2c9" (UID: "194459e3-e611-4eb3-adfe-61bec9c5f2c9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 00:58:16.139765 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.139685 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "194459e3-e611-4eb3-adfe-61bec9c5f2c9" (UID: "194459e3-e611-4eb3-adfe-61bec9c5f2c9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 00:58:16.139843 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.139820 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/194459e3-e611-4eb3-adfe-61bec9c5f2c9-kube-api-access-nz72r" (OuterVolumeSpecName: "kube-api-access-nz72r") pod "194459e3-e611-4eb3-adfe-61bec9c5f2c9" (UID: "194459e3-e611-4eb3-adfe-61bec9c5f2c9"). InnerVolumeSpecName "kube-api-access-nz72r". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 21 00:58:16.142934 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.142907 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "194459e3-e611-4eb3-adfe-61bec9c5f2c9" (UID: "194459e3-e611-4eb3-adfe-61bec9c5f2c9"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 00:58:16.148844 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.148805 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-web-config" (OuterVolumeSpecName: "web-config") pod "194459e3-e611-4eb3-adfe-61bec9c5f2c9" (UID: "194459e3-e611-4eb3-adfe-61bec9c5f2c9"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 00:58:16.235745 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.235694 2549 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-config-volume\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:16.235745 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.235715 2549 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-cluster-tls-config\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:16.235745 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.235724 2549 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:16.235745 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.235734 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nz72r\" (UniqueName: \"kubernetes.io/projected/194459e3-e611-4eb3-adfe-61bec9c5f2c9-kube-api-access-nz72r\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:16.235745 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.235745 2549 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/194459e3-e611-4eb3-adfe-61bec9c5f2c9-tls-assets\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:16.235942 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.235753 2549 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/194459e3-e611-4eb3-adfe-61bec9c5f2c9-metrics-client-ca\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:16.235942 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.235762 2549 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/194459e3-e611-4eb3-adfe-61bec9c5f2c9-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:16.235942 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.235771 2549 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-secret-alertmanager-main-tls\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:16.235942 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.235779 2549 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-web-config\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:16.235942 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.235787 2549 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/194459e3-e611-4eb3-adfe-61bec9c5f2c9-config-out\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:16.235942 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.235795 2549 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:16.235942 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.235803 2549 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/194459e3-e611-4eb3-adfe-61bec9c5f2c9-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:16.329650 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.329622 2549 generic.go:358] "Generic (PLEG): container finished" podID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerID="3cb0811f7885aa8054d84e85d4522ee45a8c5d88cd623bcfa3f979cb07706cd8" exitCode=0 May 21 00:58:16.329650 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.329648 2549 generic.go:358] "Generic (PLEG): container finished" podID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerID="20a39cd610e44b44cf25127fc9ad356e338e08d2514a4ec86df968bb55aced70" exitCode=0 May 21 00:58:16.329803 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.329723 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.329803 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.329706 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"194459e3-e611-4eb3-adfe-61bec9c5f2c9","Type":"ContainerDied","Data":"3cb0811f7885aa8054d84e85d4522ee45a8c5d88cd623bcfa3f979cb07706cd8"} May 21 00:58:16.329803 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.329772 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"194459e3-e611-4eb3-adfe-61bec9c5f2c9","Type":"ContainerDied","Data":"20a39cd610e44b44cf25127fc9ad356e338e08d2514a4ec86df968bb55aced70"} May 21 00:58:16.329803 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.329784 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"194459e3-e611-4eb3-adfe-61bec9c5f2c9","Type":"ContainerDied","Data":"36c1add786097a558910b98623273a233f974a37c226e6a851ba29338f8ca799"} May 21 00:58:16.329803 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.329798 2549 scope.go:117] "RemoveContainer" containerID="dc2ff51afa78d404e0291458d7ecce540d15c42f45cbe8bbd68f79ff3073a0fe" May 21 00:58:16.337832 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.337814 2549 scope.go:117] "RemoveContainer" containerID="3cb0811f7885aa8054d84e85d4522ee45a8c5d88cd623bcfa3f979cb07706cd8" May 21 00:58:16.344445 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.344430 2549 scope.go:117] "RemoveContainer" containerID="b7272691c929d90770ad24a57b1e097c6b225826fd2afd95ab84cc9deb0207e8" May 21 00:58:16.350621 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.350602 2549 scope.go:117] "RemoveContainer" containerID="20a39cd610e44b44cf25127fc9ad356e338e08d2514a4ec86df968bb55aced70" May 21 00:58:16.353485 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.353467 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 21 00:58:16.360045 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.360020 2549 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 21 00:58:16.361621 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.361592 2549 scope.go:117] "RemoveContainer" containerID="a7f78dca6487d7aa396c7efceadbca02dd05e03d1f674afce5f0e010d32e7319" May 21 00:58:16.368222 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.368206 2549 scope.go:117] "RemoveContainer" containerID="eb4ad103f0d181e6319143adf1d9d4c6e1b684d528a0225d1ee686107c3edfd2" May 21 00:58:16.374701 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.374686 2549 scope.go:117] "RemoveContainer" containerID="bf736ffa394dbd7440ca95cf34a40a6c802890d3afb128e13029bc87e7ee9c10" May 21 00:58:16.381459 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.381445 2549 scope.go:117] "RemoveContainer" containerID="dc2ff51afa78d404e0291458d7ecce540d15c42f45cbe8bbd68f79ff3073a0fe" May 21 00:58:16.381747 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:58:16.381725 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc2ff51afa78d404e0291458d7ecce540d15c42f45cbe8bbd68f79ff3073a0fe\": container with ID starting with dc2ff51afa78d404e0291458d7ecce540d15c42f45cbe8bbd68f79ff3073a0fe not found: ID does not exist" containerID="dc2ff51afa78d404e0291458d7ecce540d15c42f45cbe8bbd68f79ff3073a0fe" May 21 00:58:16.381832 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.381759 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc2ff51afa78d404e0291458d7ecce540d15c42f45cbe8bbd68f79ff3073a0fe"} err="failed to get container status \"dc2ff51afa78d404e0291458d7ecce540d15c42f45cbe8bbd68f79ff3073a0fe\": rpc error: code = NotFound desc = could not find container \"dc2ff51afa78d404e0291458d7ecce540d15c42f45cbe8bbd68f79ff3073a0fe\": container with ID starting with dc2ff51afa78d404e0291458d7ecce540d15c42f45cbe8bbd68f79ff3073a0fe not found: ID does not exist" May 21 00:58:16.381832 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.381785 2549 scope.go:117] "RemoveContainer" containerID="3cb0811f7885aa8054d84e85d4522ee45a8c5d88cd623bcfa3f979cb07706cd8" May 21 00:58:16.382087 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:58:16.382044 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cb0811f7885aa8054d84e85d4522ee45a8c5d88cd623bcfa3f979cb07706cd8\": container with ID starting with 3cb0811f7885aa8054d84e85d4522ee45a8c5d88cd623bcfa3f979cb07706cd8 not found: ID does not exist" containerID="3cb0811f7885aa8054d84e85d4522ee45a8c5d88cd623bcfa3f979cb07706cd8" May 21 00:58:16.382176 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.382098 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cb0811f7885aa8054d84e85d4522ee45a8c5d88cd623bcfa3f979cb07706cd8"} err="failed to get container status \"3cb0811f7885aa8054d84e85d4522ee45a8c5d88cd623bcfa3f979cb07706cd8\": rpc error: code = NotFound desc = could not find container \"3cb0811f7885aa8054d84e85d4522ee45a8c5d88cd623bcfa3f979cb07706cd8\": container with ID starting with 3cb0811f7885aa8054d84e85d4522ee45a8c5d88cd623bcfa3f979cb07706cd8 not found: ID does not exist" May 21 00:58:16.382176 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.382120 2549 scope.go:117] "RemoveContainer" containerID="b7272691c929d90770ad24a57b1e097c6b225826fd2afd95ab84cc9deb0207e8" May 21 00:58:16.382401 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:58:16.382383 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7272691c929d90770ad24a57b1e097c6b225826fd2afd95ab84cc9deb0207e8\": container with ID starting with b7272691c929d90770ad24a57b1e097c6b225826fd2afd95ab84cc9deb0207e8 not found: ID does not exist" containerID="b7272691c929d90770ad24a57b1e097c6b225826fd2afd95ab84cc9deb0207e8" May 21 00:58:16.382443 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.382406 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7272691c929d90770ad24a57b1e097c6b225826fd2afd95ab84cc9deb0207e8"} err="failed to get container status \"b7272691c929d90770ad24a57b1e097c6b225826fd2afd95ab84cc9deb0207e8\": rpc error: code = NotFound desc = could not find container \"b7272691c929d90770ad24a57b1e097c6b225826fd2afd95ab84cc9deb0207e8\": container with ID starting with b7272691c929d90770ad24a57b1e097c6b225826fd2afd95ab84cc9deb0207e8 not found: ID does not exist" May 21 00:58:16.382443 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.382421 2549 scope.go:117] "RemoveContainer" containerID="20a39cd610e44b44cf25127fc9ad356e338e08d2514a4ec86df968bb55aced70" May 21 00:58:16.382633 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:58:16.382617 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20a39cd610e44b44cf25127fc9ad356e338e08d2514a4ec86df968bb55aced70\": container with ID starting with 20a39cd610e44b44cf25127fc9ad356e338e08d2514a4ec86df968bb55aced70 not found: ID does not exist" containerID="20a39cd610e44b44cf25127fc9ad356e338e08d2514a4ec86df968bb55aced70" May 21 00:58:16.382692 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.382640 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20a39cd610e44b44cf25127fc9ad356e338e08d2514a4ec86df968bb55aced70"} err="failed to get container status \"20a39cd610e44b44cf25127fc9ad356e338e08d2514a4ec86df968bb55aced70\": rpc error: code = NotFound desc = could not find container \"20a39cd610e44b44cf25127fc9ad356e338e08d2514a4ec86df968bb55aced70\": container with ID starting with 20a39cd610e44b44cf25127fc9ad356e338e08d2514a4ec86df968bb55aced70 not found: ID does not exist" May 21 00:58:16.382692 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.382660 2549 scope.go:117] "RemoveContainer" containerID="a7f78dca6487d7aa396c7efceadbca02dd05e03d1f674afce5f0e010d32e7319" May 21 00:58:16.382786 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.382770 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 21 00:58:16.382907 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:58:16.382889 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7f78dca6487d7aa396c7efceadbca02dd05e03d1f674afce5f0e010d32e7319\": container with ID starting with a7f78dca6487d7aa396c7efceadbca02dd05e03d1f674afce5f0e010d32e7319 not found: ID does not exist" containerID="a7f78dca6487d7aa396c7efceadbca02dd05e03d1f674afce5f0e010d32e7319" May 21 00:58:16.382946 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.382914 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7f78dca6487d7aa396c7efceadbca02dd05e03d1f674afce5f0e010d32e7319"} err="failed to get container status \"a7f78dca6487d7aa396c7efceadbca02dd05e03d1f674afce5f0e010d32e7319\": rpc error: code = NotFound desc = could not find container \"a7f78dca6487d7aa396c7efceadbca02dd05e03d1f674afce5f0e010d32e7319\": container with ID starting with a7f78dca6487d7aa396c7efceadbca02dd05e03d1f674afce5f0e010d32e7319 not found: ID does not exist" May 21 00:58:16.382946 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.382935 2549 scope.go:117] "RemoveContainer" containerID="eb4ad103f0d181e6319143adf1d9d4c6e1b684d528a0225d1ee686107c3edfd2" May 21 00:58:16.383136 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383123 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerName="kube-rbac-proxy" May 21 00:58:16.383189 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383138 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerName="kube-rbac-proxy" May 21 00:58:16.383189 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383146 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerName="kube-rbac-proxy-metric" May 21 00:58:16.383189 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383152 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerName="kube-rbac-proxy-metric" May 21 00:58:16.383189 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383159 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68d5c953-f922-4671-b7ba-5436cb108e4e" containerName="registry" May 21 00:58:16.383189 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383164 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="68d5c953-f922-4671-b7ba-5436cb108e4e" containerName="registry" May 21 00:58:16.383189 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:58:16.383162 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb4ad103f0d181e6319143adf1d9d4c6e1b684d528a0225d1ee686107c3edfd2\": container with ID starting with eb4ad103f0d181e6319143adf1d9d4c6e1b684d528a0225d1ee686107c3edfd2 not found: ID does not exist" containerID="eb4ad103f0d181e6319143adf1d9d4c6e1b684d528a0225d1ee686107c3edfd2" May 21 00:58:16.383189 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383174 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerName="kube-rbac-proxy-web" May 21 00:58:16.383189 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383179 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerName="kube-rbac-proxy-web" May 21 00:58:16.383189 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383180 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb4ad103f0d181e6319143adf1d9d4c6e1b684d528a0225d1ee686107c3edfd2"} err="failed to get container status \"eb4ad103f0d181e6319143adf1d9d4c6e1b684d528a0225d1ee686107c3edfd2\": rpc error: code = NotFound desc = could not find container \"eb4ad103f0d181e6319143adf1d9d4c6e1b684d528a0225d1ee686107c3edfd2\": container with ID starting with eb4ad103f0d181e6319143adf1d9d4c6e1b684d528a0225d1ee686107c3edfd2 not found: ID does not exist" May 21 00:58:16.383507 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383195 2549 scope.go:117] "RemoveContainer" containerID="bf736ffa394dbd7440ca95cf34a40a6c802890d3afb128e13029bc87e7ee9c10" May 21 00:58:16.383507 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383187 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerName="prom-label-proxy" May 21 00:58:16.383507 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383234 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerName="prom-label-proxy" May 21 00:58:16.383507 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383248 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerName="alertmanager" May 21 00:58:16.383507 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383254 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerName="alertmanager" May 21 00:58:16.383507 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383262 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerName="init-config-reloader" May 21 00:58:16.383507 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383267 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerName="init-config-reloader" May 21 00:58:16.383507 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383276 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerName="config-reloader" May 21 00:58:16.383507 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383281 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerName="config-reloader" May 21 00:58:16.383507 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383351 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerName="kube-rbac-proxy-metric" May 21 00:58:16.383507 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383364 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="68d5c953-f922-4671-b7ba-5436cb108e4e" containerName="registry" May 21 00:58:16.383507 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383374 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerName="alertmanager" May 21 00:58:16.383507 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383384 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerName="prom-label-proxy" May 21 00:58:16.383507 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383391 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerName="config-reloader" May 21 00:58:16.383507 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383398 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerName="kube-rbac-proxy-web" May 21 00:58:16.383507 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383406 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" containerName="kube-rbac-proxy" May 21 00:58:16.383507 ip-10-0-138-101 kubenswrapper[2549]: E0521 00:58:16.383420 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf736ffa394dbd7440ca95cf34a40a6c802890d3afb128e13029bc87e7ee9c10\": container with ID starting with bf736ffa394dbd7440ca95cf34a40a6c802890d3afb128e13029bc87e7ee9c10 not found: ID does not exist" containerID="bf736ffa394dbd7440ca95cf34a40a6c802890d3afb128e13029bc87e7ee9c10" May 21 00:58:16.383507 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383449 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf736ffa394dbd7440ca95cf34a40a6c802890d3afb128e13029bc87e7ee9c10"} err="failed to get container status \"bf736ffa394dbd7440ca95cf34a40a6c802890d3afb128e13029bc87e7ee9c10\": rpc error: code = NotFound desc = could not find container \"bf736ffa394dbd7440ca95cf34a40a6c802890d3afb128e13029bc87e7ee9c10\": container with ID starting with bf736ffa394dbd7440ca95cf34a40a6c802890d3afb128e13029bc87e7ee9c10 not found: ID does not exist" May 21 00:58:16.383507 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383471 2549 scope.go:117] "RemoveContainer" containerID="dc2ff51afa78d404e0291458d7ecce540d15c42f45cbe8bbd68f79ff3073a0fe" May 21 00:58:16.384165 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383703 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc2ff51afa78d404e0291458d7ecce540d15c42f45cbe8bbd68f79ff3073a0fe"} err="failed to get container status \"dc2ff51afa78d404e0291458d7ecce540d15c42f45cbe8bbd68f79ff3073a0fe\": rpc error: code = NotFound desc = could not find container \"dc2ff51afa78d404e0291458d7ecce540d15c42f45cbe8bbd68f79ff3073a0fe\": container with ID starting with dc2ff51afa78d404e0291458d7ecce540d15c42f45cbe8bbd68f79ff3073a0fe not found: ID does not exist" May 21 00:58:16.384165 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383718 2549 scope.go:117] "RemoveContainer" containerID="3cb0811f7885aa8054d84e85d4522ee45a8c5d88cd623bcfa3f979cb07706cd8" May 21 00:58:16.384165 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383912 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cb0811f7885aa8054d84e85d4522ee45a8c5d88cd623bcfa3f979cb07706cd8"} err="failed to get container status \"3cb0811f7885aa8054d84e85d4522ee45a8c5d88cd623bcfa3f979cb07706cd8\": rpc error: code = NotFound desc = could not find container \"3cb0811f7885aa8054d84e85d4522ee45a8c5d88cd623bcfa3f979cb07706cd8\": container with ID starting with 3cb0811f7885aa8054d84e85d4522ee45a8c5d88cd623bcfa3f979cb07706cd8 not found: ID does not exist" May 21 00:58:16.384165 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.383926 2549 scope.go:117] "RemoveContainer" containerID="b7272691c929d90770ad24a57b1e097c6b225826fd2afd95ab84cc9deb0207e8" May 21 00:58:16.384309 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.384239 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7272691c929d90770ad24a57b1e097c6b225826fd2afd95ab84cc9deb0207e8"} err="failed to get container status \"b7272691c929d90770ad24a57b1e097c6b225826fd2afd95ab84cc9deb0207e8\": rpc error: code = NotFound desc = could not find container \"b7272691c929d90770ad24a57b1e097c6b225826fd2afd95ab84cc9deb0207e8\": container with ID starting with b7272691c929d90770ad24a57b1e097c6b225826fd2afd95ab84cc9deb0207e8 not found: ID does not exist" May 21 00:58:16.384309 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.384256 2549 scope.go:117] "RemoveContainer" containerID="20a39cd610e44b44cf25127fc9ad356e338e08d2514a4ec86df968bb55aced70" May 21 00:58:16.384468 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.384452 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20a39cd610e44b44cf25127fc9ad356e338e08d2514a4ec86df968bb55aced70"} err="failed to get container status \"20a39cd610e44b44cf25127fc9ad356e338e08d2514a4ec86df968bb55aced70\": rpc error: code = NotFound desc = could not find container \"20a39cd610e44b44cf25127fc9ad356e338e08d2514a4ec86df968bb55aced70\": container with ID starting with 20a39cd610e44b44cf25127fc9ad356e338e08d2514a4ec86df968bb55aced70 not found: ID does not exist" May 21 00:58:16.384519 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.384468 2549 scope.go:117] "RemoveContainer" containerID="a7f78dca6487d7aa396c7efceadbca02dd05e03d1f674afce5f0e010d32e7319" May 21 00:58:16.384677 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.384657 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7f78dca6487d7aa396c7efceadbca02dd05e03d1f674afce5f0e010d32e7319"} err="failed to get container status \"a7f78dca6487d7aa396c7efceadbca02dd05e03d1f674afce5f0e010d32e7319\": rpc error: code = NotFound desc = could not find container \"a7f78dca6487d7aa396c7efceadbca02dd05e03d1f674afce5f0e010d32e7319\": container with ID starting with a7f78dca6487d7aa396c7efceadbca02dd05e03d1f674afce5f0e010d32e7319 not found: ID does not exist" May 21 00:58:16.384722 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.384677 2549 scope.go:117] "RemoveContainer" containerID="eb4ad103f0d181e6319143adf1d9d4c6e1b684d528a0225d1ee686107c3edfd2" May 21 00:58:16.384878 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.384860 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb4ad103f0d181e6319143adf1d9d4c6e1b684d528a0225d1ee686107c3edfd2"} err="failed to get container status \"eb4ad103f0d181e6319143adf1d9d4c6e1b684d528a0225d1ee686107c3edfd2\": rpc error: code = NotFound desc = could not find container \"eb4ad103f0d181e6319143adf1d9d4c6e1b684d528a0225d1ee686107c3edfd2\": container with ID starting with eb4ad103f0d181e6319143adf1d9d4c6e1b684d528a0225d1ee686107c3edfd2 not found: ID does not exist" May 21 00:58:16.384945 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.384879 2549 scope.go:117] "RemoveContainer" containerID="bf736ffa394dbd7440ca95cf34a40a6c802890d3afb128e13029bc87e7ee9c10" May 21 00:58:16.385118 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.385100 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf736ffa394dbd7440ca95cf34a40a6c802890d3afb128e13029bc87e7ee9c10"} err="failed to get container status \"bf736ffa394dbd7440ca95cf34a40a6c802890d3afb128e13029bc87e7ee9c10\": rpc error: code = NotFound desc = could not find container \"bf736ffa394dbd7440ca95cf34a40a6c802890d3afb128e13029bc87e7ee9c10\": container with ID starting with bf736ffa394dbd7440ca95cf34a40a6c802890d3afb128e13029bc87e7ee9c10 not found: ID does not exist" May 21 00:58:16.387348 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.387334 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.389697 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.389680 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" May 21 00:58:16.389697 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.389690 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" May 21 00:58:16.389832 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.389681 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" May 21 00:58:16.389832 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.389681 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" May 21 00:58:16.390071 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.390044 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" May 21 00:58:16.390135 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.390087 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" May 21 00:58:16.390183 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.390165 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-jbtml\"" May 21 00:58:16.390218 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.390193 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" May 21 00:58:16.390391 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.390377 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" May 21 00:58:16.395371 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.395354 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" May 21 00:58:16.398085 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.398041 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 21 00:58:16.436964 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.436941 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.437055 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.436972 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.437055 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.436993 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.437055 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.437038 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.437207 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.437088 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.437207 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.437109 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26tdg\" (UniqueName: \"kubernetes.io/projected/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-kube-api-access-26tdg\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.437207 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.437151 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-config-volume\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.437207 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.437166 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-config-out\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.437207 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.437196 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.437384 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.437221 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.437384 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.437247 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.437384 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.437270 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-web-config\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.437384 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.437304 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.537941 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.537920 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.538035 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.537948 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-web-config\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.538035 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.537971 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.538035 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.537993 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.538212 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.538103 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.538212 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.538137 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.538212 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.538158 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.538212 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.538174 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.538212 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.538201 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-26tdg\" (UniqueName: \"kubernetes.io/projected/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-kube-api-access-26tdg\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.538437 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.538246 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-config-volume\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.538437 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.538269 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-config-out\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.538437 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.538313 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.538437 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.538344 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.538899 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.538875 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.538972 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.538885 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.539234 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.539212 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.541108 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.541085 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-web-config\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.541208 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.541116 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.541208 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.541170 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.541328 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.541232 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.541395 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.541370 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.541395 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.541390 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-config-out\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.541593 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.541528 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.542034 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.542017 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.543001 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.542983 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-config-volume\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.545812 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.545789 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26tdg\" (UniqueName: \"kubernetes.io/projected/ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f-kube-api-access-26tdg\") pod \"alertmanager-main-0\" (UID: \"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f\") " pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.697496 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.697476 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" May 21 00:58:16.821144 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:16.821121 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 21 00:58:16.823804 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:58:16.823778 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podada28e6c_5ee6_4f94_8e9b_63c1c527ab5f.slice/crio-0facc747f1ecc280e5e7ae3bf30c0f2e83735b775a3f091f87eb0c0dcbc6eab0 WatchSource:0}: Error finding container 0facc747f1ecc280e5e7ae3bf30c0f2e83735b775a3f091f87eb0c0dcbc6eab0: Status 404 returned error can't find the container with id 0facc747f1ecc280e5e7ae3bf30c0f2e83735b775a3f091f87eb0c0dcbc6eab0 May 21 00:58:17.335047 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:17.335011 2549 generic.go:358] "Generic (PLEG): container finished" podID="ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f" containerID="f5cec1ba2c6cfba4c4b9b500e0f97b2ba4a1b3bacc0569fe38d9081ce0d96a1b" exitCode=0 May 21 00:58:17.335530 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:17.335054 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f","Type":"ContainerDied","Data":"f5cec1ba2c6cfba4c4b9b500e0f97b2ba4a1b3bacc0569fe38d9081ce0d96a1b"} May 21 00:58:17.335530 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:17.335097 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f","Type":"ContainerStarted","Data":"0facc747f1ecc280e5e7ae3bf30c0f2e83735b775a3f091f87eb0c0dcbc6eab0"} May 21 00:58:17.676124 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:17.672502 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="194459e3-e611-4eb3-adfe-61bec9c5f2c9" path="/var/lib/kubelet/pods/194459e3-e611-4eb3-adfe-61bec9c5f2c9/volumes" May 21 00:58:18.340457 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:18.340412 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f","Type":"ContainerStarted","Data":"8132238f50641b76347969784ca5dd7faeb09ac80f011614323ebc3469112b04"} May 21 00:58:18.340457 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:18.340457 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f","Type":"ContainerStarted","Data":"6048a9700a6480813a6caac2b611251d9958cc2476f86c0b7d02217f9835c461"} May 21 00:58:18.340851 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:18.340473 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f","Type":"ContainerStarted","Data":"2eec546ad731b2d1bd3f14eac0ed423f3a019655cef69f52a5a07f5597f04806"} May 21 00:58:18.340851 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:18.340487 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f","Type":"ContainerStarted","Data":"e9500f02b0a5f3b4e7616f5cd1f5257aa6ff51df507a8268e585679d6b82fb98"} May 21 00:58:18.340851 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:18.340500 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f","Type":"ContainerStarted","Data":"430fe4a48857362dfae83932046649f959ad2fc4629cbe4cc52a4cc60513fe98"} May 21 00:58:18.340851 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:18.340511 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f","Type":"ContainerStarted","Data":"2e0d117e5ad4a299a8568488cebb0a25e6d7f5651437e5fac76b1c9424b21692"} May 21 00:58:18.367021 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:18.366961 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.366945105 podStartE2EDuration="2.366945105s" podCreationTimestamp="2026-05-21 00:58:16 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 00:58:18.364628129 +0000 UTC m=+217.261573451" watchObservedRunningTime="2026-05-21 00:58:18.366945105 +0000 UTC m=+217.263890418" May 21 00:58:18.861586 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:18.861554 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6f9975d58-ddfdp"] May 21 00:58:18.864319 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:18.864303 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" May 21 00:58:18.866587 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:18.866565 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" May 21 00:58:18.866726 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:18.866707 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" May 21 00:58:18.866790 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:18.866749 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-cjd6l\"" May 21 00:58:18.866942 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:18.866914 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" May 21 00:58:18.866942 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:18.866931 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" May 21 00:58:18.867146 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:18.866974 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" May 21 00:58:18.870779 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:18.870755 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" May 21 00:58:18.874149 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:18.874131 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6f9975d58-ddfdp"] May 21 00:58:18.959803 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:18.959783 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5402b5c0-1943-46b8-949f-009634baffb3-metrics-client-ca\") pod \"telemeter-client-6f9975d58-ddfdp\" (UID: \"5402b5c0-1943-46b8-949f-009634baffb3\") " pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" May 21 00:58:18.959903 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:18.959824 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5402b5c0-1943-46b8-949f-009634baffb3-serving-certs-ca-bundle\") pod \"telemeter-client-6f9975d58-ddfdp\" (UID: \"5402b5c0-1943-46b8-949f-009634baffb3\") " pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" May 21 00:58:18.959903 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:18.959844 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5402b5c0-1943-46b8-949f-009634baffb3-federate-client-tls\") pod \"telemeter-client-6f9975d58-ddfdp\" (UID: \"5402b5c0-1943-46b8-949f-009634baffb3\") " pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" May 21 00:58:18.959903 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:18.959869 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5402b5c0-1943-46b8-949f-009634baffb3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6f9975d58-ddfdp\" (UID: \"5402b5c0-1943-46b8-949f-009634baffb3\") " pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" May 21 00:58:18.959903 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:18.959888 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5402b5c0-1943-46b8-949f-009634baffb3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6f9975d58-ddfdp\" (UID: \"5402b5c0-1943-46b8-949f-009634baffb3\") " pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" May 21 00:58:18.960031 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:18.959917 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn9jc\" (UniqueName: \"kubernetes.io/projected/5402b5c0-1943-46b8-949f-009634baffb3-kube-api-access-mn9jc\") pod \"telemeter-client-6f9975d58-ddfdp\" (UID: \"5402b5c0-1943-46b8-949f-009634baffb3\") " pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" May 21 00:58:18.960031 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:18.959943 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5402b5c0-1943-46b8-949f-009634baffb3-telemeter-client-tls\") pod \"telemeter-client-6f9975d58-ddfdp\" (UID: \"5402b5c0-1943-46b8-949f-009634baffb3\") " pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" May 21 00:58:18.960031 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:18.959978 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5402b5c0-1943-46b8-949f-009634baffb3-secret-telemeter-client\") pod \"telemeter-client-6f9975d58-ddfdp\" (UID: \"5402b5c0-1943-46b8-949f-009634baffb3\") " pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" May 21 00:58:19.060514 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.060482 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mn9jc\" (UniqueName: \"kubernetes.io/projected/5402b5c0-1943-46b8-949f-009634baffb3-kube-api-access-mn9jc\") pod \"telemeter-client-6f9975d58-ddfdp\" (UID: \"5402b5c0-1943-46b8-949f-009634baffb3\") " pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" May 21 00:58:19.060599 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.060528 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5402b5c0-1943-46b8-949f-009634baffb3-telemeter-client-tls\") pod \"telemeter-client-6f9975d58-ddfdp\" (UID: \"5402b5c0-1943-46b8-949f-009634baffb3\") " pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" May 21 00:58:19.060599 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.060554 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5402b5c0-1943-46b8-949f-009634baffb3-secret-telemeter-client\") pod \"telemeter-client-6f9975d58-ddfdp\" (UID: \"5402b5c0-1943-46b8-949f-009634baffb3\") " pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" May 21 00:58:19.060599 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.060589 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5402b5c0-1943-46b8-949f-009634baffb3-metrics-client-ca\") pod \"telemeter-client-6f9975d58-ddfdp\" (UID: \"5402b5c0-1943-46b8-949f-009634baffb3\") " pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" May 21 00:58:19.060721 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.060640 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5402b5c0-1943-46b8-949f-009634baffb3-serving-certs-ca-bundle\") pod \"telemeter-client-6f9975d58-ddfdp\" (UID: \"5402b5c0-1943-46b8-949f-009634baffb3\") " pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" May 21 00:58:19.060721 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.060662 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5402b5c0-1943-46b8-949f-009634baffb3-federate-client-tls\") pod \"telemeter-client-6f9975d58-ddfdp\" (UID: \"5402b5c0-1943-46b8-949f-009634baffb3\") " pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" May 21 00:58:19.060721 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.060691 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5402b5c0-1943-46b8-949f-009634baffb3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6f9975d58-ddfdp\" (UID: \"5402b5c0-1943-46b8-949f-009634baffb3\") " pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" May 21 00:58:19.060858 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.060723 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5402b5c0-1943-46b8-949f-009634baffb3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6f9975d58-ddfdp\" (UID: \"5402b5c0-1943-46b8-949f-009634baffb3\") " pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" May 21 00:58:19.061439 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.061416 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5402b5c0-1943-46b8-949f-009634baffb3-metrics-client-ca\") pod \"telemeter-client-6f9975d58-ddfdp\" (UID: \"5402b5c0-1943-46b8-949f-009634baffb3\") " pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" May 21 00:58:19.061439 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.061430 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5402b5c0-1943-46b8-949f-009634baffb3-serving-certs-ca-bundle\") pod \"telemeter-client-6f9975d58-ddfdp\" (UID: \"5402b5c0-1943-46b8-949f-009634baffb3\") " pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" May 21 00:58:19.061614 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.061594 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5402b5c0-1943-46b8-949f-009634baffb3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6f9975d58-ddfdp\" (UID: \"5402b5c0-1943-46b8-949f-009634baffb3\") " pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" May 21 00:58:19.063099 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.063055 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5402b5c0-1943-46b8-949f-009634baffb3-federate-client-tls\") pod \"telemeter-client-6f9975d58-ddfdp\" (UID: \"5402b5c0-1943-46b8-949f-009634baffb3\") " pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" May 21 00:58:19.063177 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.063121 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5402b5c0-1943-46b8-949f-009634baffb3-telemeter-client-tls\") pod \"telemeter-client-6f9975d58-ddfdp\" (UID: \"5402b5c0-1943-46b8-949f-009634baffb3\") " pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" May 21 00:58:19.063388 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.063371 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5402b5c0-1943-46b8-949f-009634baffb3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6f9975d58-ddfdp\" (UID: \"5402b5c0-1943-46b8-949f-009634baffb3\") " pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" May 21 00:58:19.063576 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.063555 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5402b5c0-1943-46b8-949f-009634baffb3-secret-telemeter-client\") pod \"telemeter-client-6f9975d58-ddfdp\" (UID: \"5402b5c0-1943-46b8-949f-009634baffb3\") " pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" May 21 00:58:19.067902 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.067882 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn9jc\" (UniqueName: \"kubernetes.io/projected/5402b5c0-1943-46b8-949f-009634baffb3-kube-api-access-mn9jc\") pod \"telemeter-client-6f9975d58-ddfdp\" (UID: \"5402b5c0-1943-46b8-949f-009634baffb3\") " pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" May 21 00:58:19.165416 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.165362 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 21 00:58:19.165790 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.165767 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerName="prometheus" containerID="cri-o://e5e25146683be19210d6c9d99a216062464b0b144ce801d09cd561e0c1a67d2d" gracePeriod=600 May 21 00:58:19.165859 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.165785 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerName="thanos-sidecar" containerID="cri-o://c57e36c51bf23d8653609cccd55d866d0253ea34d64617cfe99de35295e0bf4f" gracePeriod=600 May 21 00:58:19.165859 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.165807 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerName="kube-rbac-proxy-web" containerID="cri-o://c0ea5f184f696d6e218a393b6bad93c3a47932343a6f387768b03d6724cd8aff" gracePeriod=600 May 21 00:58:19.165859 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.165772 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerName="kube-rbac-proxy" containerID="cri-o://cf21519a24a3c5eff6025331e7995abc101a69c2eb0ea03c1ddd53b20572de84" gracePeriod=600 May 21 00:58:19.165859 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.165825 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerName="config-reloader" containerID="cri-o://7fad54173c9cf8d73bb3b2ddcdc65958ece553b10dc89aaa20b8fdb0c6f0690a" gracePeriod=600 May 21 00:58:19.166084 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.165898 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerName="kube-rbac-proxy-thanos" containerID="cri-o://11c419f9c728201a34d0421122be6c427c6de37f48c85ad55d841a5732e10128" gracePeriod=600 May 21 00:58:19.174986 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.174363 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" May 21 00:58:19.309520 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.309492 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6f9975d58-ddfdp"] May 21 00:58:19.311102 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:58:19.311075 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5402b5c0_1943_46b8_949f_009634baffb3.slice/crio-b937a9d2dbcc5500c01abc96f52508b75f2b3d9dfe1aafe9b9993e2409f3659d WatchSource:0}: Error finding container b937a9d2dbcc5500c01abc96f52508b75f2b3d9dfe1aafe9b9993e2409f3659d: Status 404 returned error can't find the container with id b937a9d2dbcc5500c01abc96f52508b75f2b3d9dfe1aafe9b9993e2409f3659d May 21 00:58:19.346495 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.346465 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" event={"ID":"5402b5c0-1943-46b8-949f-009634baffb3","Type":"ContainerStarted","Data":"b937a9d2dbcc5500c01abc96f52508b75f2b3d9dfe1aafe9b9993e2409f3659d"} May 21 00:58:19.349489 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.349467 2549 generic.go:358] "Generic (PLEG): container finished" podID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerID="11c419f9c728201a34d0421122be6c427c6de37f48c85ad55d841a5732e10128" exitCode=0 May 21 00:58:19.349489 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.349486 2549 generic.go:358] "Generic (PLEG): container finished" podID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerID="cf21519a24a3c5eff6025331e7995abc101a69c2eb0ea03c1ddd53b20572de84" exitCode=0 May 21 00:58:19.349489 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.349492 2549 generic.go:358] "Generic (PLEG): container finished" podID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerID="c0ea5f184f696d6e218a393b6bad93c3a47932343a6f387768b03d6724cd8aff" exitCode=0 May 21 00:58:19.349665 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.349498 2549 generic.go:358] "Generic (PLEG): container finished" podID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerID="c57e36c51bf23d8653609cccd55d866d0253ea34d64617cfe99de35295e0bf4f" exitCode=0 May 21 00:58:19.349665 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.349503 2549 generic.go:358] "Generic (PLEG): container finished" podID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerID="7fad54173c9cf8d73bb3b2ddcdc65958ece553b10dc89aaa20b8fdb0c6f0690a" exitCode=0 May 21 00:58:19.349665 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.349507 2549 generic.go:358] "Generic (PLEG): container finished" podID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerID="e5e25146683be19210d6c9d99a216062464b0b144ce801d09cd561e0c1a67d2d" exitCode=0 May 21 00:58:19.349665 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.349539 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880694c0-306c-452e-bdfe-854e18b4f7bc","Type":"ContainerDied","Data":"11c419f9c728201a34d0421122be6c427c6de37f48c85ad55d841a5732e10128"} May 21 00:58:19.349665 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.349580 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880694c0-306c-452e-bdfe-854e18b4f7bc","Type":"ContainerDied","Data":"cf21519a24a3c5eff6025331e7995abc101a69c2eb0ea03c1ddd53b20572de84"} May 21 00:58:19.349665 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.349597 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880694c0-306c-452e-bdfe-854e18b4f7bc","Type":"ContainerDied","Data":"c0ea5f184f696d6e218a393b6bad93c3a47932343a6f387768b03d6724cd8aff"} May 21 00:58:19.349665 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.349610 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880694c0-306c-452e-bdfe-854e18b4f7bc","Type":"ContainerDied","Data":"c57e36c51bf23d8653609cccd55d866d0253ea34d64617cfe99de35295e0bf4f"} May 21 00:58:19.349665 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.349624 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880694c0-306c-452e-bdfe-854e18b4f7bc","Type":"ContainerDied","Data":"7fad54173c9cf8d73bb3b2ddcdc65958ece553b10dc89aaa20b8fdb0c6f0690a"} May 21 00:58:19.349665 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.349637 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880694c0-306c-452e-bdfe-854e18b4f7bc","Type":"ContainerDied","Data":"e5e25146683be19210d6c9d99a216062464b0b144ce801d09cd561e0c1a67d2d"} May 21 00:58:19.401803 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.401783 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:19.464108 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.464035 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/880694c0-306c-452e-bdfe-854e18b4f7bc-config-out\") pod \"880694c0-306c-452e-bdfe-854e18b4f7bc\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " May 21 00:58:19.464225 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.464102 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/880694c0-306c-452e-bdfe-854e18b4f7bc-tls-assets\") pod \"880694c0-306c-452e-bdfe-854e18b4f7bc\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " May 21 00:58:19.464225 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.464146 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"880694c0-306c-452e-bdfe-854e18b4f7bc\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " May 21 00:58:19.464225 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.464200 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-prometheus-trusted-ca-bundle\") pod \"880694c0-306c-452e-bdfe-854e18b4f7bc\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " May 21 00:58:19.464341 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.464271 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-prometheus-k8s-rulefiles-0\") pod \"880694c0-306c-452e-bdfe-854e18b4f7bc\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " May 21 00:58:19.464341 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.464302 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-configmap-kubelet-serving-ca-bundle\") pod \"880694c0-306c-452e-bdfe-854e18b4f7bc\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " May 21 00:58:19.464341 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.464333 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-kube-rbac-proxy\") pod \"880694c0-306c-452e-bdfe-854e18b4f7bc\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " May 21 00:58:19.464494 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.464379 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4wcsn\" (UniqueName: \"kubernetes.io/projected/880694c0-306c-452e-bdfe-854e18b4f7bc-kube-api-access-4wcsn\") pod \"880694c0-306c-452e-bdfe-854e18b4f7bc\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " May 21 00:58:19.464494 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.464411 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-prometheus-k8s-tls\") pod \"880694c0-306c-452e-bdfe-854e18b4f7bc\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " May 21 00:58:19.464494 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.464448 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"880694c0-306c-452e-bdfe-854e18b4f7bc\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " May 21 00:58:19.464494 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.464478 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-thanos-prometheus-http-client-file\") pod \"880694c0-306c-452e-bdfe-854e18b4f7bc\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " May 21 00:58:19.464753 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.464516 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-web-config\") pod \"880694c0-306c-452e-bdfe-854e18b4f7bc\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " May 21 00:58:19.464753 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.464558 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-configmap-serving-certs-ca-bundle\") pod \"880694c0-306c-452e-bdfe-854e18b4f7bc\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " May 21 00:58:19.464753 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.464594 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-configmap-metrics-client-ca\") pod \"880694c0-306c-452e-bdfe-854e18b4f7bc\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " May 21 00:58:19.464753 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.464624 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/880694c0-306c-452e-bdfe-854e18b4f7bc-prometheus-k8s-db\") pod \"880694c0-306c-452e-bdfe-854e18b4f7bc\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " May 21 00:58:19.464753 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.464624 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "880694c0-306c-452e-bdfe-854e18b4f7bc" (UID: "880694c0-306c-452e-bdfe-854e18b4f7bc"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 21 00:58:19.464753 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.464653 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-metrics-client-certs\") pod \"880694c0-306c-452e-bdfe-854e18b4f7bc\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " May 21 00:58:19.464753 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.464692 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-grpc-tls\") pod \"880694c0-306c-452e-bdfe-854e18b4f7bc\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " May 21 00:58:19.464753 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.464756 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-config\") pod \"880694c0-306c-452e-bdfe-854e18b4f7bc\" (UID: \"880694c0-306c-452e-bdfe-854e18b4f7bc\") " May 21 00:58:19.465144 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.464903 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "880694c0-306c-452e-bdfe-854e18b4f7bc" (UID: "880694c0-306c-452e-bdfe-854e18b4f7bc"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 21 00:58:19.465424 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.465297 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "880694c0-306c-452e-bdfe-854e18b4f7bc" (UID: "880694c0-306c-452e-bdfe-854e18b4f7bc"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 21 00:58:19.465424 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.465387 2549 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:19.465424 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.465411 2549 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-prometheus-trusted-ca-bundle\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:19.465424 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.465427 2549 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:19.466214 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.465982 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "880694c0-306c-452e-bdfe-854e18b4f7bc" (UID: "880694c0-306c-452e-bdfe-854e18b4f7bc"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 21 00:58:19.466908 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.466635 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "880694c0-306c-452e-bdfe-854e18b4f7bc" (UID: "880694c0-306c-452e-bdfe-854e18b4f7bc"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 21 00:58:19.466908 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.466651 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/880694c0-306c-452e-bdfe-854e18b4f7bc-config-out" (OuterVolumeSpecName: "config-out") pod "880694c0-306c-452e-bdfe-854e18b4f7bc" (UID: "880694c0-306c-452e-bdfe-854e18b4f7bc"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 21 00:58:19.467036 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.466910 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/880694c0-306c-452e-bdfe-854e18b4f7bc-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "880694c0-306c-452e-bdfe-854e18b4f7bc" (UID: "880694c0-306c-452e-bdfe-854e18b4f7bc"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 21 00:58:19.467392 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.467138 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "880694c0-306c-452e-bdfe-854e18b4f7bc" (UID: "880694c0-306c-452e-bdfe-854e18b4f7bc"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 00:58:19.467705 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.467684 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/880694c0-306c-452e-bdfe-854e18b4f7bc-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "880694c0-306c-452e-bdfe-854e18b4f7bc" (UID: "880694c0-306c-452e-bdfe-854e18b4f7bc"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 21 00:58:19.468902 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.468466 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "880694c0-306c-452e-bdfe-854e18b4f7bc" (UID: "880694c0-306c-452e-bdfe-854e18b4f7bc"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 00:58:19.469478 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.469451 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "880694c0-306c-452e-bdfe-854e18b4f7bc" (UID: "880694c0-306c-452e-bdfe-854e18b4f7bc"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 00:58:19.469583 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.469518 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "880694c0-306c-452e-bdfe-854e18b4f7bc" (UID: "880694c0-306c-452e-bdfe-854e18b4f7bc"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 00:58:19.469882 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.469856 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "880694c0-306c-452e-bdfe-854e18b4f7bc" (UID: "880694c0-306c-452e-bdfe-854e18b4f7bc"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 00:58:19.470043 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.470000 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "880694c0-306c-452e-bdfe-854e18b4f7bc" (UID: "880694c0-306c-452e-bdfe-854e18b4f7bc"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 00:58:19.470146 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.470123 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-config" (OuterVolumeSpecName: "config") pod "880694c0-306c-452e-bdfe-854e18b4f7bc" (UID: "880694c0-306c-452e-bdfe-854e18b4f7bc"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 00:58:19.470683 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.470663 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/880694c0-306c-452e-bdfe-854e18b4f7bc-kube-api-access-4wcsn" (OuterVolumeSpecName: "kube-api-access-4wcsn") pod "880694c0-306c-452e-bdfe-854e18b4f7bc" (UID: "880694c0-306c-452e-bdfe-854e18b4f7bc"). InnerVolumeSpecName "kube-api-access-4wcsn". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 21 00:58:19.470858 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.470833 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "880694c0-306c-452e-bdfe-854e18b4f7bc" (UID: "880694c0-306c-452e-bdfe-854e18b4f7bc"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 00:58:19.478358 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.478336 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-web-config" (OuterVolumeSpecName: "web-config") pod "880694c0-306c-452e-bdfe-854e18b4f7bc" (UID: "880694c0-306c-452e-bdfe-854e18b4f7bc"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 00:58:19.566675 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.566652 2549 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-prometheus-k8s-tls\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:19.566675 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.566674 2549 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:19.566783 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.566684 2549 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-thanos-prometheus-http-client-file\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:19.566783 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.566695 2549 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-web-config\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:19.566783 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.566705 2549 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-configmap-metrics-client-ca\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:19.566783 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.566714 2549 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/880694c0-306c-452e-bdfe-854e18b4f7bc-prometheus-k8s-db\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:19.566783 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.566724 2549 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-metrics-client-certs\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:19.566783 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.566733 2549 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-grpc-tls\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:19.566783 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.566741 2549 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-config\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:19.566783 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.566750 2549 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/880694c0-306c-452e-bdfe-854e18b4f7bc-config-out\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:19.566783 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.566757 2549 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/880694c0-306c-452e-bdfe-854e18b4f7bc-tls-assets\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:19.566783 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.566765 2549 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:19.566783 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.566775 2549 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/880694c0-306c-452e-bdfe-854e18b4f7bc-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:19.566783 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.566784 2549 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/880694c0-306c-452e-bdfe-854e18b4f7bc-secret-kube-rbac-proxy\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:19.567149 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:19.566792 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wcsn\" (UniqueName: \"kubernetes.io/projected/880694c0-306c-452e-bdfe-854e18b4f7bc-kube-api-access-4wcsn\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 00:58:20.355264 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.355144 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880694c0-306c-452e-bdfe-854e18b4f7bc","Type":"ContainerDied","Data":"624118699cda12edfd61545347195d039d44cbdf91d00708c61dea103b65920e"} May 21 00:58:20.355264 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.355202 2549 scope.go:117] "RemoveContainer" containerID="11c419f9c728201a34d0421122be6c427c6de37f48c85ad55d841a5732e10128" May 21 00:58:20.355264 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.355254 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.364293 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.364275 2549 scope.go:117] "RemoveContainer" containerID="cf21519a24a3c5eff6025331e7995abc101a69c2eb0ea03c1ddd53b20572de84" May 21 00:58:20.371927 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.371868 2549 scope.go:117] "RemoveContainer" containerID="c0ea5f184f696d6e218a393b6bad93c3a47932343a6f387768b03d6724cd8aff" May 21 00:58:20.376758 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.376738 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 21 00:58:20.379811 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.379792 2549 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 21 00:58:20.380482 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.380462 2549 scope.go:117] "RemoveContainer" containerID="c57e36c51bf23d8653609cccd55d866d0253ea34d64617cfe99de35295e0bf4f" May 21 00:58:20.388016 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.388002 2549 scope.go:117] "RemoveContainer" containerID="7fad54173c9cf8d73bb3b2ddcdc65958ece553b10dc89aaa20b8fdb0c6f0690a" May 21 00:58:20.395028 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.395012 2549 scope.go:117] "RemoveContainer" containerID="e5e25146683be19210d6c9d99a216062464b0b144ce801d09cd561e0c1a67d2d" May 21 00:58:20.402973 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.402950 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 21 00:58:20.403307 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.403252 2549 scope.go:117] "RemoveContainer" containerID="11f835b1bc28aaa5a878f188a3dfeebf0bfd8b7ce44e6af3ed388750d38d32fe" May 21 00:58:20.403441 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.403425 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerName="thanos-sidecar" May 21 00:58:20.403512 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.403445 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerName="thanos-sidecar" May 21 00:58:20.403512 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.403456 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerName="kube-rbac-proxy-web" May 21 00:58:20.403512 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.403465 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerName="kube-rbac-proxy-web" May 21 00:58:20.403512 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.403486 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerName="prometheus" May 21 00:58:20.403512 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.403495 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerName="prometheus" May 21 00:58:20.403512 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.403507 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerName="kube-rbac-proxy" May 21 00:58:20.403776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.403516 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerName="kube-rbac-proxy" May 21 00:58:20.403776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.403541 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerName="config-reloader" May 21 00:58:20.403776 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.403550 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerName="config-reloader" May 21 00:58:20.403909 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.403564 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerName="init-config-reloader" May 21 00:58:20.403963 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.403912 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerName="init-config-reloader" May 21 00:58:20.403963 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.403926 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerName="kube-rbac-proxy-thanos" May 21 00:58:20.403963 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.403935 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerName="kube-rbac-proxy-thanos" May 21 00:58:20.404217 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.404018 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerName="thanos-sidecar" May 21 00:58:20.404217 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.404033 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerName="kube-rbac-proxy-thanos" May 21 00:58:20.404217 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.404045 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerName="config-reloader" May 21 00:58:20.404217 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.404073 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerName="kube-rbac-proxy" May 21 00:58:20.404217 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.404086 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerName="kube-rbac-proxy-web" May 21 00:58:20.404217 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.404098 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="880694c0-306c-452e-bdfe-854e18b4f7bc" containerName="prometheus" May 21 00:58:20.408242 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.408221 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.410792 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.410767 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" May 21 00:58:20.411092 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.411043 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" May 21 00:58:20.413771 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.413754 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" May 21 00:58:20.413771 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.413762 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" May 21 00:58:20.413923 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.413892 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3qf6fgkonbhbg\"" May 21 00:58:20.414802 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.414629 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-fgg2s\"" May 21 00:58:20.414802 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.414645 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" May 21 00:58:20.414802 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.414685 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" May 21 00:58:20.414802 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.414645 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" May 21 00:58:20.415090 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.414954 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" May 21 00:58:20.415150 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.415115 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" May 21 00:58:20.415748 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.415577 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" May 21 00:58:20.417008 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.416968 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" May 21 00:58:20.419325 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.419306 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 21 00:58:20.419901 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.419838 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" May 21 00:58:20.475752 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.475727 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ff098155-6649-42d8-a482-ba283cbc30a3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.475893 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.475771 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ff098155-6649-42d8-a482-ba283cbc30a3-web-config\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.475893 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.475794 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff098155-6649-42d8-a482-ba283cbc30a3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.475893 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.475876 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ff098155-6649-42d8-a482-ba283cbc30a3-config-out\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.476047 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.475907 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff098155-6649-42d8-a482-ba283cbc30a3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.476047 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.475946 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ff098155-6649-42d8-a482-ba283cbc30a3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.476047 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.475990 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ff098155-6649-42d8-a482-ba283cbc30a3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.476047 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.476025 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ff098155-6649-42d8-a482-ba283cbc30a3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.476260 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.476068 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ff098155-6649-42d8-a482-ba283cbc30a3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.476260 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.476110 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ff098155-6649-42d8-a482-ba283cbc30a3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.476260 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.476164 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff098155-6649-42d8-a482-ba283cbc30a3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.476260 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.476189 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmgmd\" (UniqueName: \"kubernetes.io/projected/ff098155-6649-42d8-a482-ba283cbc30a3-kube-api-access-hmgmd\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.476260 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.476237 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ff098155-6649-42d8-a482-ba283cbc30a3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.476440 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.476306 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff098155-6649-42d8-a482-ba283cbc30a3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.476440 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.476371 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ff098155-6649-42d8-a482-ba283cbc30a3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.476440 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.476419 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ff098155-6649-42d8-a482-ba283cbc30a3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.476440 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.476440 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ff098155-6649-42d8-a482-ba283cbc30a3-config\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.476578 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.476455 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ff098155-6649-42d8-a482-ba283cbc30a3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.577345 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.577315 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff098155-6649-42d8-a482-ba283cbc30a3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.577345 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.577342 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hmgmd\" (UniqueName: \"kubernetes.io/projected/ff098155-6649-42d8-a482-ba283cbc30a3-kube-api-access-hmgmd\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.577493 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.577368 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ff098155-6649-42d8-a482-ba283cbc30a3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.577651 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.577632 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff098155-6649-42d8-a482-ba283cbc30a3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.577704 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.577676 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ff098155-6649-42d8-a482-ba283cbc30a3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.577754 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.577729 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ff098155-6649-42d8-a482-ba283cbc30a3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.577802 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.577753 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ff098155-6649-42d8-a482-ba283cbc30a3-config\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.577802 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.577779 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ff098155-6649-42d8-a482-ba283cbc30a3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.577902 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.577807 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ff098155-6649-42d8-a482-ba283cbc30a3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.577902 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.577837 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ff098155-6649-42d8-a482-ba283cbc30a3-web-config\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.577902 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.577862 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff098155-6649-42d8-a482-ba283cbc30a3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.577902 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.577895 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ff098155-6649-42d8-a482-ba283cbc30a3-config-out\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.579386 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.577924 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff098155-6649-42d8-a482-ba283cbc30a3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.579386 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.577954 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ff098155-6649-42d8-a482-ba283cbc30a3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.579386 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.577978 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ff098155-6649-42d8-a482-ba283cbc30a3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.579386 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.578003 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ff098155-6649-42d8-a482-ba283cbc30a3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.579386 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.578028 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ff098155-6649-42d8-a482-ba283cbc30a3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.579386 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.578072 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ff098155-6649-42d8-a482-ba283cbc30a3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.579386 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.578298 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff098155-6649-42d8-a482-ba283cbc30a3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.579386 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.578335 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff098155-6649-42d8-a482-ba283cbc30a3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.579386 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.578956 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff098155-6649-42d8-a482-ba283cbc30a3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.581075 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.581035 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ff098155-6649-42d8-a482-ba283cbc30a3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.581426 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.581338 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ff098155-6649-42d8-a482-ba283cbc30a3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.581426 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.581377 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ff098155-6649-42d8-a482-ba283cbc30a3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.581592 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.581499 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ff098155-6649-42d8-a482-ba283cbc30a3-web-config\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.582034 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.582008 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ff098155-6649-42d8-a482-ba283cbc30a3-config\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.582254 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.582188 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ff098155-6649-42d8-a482-ba283cbc30a3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.582254 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.582226 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ff098155-6649-42d8-a482-ba283cbc30a3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.582386 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.582285 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ff098155-6649-42d8-a482-ba283cbc30a3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.582815 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.582763 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ff098155-6649-42d8-a482-ba283cbc30a3-config-out\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.583363 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.582994 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff098155-6649-42d8-a482-ba283cbc30a3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.583879 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.583860 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ff098155-6649-42d8-a482-ba283cbc30a3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.584090 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.584047 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ff098155-6649-42d8-a482-ba283cbc30a3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.584968 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.584949 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ff098155-6649-42d8-a482-ba283cbc30a3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.585654 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.585626 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ff098155-6649-42d8-a482-ba283cbc30a3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.585808 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.585793 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmgmd\" (UniqueName: \"kubernetes.io/projected/ff098155-6649-42d8-a482-ba283cbc30a3-kube-api-access-hmgmd\") pod \"prometheus-k8s-0\" (UID: \"ff098155-6649-42d8-a482-ba283cbc30a3\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:20.723379 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:20.723299 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" May 21 00:58:21.200410 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:21.200385 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 21 00:58:21.202217 ip-10-0-138-101 kubenswrapper[2549]: W0521 00:58:21.202190 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff098155_6649_42d8_a482_ba283cbc30a3.slice/crio-92f02af79891bb666c5c4da0133ccc59ca1305a7d0d95ce221e71ddbc44ccf6e WatchSource:0}: Error finding container 92f02af79891bb666c5c4da0133ccc59ca1305a7d0d95ce221e71ddbc44ccf6e: Status 404 returned error can't find the container with id 92f02af79891bb666c5c4da0133ccc59ca1305a7d0d95ce221e71ddbc44ccf6e May 21 00:58:21.358696 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:21.358623 2549 generic.go:358] "Generic (PLEG): container finished" podID="ff098155-6649-42d8-a482-ba283cbc30a3" containerID="f37ecfc58b144885021d9578c473ce457617a552ef5bd52e1281b34607fcbb60" exitCode=0 May 21 00:58:21.359052 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:21.358708 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ff098155-6649-42d8-a482-ba283cbc30a3","Type":"ContainerDied","Data":"f37ecfc58b144885021d9578c473ce457617a552ef5bd52e1281b34607fcbb60"} May 21 00:58:21.359052 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:21.358752 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ff098155-6649-42d8-a482-ba283cbc30a3","Type":"ContainerStarted","Data":"92f02af79891bb666c5c4da0133ccc59ca1305a7d0d95ce221e71ddbc44ccf6e"} May 21 00:58:21.361284 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:21.361261 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" event={"ID":"5402b5c0-1943-46b8-949f-009634baffb3","Type":"ContainerStarted","Data":"af1fc3e6821eb468e49b5c4ed592ddfc5eee4df153854476a98b763dafc8913f"} May 21 00:58:21.361378 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:21.361288 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" event={"ID":"5402b5c0-1943-46b8-949f-009634baffb3","Type":"ContainerStarted","Data":"9da9b75a2ab8da42c0b215b56320428065958532288282173bbcc87552f0e918"} May 21 00:58:21.361378 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:21.361297 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" event={"ID":"5402b5c0-1943-46b8-949f-009634baffb3","Type":"ContainerStarted","Data":"563e335a8c24b3f867bb342598d0a9959d9d10e8318ea0f0a01fe7b316f7c63f"} May 21 00:58:21.399400 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:21.399361 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6f9975d58-ddfdp" podStartSLOduration=1.6351355459999999 podStartE2EDuration="3.399347715s" podCreationTimestamp="2026-05-21 00:58:18 +0000 UTC" firstStartedPulling="2026-05-21 00:58:19.312833882 +0000 UTC m=+218.209779172" lastFinishedPulling="2026-05-21 00:58:21.077046044 +0000 UTC m=+219.973991341" observedRunningTime="2026-05-21 00:58:21.397822509 +0000 UTC m=+220.294767820" watchObservedRunningTime="2026-05-21 00:58:21.399347715 +0000 UTC m=+220.296293026" May 21 00:58:21.672706 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:21.672680 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="880694c0-306c-452e-bdfe-854e18b4f7bc" path="/var/lib/kubelet/pods/880694c0-306c-452e-bdfe-854e18b4f7bc/volumes" May 21 00:58:22.366914 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:22.366877 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ff098155-6649-42d8-a482-ba283cbc30a3","Type":"ContainerStarted","Data":"6b084f441929bb019e14942c1123a834027b8cd4ed4c44fe1f859b4b723c4b96"} May 21 00:58:22.366914 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:22.366918 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ff098155-6649-42d8-a482-ba283cbc30a3","Type":"ContainerStarted","Data":"37dc82c83e50ddaf59fbf9b9ae7ecf09213b7e3aea3c56bcb4588b388c53c057"} May 21 00:58:22.367368 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:22.366934 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ff098155-6649-42d8-a482-ba283cbc30a3","Type":"ContainerStarted","Data":"8b6c206acea31969b7b711904ddf51d85d33c25e2442e61ef999c97ee1f64034"} May 21 00:58:22.367368 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:22.366944 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ff098155-6649-42d8-a482-ba283cbc30a3","Type":"ContainerStarted","Data":"f30cc1b43f3ec393eca07f1294bcd1db268b476f0d8635167cc41ec0fd66d890"} May 21 00:58:22.367368 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:22.366958 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ff098155-6649-42d8-a482-ba283cbc30a3","Type":"ContainerStarted","Data":"07cc2a34171d3839dfa54ebdc37c8c711785b2419f921c05ad47b7e363674664"} May 21 00:58:22.367368 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:22.366969 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ff098155-6649-42d8-a482-ba283cbc30a3","Type":"ContainerStarted","Data":"9ae64dedd9f387f15801cdffe3f210dcee0be39d7bcb3acc1614e399531f98fc"} May 21 00:58:22.392085 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:22.391992 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.391974152 podStartE2EDuration="2.391974152s" podCreationTimestamp="2026-05-21 00:58:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 00:58:22.39066425 +0000 UTC m=+221.287609561" watchObservedRunningTime="2026-05-21 00:58:22.391974152 +0000 UTC m=+221.288919463" May 21 00:58:25.724351 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:58:25.724321 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" May 21 00:59:20.723910 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:59:20.723871 2549 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" May 21 00:59:20.739139 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:59:20.739114 2549 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" May 21 00:59:21.556692 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:59:21.556665 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" May 21 00:59:41.595124 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:59:41.595099 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgxp9_12a9408a-b799-4b3c-a790-a4d6e6d67cbd/ovn-acl-logging/0.log" May 21 00:59:41.596298 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:59:41.596274 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgxp9_12a9408a-b799-4b3c-a790-a4d6e6d67cbd/ovn-acl-logging/0.log" May 21 00:59:41.598632 ip-10-0-138-101 kubenswrapper[2549]: I0521 00:59:41.598615 2549 kubelet.go:1628] "Image garbage collection succeeded" May 21 01:00:48.456851 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:00:48.456817 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-th9tc"] May 21 01:00:48.459407 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:00:48.459386 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-th9tc" May 21 01:00:48.461668 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:00:48.461645 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" May 21 01:00:48.461856 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:00:48.461843 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" May 21 01:00:48.462647 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:00:48.462632 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-hq8rp\"" May 21 01:00:48.465445 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:00:48.465426 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-th9tc"] May 21 01:00:48.534021 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:00:48.533993 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a8d6d9c8-e902-4743-8dda-26a1240efd15-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-th9tc\" (UID: \"a8d6d9c8-e902-4743-8dda-26a1240efd15\") " pod="cert-manager/cert-manager-cainjector-68b757865b-th9tc" May 21 01:00:48.534139 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:00:48.534049 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnkkz\" (UniqueName: \"kubernetes.io/projected/a8d6d9c8-e902-4743-8dda-26a1240efd15-kube-api-access-hnkkz\") pod \"cert-manager-cainjector-68b757865b-th9tc\" (UID: \"a8d6d9c8-e902-4743-8dda-26a1240efd15\") " pod="cert-manager/cert-manager-cainjector-68b757865b-th9tc" May 21 01:00:48.635430 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:00:48.635405 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hnkkz\" (UniqueName: \"kubernetes.io/projected/a8d6d9c8-e902-4743-8dda-26a1240efd15-kube-api-access-hnkkz\") pod \"cert-manager-cainjector-68b757865b-th9tc\" (UID: \"a8d6d9c8-e902-4743-8dda-26a1240efd15\") " pod="cert-manager/cert-manager-cainjector-68b757865b-th9tc" May 21 01:00:48.635541 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:00:48.635462 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a8d6d9c8-e902-4743-8dda-26a1240efd15-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-th9tc\" (UID: \"a8d6d9c8-e902-4743-8dda-26a1240efd15\") " pod="cert-manager/cert-manager-cainjector-68b757865b-th9tc" May 21 01:00:48.642940 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:00:48.642912 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a8d6d9c8-e902-4743-8dda-26a1240efd15-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-th9tc\" (UID: \"a8d6d9c8-e902-4743-8dda-26a1240efd15\") " pod="cert-manager/cert-manager-cainjector-68b757865b-th9tc" May 21 01:00:48.643022 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:00:48.642917 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnkkz\" (UniqueName: \"kubernetes.io/projected/a8d6d9c8-e902-4743-8dda-26a1240efd15-kube-api-access-hnkkz\") pod \"cert-manager-cainjector-68b757865b-th9tc\" (UID: \"a8d6d9c8-e902-4743-8dda-26a1240efd15\") " pod="cert-manager/cert-manager-cainjector-68b757865b-th9tc" May 21 01:00:48.787886 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:00:48.787866 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-th9tc" May 21 01:00:48.910826 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:00:48.910804 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-th9tc"] May 21 01:00:48.913728 ip-10-0-138-101 kubenswrapper[2549]: W0521 01:00:48.913702 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda8d6d9c8_e902_4743_8dda_26a1240efd15.slice/crio-5cfffa9f9348042266fdf2a6c61750dab81eb27b86c9352b093d4b81c4543076 WatchSource:0}: Error finding container 5cfffa9f9348042266fdf2a6c61750dab81eb27b86c9352b093d4b81c4543076: Status 404 returned error can't find the container with id 5cfffa9f9348042266fdf2a6c61750dab81eb27b86c9352b093d4b81c4543076 May 21 01:00:48.915879 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:00:48.915864 2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 21 01:00:49.792504 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:00:49.792474 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-th9tc" event={"ID":"a8d6d9c8-e902-4743-8dda-26a1240efd15","Type":"ContainerStarted","Data":"5cfffa9f9348042266fdf2a6c61750dab81eb27b86c9352b093d4b81c4543076"} May 21 01:00:53.811410 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:00:53.811375 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-th9tc" event={"ID":"a8d6d9c8-e902-4743-8dda-26a1240efd15","Type":"ContainerStarted","Data":"a6263737dcea5361139d11cd6fb0f22247a4127d9c8f90865472af7c7bda854d"} May 21 01:00:53.825233 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:00:53.825193 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-68b757865b-th9tc" podStartSLOduration=1.801902195 podStartE2EDuration="5.825177545s" podCreationTimestamp="2026-05-21 01:00:48 +0000 UTC" firstStartedPulling="2026-05-21 01:00:48.915993752 +0000 UTC m=+367.812939040" lastFinishedPulling="2026-05-21 01:00:52.939269096 +0000 UTC m=+371.836214390" observedRunningTime="2026-05-21 01:00:53.824574318 +0000 UTC m=+372.721519630" watchObservedRunningTime="2026-05-21 01:00:53.825177545 +0000 UTC m=+372.722122856" May 21 01:01:22.890686 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:22.890655 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6575999789-q7qn8"] May 21 01:01:22.895689 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:22.895665 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6575999789-q7qn8" May 21 01:01:22.898051 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:22.898029 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" May 21 01:01:22.898187 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:22.898083 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" May 21 01:01:22.898246 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:22.898201 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" May 21 01:01:22.898606 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:22.898590 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-rq7xx\"" May 21 01:01:22.898662 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:22.898619 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" May 21 01:01:22.912363 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:22.912343 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6575999789-q7qn8"] May 21 01:01:22.994880 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:22.994859 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/10b7eb4e-4524-4504-ad24-f5028c8a865c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6575999789-q7qn8\" (UID: \"10b7eb4e-4524-4504-ad24-f5028c8a865c\") " pod="opendatahub/opendatahub-operator-controller-manager-6575999789-q7qn8" May 21 01:01:22.995013 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:22.994898 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10b7eb4e-4524-4504-ad24-f5028c8a865c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6575999789-q7qn8\" (UID: \"10b7eb4e-4524-4504-ad24-f5028c8a865c\") " pod="opendatahub/opendatahub-operator-controller-manager-6575999789-q7qn8" May 21 01:01:22.995013 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:22.994992 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5b2b\" (UniqueName: \"kubernetes.io/projected/10b7eb4e-4524-4504-ad24-f5028c8a865c-kube-api-access-m5b2b\") pod \"opendatahub-operator-controller-manager-6575999789-q7qn8\" (UID: \"10b7eb4e-4524-4504-ad24-f5028c8a865c\") " pod="opendatahub/opendatahub-operator-controller-manager-6575999789-q7qn8" May 21 01:01:23.096440 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:23.096416 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/10b7eb4e-4524-4504-ad24-f5028c8a865c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6575999789-q7qn8\" (UID: \"10b7eb4e-4524-4504-ad24-f5028c8a865c\") " pod="opendatahub/opendatahub-operator-controller-manager-6575999789-q7qn8" May 21 01:01:23.096575 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:23.096449 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10b7eb4e-4524-4504-ad24-f5028c8a865c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6575999789-q7qn8\" (UID: \"10b7eb4e-4524-4504-ad24-f5028c8a865c\") " pod="opendatahub/opendatahub-operator-controller-manager-6575999789-q7qn8" May 21 01:01:23.096575 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:23.096490 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m5b2b\" (UniqueName: \"kubernetes.io/projected/10b7eb4e-4524-4504-ad24-f5028c8a865c-kube-api-access-m5b2b\") pod \"opendatahub-operator-controller-manager-6575999789-q7qn8\" (UID: \"10b7eb4e-4524-4504-ad24-f5028c8a865c\") " pod="opendatahub/opendatahub-operator-controller-manager-6575999789-q7qn8" May 21 01:01:23.099024 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:23.098999 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/10b7eb4e-4524-4504-ad24-f5028c8a865c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6575999789-q7qn8\" (UID: \"10b7eb4e-4524-4504-ad24-f5028c8a865c\") " pod="opendatahub/opendatahub-operator-controller-manager-6575999789-q7qn8" May 21 01:01:23.099155 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:23.099053 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10b7eb4e-4524-4504-ad24-f5028c8a865c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6575999789-q7qn8\" (UID: \"10b7eb4e-4524-4504-ad24-f5028c8a865c\") " pod="opendatahub/opendatahub-operator-controller-manager-6575999789-q7qn8" May 21 01:01:23.104937 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:23.104915 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5b2b\" (UniqueName: \"kubernetes.io/projected/10b7eb4e-4524-4504-ad24-f5028c8a865c-kube-api-access-m5b2b\") pod \"opendatahub-operator-controller-manager-6575999789-q7qn8\" (UID: \"10b7eb4e-4524-4504-ad24-f5028c8a865c\") " pod="opendatahub/opendatahub-operator-controller-manager-6575999789-q7qn8" May 21 01:01:23.206182 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:23.206113 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6575999789-q7qn8" May 21 01:01:23.328791 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:23.328757 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6575999789-q7qn8"] May 21 01:01:23.333642 ip-10-0-138-101 kubenswrapper[2549]: W0521 01:01:23.333612 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10b7eb4e_4524_4504_ad24_f5028c8a865c.slice/crio-a715674561b1683c268a2b866d769c64afe8244ffedbd782f8bc1ff69a3e7995 WatchSource:0}: Error finding container a715674561b1683c268a2b866d769c64afe8244ffedbd782f8bc1ff69a3e7995: Status 404 returned error can't find the container with id a715674561b1683c268a2b866d769c64afe8244ffedbd782f8bc1ff69a3e7995 May 21 01:01:23.898728 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:23.898678 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6575999789-q7qn8" event={"ID":"10b7eb4e-4524-4504-ad24-f5028c8a865c","Type":"ContainerStarted","Data":"a715674561b1683c268a2b866d769c64afe8244ffedbd782f8bc1ff69a3e7995"} May 21 01:01:25.907117 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:25.907071 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6575999789-q7qn8" event={"ID":"10b7eb4e-4524-4504-ad24-f5028c8a865c","Type":"ContainerStarted","Data":"ed1de9cb33430d8678a38f2c030386c2c1e8c8bcc7464590166604616c12a45b"} May 21 01:01:25.907512 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:25.907203 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6575999789-q7qn8" May 21 01:01:25.927493 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:25.927449 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6575999789-q7qn8" podStartSLOduration=1.554605641 podStartE2EDuration="3.927433512s" podCreationTimestamp="2026-05-21 01:01:22 +0000 UTC" firstStartedPulling="2026-05-21 01:01:23.335363306 +0000 UTC m=+402.232308594" lastFinishedPulling="2026-05-21 01:01:25.708191174 +0000 UTC m=+404.605136465" observedRunningTime="2026-05-21 01:01:25.925580156 +0000 UTC m=+404.822525467" watchObservedRunningTime="2026-05-21 01:01:25.927433512 +0000 UTC m=+404.824378824" May 21 01:01:36.912802 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:36.912766 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6575999789-q7qn8" May 21 01:01:40.168074 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:40.168030 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-857f4b55dc-tjqfl"] May 21 01:01:40.170582 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:40.170562 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-857f4b55dc-tjqfl" May 21 01:01:40.172869 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:40.172846 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-sf5qd\"" May 21 01:01:40.172984 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:40.172894 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" May 21 01:01:40.172984 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:40.172922 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" May 21 01:01:40.187420 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:40.187394 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-857f4b55dc-tjqfl"] May 21 01:01:40.245619 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:40.245574 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d65b98da-1e62-477a-8bd9-9f5154fe16aa-tls-certs\") pod \"kube-auth-proxy-857f4b55dc-tjqfl\" (UID: \"d65b98da-1e62-477a-8bd9-9f5154fe16aa\") " pod="openshift-ingress/kube-auth-proxy-857f4b55dc-tjqfl" May 21 01:01:40.245787 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:40.245641 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkhtl\" (UniqueName: \"kubernetes.io/projected/d65b98da-1e62-477a-8bd9-9f5154fe16aa-kube-api-access-hkhtl\") pod \"kube-auth-proxy-857f4b55dc-tjqfl\" (UID: \"d65b98da-1e62-477a-8bd9-9f5154fe16aa\") " pod="openshift-ingress/kube-auth-proxy-857f4b55dc-tjqfl" May 21 01:01:40.245787 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:40.245729 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d65b98da-1e62-477a-8bd9-9f5154fe16aa-tmp\") pod \"kube-auth-proxy-857f4b55dc-tjqfl\" (UID: \"d65b98da-1e62-477a-8bd9-9f5154fe16aa\") " pod="openshift-ingress/kube-auth-proxy-857f4b55dc-tjqfl" May 21 01:01:40.346729 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:40.346678 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d65b98da-1e62-477a-8bd9-9f5154fe16aa-tls-certs\") pod \"kube-auth-proxy-857f4b55dc-tjqfl\" (UID: \"d65b98da-1e62-477a-8bd9-9f5154fe16aa\") " pod="openshift-ingress/kube-auth-proxy-857f4b55dc-tjqfl" May 21 01:01:40.346928 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:40.346764 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hkhtl\" (UniqueName: \"kubernetes.io/projected/d65b98da-1e62-477a-8bd9-9f5154fe16aa-kube-api-access-hkhtl\") pod \"kube-auth-proxy-857f4b55dc-tjqfl\" (UID: \"d65b98da-1e62-477a-8bd9-9f5154fe16aa\") " pod="openshift-ingress/kube-auth-proxy-857f4b55dc-tjqfl" May 21 01:01:40.346928 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:40.346806 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d65b98da-1e62-477a-8bd9-9f5154fe16aa-tmp\") pod \"kube-auth-proxy-857f4b55dc-tjqfl\" (UID: \"d65b98da-1e62-477a-8bd9-9f5154fe16aa\") " pod="openshift-ingress/kube-auth-proxy-857f4b55dc-tjqfl" May 21 01:01:40.346928 ip-10-0-138-101 kubenswrapper[2549]: E0521 01:01:40.346844 2549 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found May 21 01:01:40.346928 ip-10-0-138-101 kubenswrapper[2549]: E0521 01:01:40.346919 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d65b98da-1e62-477a-8bd9-9f5154fe16aa-tls-certs podName:d65b98da-1e62-477a-8bd9-9f5154fe16aa nodeName:}" failed. No retries permitted until 2026-05-21 01:01:40.846897212 +0000 UTC m=+419.743842504 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d65b98da-1e62-477a-8bd9-9f5154fe16aa-tls-certs") pod "kube-auth-proxy-857f4b55dc-tjqfl" (UID: "d65b98da-1e62-477a-8bd9-9f5154fe16aa") : secret "kube-auth-proxy-tls" not found May 21 01:01:40.349204 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:40.349178 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d65b98da-1e62-477a-8bd9-9f5154fe16aa-tmp\") pod \"kube-auth-proxy-857f4b55dc-tjqfl\" (UID: \"d65b98da-1e62-477a-8bd9-9f5154fe16aa\") " pod="openshift-ingress/kube-auth-proxy-857f4b55dc-tjqfl" May 21 01:01:40.355415 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:40.355384 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkhtl\" (UniqueName: \"kubernetes.io/projected/d65b98da-1e62-477a-8bd9-9f5154fe16aa-kube-api-access-hkhtl\") pod \"kube-auth-proxy-857f4b55dc-tjqfl\" (UID: \"d65b98da-1e62-477a-8bd9-9f5154fe16aa\") " pod="openshift-ingress/kube-auth-proxy-857f4b55dc-tjqfl" May 21 01:01:40.851487 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:40.851452 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d65b98da-1e62-477a-8bd9-9f5154fe16aa-tls-certs\") pod \"kube-auth-proxy-857f4b55dc-tjqfl\" (UID: \"d65b98da-1e62-477a-8bd9-9f5154fe16aa\") " pod="openshift-ingress/kube-auth-proxy-857f4b55dc-tjqfl" May 21 01:01:40.853975 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:40.853942 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d65b98da-1e62-477a-8bd9-9f5154fe16aa-tls-certs\") pod \"kube-auth-proxy-857f4b55dc-tjqfl\" (UID: \"d65b98da-1e62-477a-8bd9-9f5154fe16aa\") " pod="openshift-ingress/kube-auth-proxy-857f4b55dc-tjqfl" May 21 01:01:41.080030 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:41.079996 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-857f4b55dc-tjqfl" May 21 01:01:41.199935 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:41.199872 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-857f4b55dc-tjqfl"] May 21 01:01:41.202092 ip-10-0-138-101 kubenswrapper[2549]: W0521 01:01:41.202036 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd65b98da_1e62_477a_8bd9_9f5154fe16aa.slice/crio-b6c17634f31bc227230d4017a30d162ebd30001a5a1d5cde299333161731f7f7 WatchSource:0}: Error finding container b6c17634f31bc227230d4017a30d162ebd30001a5a1d5cde299333161731f7f7: Status 404 returned error can't find the container with id b6c17634f31bc227230d4017a30d162ebd30001a5a1d5cde299333161731f7f7 May 21 01:01:41.959828 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:41.959782 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-857f4b55dc-tjqfl" event={"ID":"d65b98da-1e62-477a-8bd9-9f5154fe16aa","Type":"ContainerStarted","Data":"b6c17634f31bc227230d4017a30d162ebd30001a5a1d5cde299333161731f7f7"} May 21 01:01:43.765734 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:43.765694 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-ghjff"] May 21 01:01:43.768693 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:43.768663 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-ghjff" May 21 01:01:43.770849 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:43.770826 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" May 21 01:01:43.771045 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:43.770906 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-sfgv7\"" May 21 01:01:43.775413 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:43.775385 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-ghjff"] May 21 01:01:43.877610 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:43.877554 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/984a1d4b-7192-4318-accb-f02807829edb-cert\") pod \"odh-model-controller-858dbf95b8-ghjff\" (UID: \"984a1d4b-7192-4318-accb-f02807829edb\") " pod="opendatahub/odh-model-controller-858dbf95b8-ghjff" May 21 01:01:43.877793 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:43.877612 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42lff\" (UniqueName: \"kubernetes.io/projected/984a1d4b-7192-4318-accb-f02807829edb-kube-api-access-42lff\") pod \"odh-model-controller-858dbf95b8-ghjff\" (UID: \"984a1d4b-7192-4318-accb-f02807829edb\") " pod="opendatahub/odh-model-controller-858dbf95b8-ghjff" May 21 01:01:43.978516 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:43.978487 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/984a1d4b-7192-4318-accb-f02807829edb-cert\") pod \"odh-model-controller-858dbf95b8-ghjff\" (UID: \"984a1d4b-7192-4318-accb-f02807829edb\") " pod="opendatahub/odh-model-controller-858dbf95b8-ghjff" May 21 01:01:43.978704 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:43.978526 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-42lff\" (UniqueName: \"kubernetes.io/projected/984a1d4b-7192-4318-accb-f02807829edb-kube-api-access-42lff\") pod \"odh-model-controller-858dbf95b8-ghjff\" (UID: \"984a1d4b-7192-4318-accb-f02807829edb\") " pod="opendatahub/odh-model-controller-858dbf95b8-ghjff" May 21 01:01:43.978773 ip-10-0-138-101 kubenswrapper[2549]: E0521 01:01:43.978723 2549 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found May 21 01:01:43.978827 ip-10-0-138-101 kubenswrapper[2549]: E0521 01:01:43.978798 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/984a1d4b-7192-4318-accb-f02807829edb-cert podName:984a1d4b-7192-4318-accb-f02807829edb nodeName:}" failed. No retries permitted until 2026-05-21 01:01:44.478773685 +0000 UTC m=+423.375718983 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/984a1d4b-7192-4318-accb-f02807829edb-cert") pod "odh-model-controller-858dbf95b8-ghjff" (UID: "984a1d4b-7192-4318-accb-f02807829edb") : secret "odh-model-controller-webhook-cert" not found May 21 01:01:43.988752 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:43.988705 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42lff\" (UniqueName: \"kubernetes.io/projected/984a1d4b-7192-4318-accb-f02807829edb-kube-api-access-42lff\") pod \"odh-model-controller-858dbf95b8-ghjff\" (UID: \"984a1d4b-7192-4318-accb-f02807829edb\") " pod="opendatahub/odh-model-controller-858dbf95b8-ghjff" May 21 01:01:44.138804 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:44.138779 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" May 21 01:01:44.483408 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:44.483316 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/984a1d4b-7192-4318-accb-f02807829edb-cert\") pod \"odh-model-controller-858dbf95b8-ghjff\" (UID: \"984a1d4b-7192-4318-accb-f02807829edb\") " pod="opendatahub/odh-model-controller-858dbf95b8-ghjff" May 21 01:01:44.485718 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:44.485689 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/984a1d4b-7192-4318-accb-f02807829edb-cert\") pod \"odh-model-controller-858dbf95b8-ghjff\" (UID: \"984a1d4b-7192-4318-accb-f02807829edb\") " pod="opendatahub/odh-model-controller-858dbf95b8-ghjff" May 21 01:01:44.682484 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:44.682449 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-ghjff" May 21 01:01:44.812808 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:44.812783 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-ghjff"] May 21 01:01:44.971448 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:44.971407 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-ghjff" event={"ID":"984a1d4b-7192-4318-accb-f02807829edb","Type":"ContainerStarted","Data":"6722196e2efa7e0efbfaa7ff1f8b0546c79b2a5228569c16265bdfd30dc07837"} May 21 01:01:44.972857 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:44.972827 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-857f4b55dc-tjqfl" event={"ID":"d65b98da-1e62-477a-8bd9-9f5154fe16aa","Type":"ContainerStarted","Data":"b2be5f0f25fdb3fc2533217a222ce5ac72522b8aa6777b8cbe8dc9dfa9feee3d"} May 21 01:01:44.988634 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:44.988553 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-857f4b55dc-tjqfl" podStartSLOduration=2.056118639 podStartE2EDuration="4.988541379s" podCreationTimestamp="2026-05-21 01:01:40 +0000 UTC" firstStartedPulling="2026-05-21 01:01:41.20368167 +0000 UTC m=+420.100626958" lastFinishedPulling="2026-05-21 01:01:44.136104405 +0000 UTC m=+423.033049698" observedRunningTime="2026-05-21 01:01:44.987018729 +0000 UTC m=+423.883964041" watchObservedRunningTime="2026-05-21 01:01:44.988541379 +0000 UTC m=+423.885486690" May 21 01:01:48.989921 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:48.989893 2549 generic.go:358] "Generic (PLEG): container finished" podID="984a1d4b-7192-4318-accb-f02807829edb" containerID="8abd68ca3c2da36366ad11d3375eaf3e8ba09260f2187c1620eaf619fcaf0d48" exitCode=1 May 21 01:01:48.990295 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:48.989952 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-ghjff" event={"ID":"984a1d4b-7192-4318-accb-f02807829edb","Type":"ContainerDied","Data":"8abd68ca3c2da36366ad11d3375eaf3e8ba09260f2187c1620eaf619fcaf0d48"} May 21 01:01:48.990295 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:48.990194 2549 scope.go:117] "RemoveContainer" containerID="8abd68ca3c2da36366ad11d3375eaf3e8ba09260f2187c1620eaf619fcaf0d48" May 21 01:01:49.996646 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:49.996609 2549 generic.go:358] "Generic (PLEG): container finished" podID="984a1d4b-7192-4318-accb-f02807829edb" containerID="565ef1544059a606672d70c70ccded6a41c6f81e9967a8982cd96e096d200e87" exitCode=1 May 21 01:01:49.997089 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:49.996669 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-ghjff" event={"ID":"984a1d4b-7192-4318-accb-f02807829edb","Type":"ContainerDied","Data":"565ef1544059a606672d70c70ccded6a41c6f81e9967a8982cd96e096d200e87"} May 21 01:01:49.997089 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:49.996717 2549 scope.go:117] "RemoveContainer" containerID="8abd68ca3c2da36366ad11d3375eaf3e8ba09260f2187c1620eaf619fcaf0d48" May 21 01:01:49.997089 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:49.996938 2549 scope.go:117] "RemoveContainer" containerID="565ef1544059a606672d70c70ccded6a41c6f81e9967a8982cd96e096d200e87" May 21 01:01:49.997224 ip-10-0-138-101 kubenswrapper[2549]: E0521 01:01:49.997178 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-ghjff_opendatahub(984a1d4b-7192-4318-accb-f02807829edb)\"" pod="opendatahub/odh-model-controller-858dbf95b8-ghjff" podUID="984a1d4b-7192-4318-accb-f02807829edb" May 21 01:01:50.587112 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:50.587080 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-fjfvj"] May 21 01:01:50.590338 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:50.590318 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-fjfvj" May 21 01:01:50.592534 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:50.592519 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" May 21 01:01:50.592595 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:50.592538 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-m5xpj\"" May 21 01:01:50.601210 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:50.601188 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-fjfvj"] May 21 01:01:50.745860 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:50.745827 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c8fm\" (UniqueName: \"kubernetes.io/projected/45ed8e6a-dec5-4cf2-8b2c-b9d88aa53d0d-kube-api-access-2c8fm\") pod \"kserve-controller-manager-856948b99f-fjfvj\" (UID: \"45ed8e6a-dec5-4cf2-8b2c-b9d88aa53d0d\") " pod="opendatahub/kserve-controller-manager-856948b99f-fjfvj" May 21 01:01:50.746045 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:50.745978 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/45ed8e6a-dec5-4cf2-8b2c-b9d88aa53d0d-cert\") pod \"kserve-controller-manager-856948b99f-fjfvj\" (UID: \"45ed8e6a-dec5-4cf2-8b2c-b9d88aa53d0d\") " pod="opendatahub/kserve-controller-manager-856948b99f-fjfvj" May 21 01:01:50.847204 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:50.847107 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/45ed8e6a-dec5-4cf2-8b2c-b9d88aa53d0d-cert\") pod \"kserve-controller-manager-856948b99f-fjfvj\" (UID: \"45ed8e6a-dec5-4cf2-8b2c-b9d88aa53d0d\") " pod="opendatahub/kserve-controller-manager-856948b99f-fjfvj" May 21 01:01:50.847204 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:50.847198 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2c8fm\" (UniqueName: \"kubernetes.io/projected/45ed8e6a-dec5-4cf2-8b2c-b9d88aa53d0d-kube-api-access-2c8fm\") pod \"kserve-controller-manager-856948b99f-fjfvj\" (UID: \"45ed8e6a-dec5-4cf2-8b2c-b9d88aa53d0d\") " pod="opendatahub/kserve-controller-manager-856948b99f-fjfvj" May 21 01:01:50.847426 ip-10-0-138-101 kubenswrapper[2549]: E0521 01:01:50.847250 2549 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found May 21 01:01:50.847426 ip-10-0-138-101 kubenswrapper[2549]: E0521 01:01:50.847336 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45ed8e6a-dec5-4cf2-8b2c-b9d88aa53d0d-cert podName:45ed8e6a-dec5-4cf2-8b2c-b9d88aa53d0d nodeName:}" failed. No retries permitted until 2026-05-21 01:01:51.347311831 +0000 UTC m=+430.244257136 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/45ed8e6a-dec5-4cf2-8b2c-b9d88aa53d0d-cert") pod "kserve-controller-manager-856948b99f-fjfvj" (UID: "45ed8e6a-dec5-4cf2-8b2c-b9d88aa53d0d") : secret "kserve-webhook-server-cert" not found May 21 01:01:50.860460 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:50.860437 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c8fm\" (UniqueName: \"kubernetes.io/projected/45ed8e6a-dec5-4cf2-8b2c-b9d88aa53d0d-kube-api-access-2c8fm\") pod \"kserve-controller-manager-856948b99f-fjfvj\" (UID: \"45ed8e6a-dec5-4cf2-8b2c-b9d88aa53d0d\") " pod="opendatahub/kserve-controller-manager-856948b99f-fjfvj" May 21 01:01:51.001528 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:51.001497 2549 scope.go:117] "RemoveContainer" containerID="565ef1544059a606672d70c70ccded6a41c6f81e9967a8982cd96e096d200e87" May 21 01:01:51.001899 ip-10-0-138-101 kubenswrapper[2549]: E0521 01:01:51.001695 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-ghjff_opendatahub(984a1d4b-7192-4318-accb-f02807829edb)\"" pod="opendatahub/odh-model-controller-858dbf95b8-ghjff" podUID="984a1d4b-7192-4318-accb-f02807829edb" May 21 01:01:51.352161 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:51.352129 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/45ed8e6a-dec5-4cf2-8b2c-b9d88aa53d0d-cert\") pod \"kserve-controller-manager-856948b99f-fjfvj\" (UID: \"45ed8e6a-dec5-4cf2-8b2c-b9d88aa53d0d\") " pod="opendatahub/kserve-controller-manager-856948b99f-fjfvj" May 21 01:01:51.354488 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:51.354467 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/45ed8e6a-dec5-4cf2-8b2c-b9d88aa53d0d-cert\") pod \"kserve-controller-manager-856948b99f-fjfvj\" (UID: \"45ed8e6a-dec5-4cf2-8b2c-b9d88aa53d0d\") " pod="opendatahub/kserve-controller-manager-856948b99f-fjfvj" May 21 01:01:51.501832 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:51.501796 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-fjfvj" May 21 01:01:51.624783 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:51.624758 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-fjfvj"] May 21 01:01:51.627261 ip-10-0-138-101 kubenswrapper[2549]: W0521 01:01:51.627235 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45ed8e6a_dec5_4cf2_8b2c_b9d88aa53d0d.slice/crio-8bd217c978e948a30ad07220825bf76369bc4fa2faf266ef4d3be3f68490e627 WatchSource:0}: Error finding container 8bd217c978e948a30ad07220825bf76369bc4fa2faf266ef4d3be3f68490e627: Status 404 returned error can't find the container with id 8bd217c978e948a30ad07220825bf76369bc4fa2faf266ef4d3be3f68490e627 May 21 01:01:52.005720 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:52.005689 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-fjfvj" event={"ID":"45ed8e6a-dec5-4cf2-8b2c-b9d88aa53d0d","Type":"ContainerStarted","Data":"8bd217c978e948a30ad07220825bf76369bc4fa2faf266ef4d3be3f68490e627"} May 21 01:01:54.683541 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:54.683508 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-ghjff" May 21 01:01:54.683997 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:54.683927 2549 scope.go:117] "RemoveContainer" containerID="565ef1544059a606672d70c70ccded6a41c6f81e9967a8982cd96e096d200e87" May 21 01:01:54.684174 ip-10-0-138-101 kubenswrapper[2549]: E0521 01:01:54.684153 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-ghjff_opendatahub(984a1d4b-7192-4318-accb-f02807829edb)\"" pod="opendatahub/odh-model-controller-858dbf95b8-ghjff" podUID="984a1d4b-7192-4318-accb-f02807829edb" May 21 01:01:55.016561 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:55.016531 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-fjfvj" event={"ID":"45ed8e6a-dec5-4cf2-8b2c-b9d88aa53d0d","Type":"ContainerStarted","Data":"1e39b6eac3404f406591ab16fae8422b7ae9d5056129191013647a8f96c4408e"} May 21 01:01:55.016673 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:55.016650 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-856948b99f-fjfvj" May 21 01:01:55.034562 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:55.034399 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-856948b99f-fjfvj" podStartSLOduration=1.7123254110000001 podStartE2EDuration="5.034381086s" podCreationTimestamp="2026-05-21 01:01:50 +0000 UTC" firstStartedPulling="2026-05-21 01:01:51.628665989 +0000 UTC m=+430.525611278" lastFinishedPulling="2026-05-21 01:01:54.950721657 +0000 UTC m=+433.847666953" observedRunningTime="2026-05-21 01:01:55.034349535 +0000 UTC m=+433.931294847" watchObservedRunningTime="2026-05-21 01:01:55.034381086 +0000 UTC m=+433.931326399" May 21 01:01:55.360404 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:55.360371 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-qrwh4"] May 21 01:01:55.363103 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:55.363087 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-qrwh4" May 21 01:01:55.367966 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:55.367945 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\"" May 21 01:01:55.368092 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:55.368052 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-gcfzz\"" May 21 01:01:55.368092 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:55.368085 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\"" May 21 01:01:55.375303 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:55.375278 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-qrwh4"] May 21 01:01:55.491022 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:55.490989 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/50ae20d4-89c3-4f93-b165-2dce21979024-operator-config\") pod \"servicemesh-operator3-55f49c5f94-qrwh4\" (UID: \"50ae20d4-89c3-4f93-b165-2dce21979024\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-qrwh4" May 21 01:01:55.491201 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:55.491089 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnqvp\" (UniqueName: \"kubernetes.io/projected/50ae20d4-89c3-4f93-b165-2dce21979024-kube-api-access-hnqvp\") pod \"servicemesh-operator3-55f49c5f94-qrwh4\" (UID: \"50ae20d4-89c3-4f93-b165-2dce21979024\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-qrwh4" May 21 01:01:55.591929 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:55.591896 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hnqvp\" (UniqueName: \"kubernetes.io/projected/50ae20d4-89c3-4f93-b165-2dce21979024-kube-api-access-hnqvp\") pod \"servicemesh-operator3-55f49c5f94-qrwh4\" (UID: \"50ae20d4-89c3-4f93-b165-2dce21979024\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-qrwh4" May 21 01:01:55.591929 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:55.591969 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/50ae20d4-89c3-4f93-b165-2dce21979024-operator-config\") pod \"servicemesh-operator3-55f49c5f94-qrwh4\" (UID: \"50ae20d4-89c3-4f93-b165-2dce21979024\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-qrwh4" May 21 01:01:55.594584 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:55.594560 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/50ae20d4-89c3-4f93-b165-2dce21979024-operator-config\") pod \"servicemesh-operator3-55f49c5f94-qrwh4\" (UID: \"50ae20d4-89c3-4f93-b165-2dce21979024\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-qrwh4" May 21 01:01:55.599842 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:55.599817 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnqvp\" (UniqueName: \"kubernetes.io/projected/50ae20d4-89c3-4f93-b165-2dce21979024-kube-api-access-hnqvp\") pod \"servicemesh-operator3-55f49c5f94-qrwh4\" (UID: \"50ae20d4-89c3-4f93-b165-2dce21979024\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-qrwh4" May 21 01:01:55.677929 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:55.672210 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-qrwh4" May 21 01:01:55.815378 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:55.815351 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-qrwh4"] May 21 01:01:55.817606 ip-10-0-138-101 kubenswrapper[2549]: W0521 01:01:55.817579 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50ae20d4_89c3_4f93_b165_2dce21979024.slice/crio-582bd72ed8adbd1f300e3f97619569f1f12bd68bed061933c11c06fcb481f68d WatchSource:0}: Error finding container 582bd72ed8adbd1f300e3f97619569f1f12bd68bed061933c11c06fcb481f68d: Status 404 returned error can't find the container with id 582bd72ed8adbd1f300e3f97619569f1f12bd68bed061933c11c06fcb481f68d May 21 01:01:56.020702 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:01:56.020663 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-qrwh4" event={"ID":"50ae20d4-89c3-4f93-b165-2dce21979024","Type":"ContainerStarted","Data":"582bd72ed8adbd1f300e3f97619569f1f12bd68bed061933c11c06fcb481f68d"} May 21 01:02:01.040393 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:01.040351 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-qrwh4" event={"ID":"50ae20d4-89c3-4f93-b165-2dce21979024","Type":"ContainerStarted","Data":"525baa9ff143dcb3b4b9f02355c91715831d0bdf9044342c0259c511534d7e94"} May 21 01:02:01.040902 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:01.040488 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-qrwh4" May 21 01:02:01.060764 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:01.060704 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-55f49c5f94-qrwh4" podStartSLOduration=1.058746321 podStartE2EDuration="6.060686824s" podCreationTimestamp="2026-05-21 01:01:55 +0000 UTC" firstStartedPulling="2026-05-21 01:01:55.820033552 +0000 UTC m=+434.716978841" lastFinishedPulling="2026-05-21 01:02:00.821974046 +0000 UTC m=+439.718919344" observedRunningTime="2026-05-21 01:02:01.059029014 +0000 UTC m=+439.955974326" watchObservedRunningTime="2026-05-21 01:02:01.060686824 +0000 UTC m=+439.957632135" May 21 01:02:04.683012 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:04.682979 2549 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/odh-model-controller-858dbf95b8-ghjff" May 21 01:02:04.683399 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:04.683335 2549 scope.go:117] "RemoveContainer" containerID="565ef1544059a606672d70c70ccded6a41c6f81e9967a8982cd96e096d200e87" May 21 01:02:06.057860 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:06.057825 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-ghjff" event={"ID":"984a1d4b-7192-4318-accb-f02807829edb","Type":"ContainerStarted","Data":"80f363334106b97f627b27d80eeec5b9e324f199d63704606ae02c73a6cec963"} May 21 01:02:06.058332 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:06.058035 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-ghjff" May 21 01:02:06.073890 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:06.073843 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/odh-model-controller-858dbf95b8-ghjff" podStartSLOduration=2.810664456 podStartE2EDuration="23.073829416s" podCreationTimestamp="2026-05-21 01:01:43 +0000 UTC" firstStartedPulling="2026-05-21 01:01:44.82002609 +0000 UTC m=+423.716971393" lastFinishedPulling="2026-05-21 01:02:05.08319106 +0000 UTC m=+443.980136353" observedRunningTime="2026-05-21 01:02:06.072138941 +0000 UTC m=+444.969084253" watchObservedRunningTime="2026-05-21 01:02:06.073829416 +0000 UTC m=+444.970774726" May 21 01:02:12.009377 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.009339 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv"] May 21 01:02:12.011878 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.011857 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" May 21 01:02:12.014244 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.014220 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" May 21 01:02:12.014244 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.014234 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" May 21 01:02:12.014440 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.014242 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" May 21 01:02:12.014502 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.014486 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" May 21 01:02:12.014568 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.014553 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-hwd5z\"" May 21 01:02:12.024521 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.024500 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv"] May 21 01:02:12.044797 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.044752 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/56cee210-55b5-4da5-9766-455bb56d7cf8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-5lsxv\" (UID: \"56cee210-55b5-4da5-9766-455bb56d7cf8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" May 21 01:02:12.044936 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.044804 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/56cee210-55b5-4da5-9766-455bb56d7cf8-cacerts\") pod \"istiod-openshift-gateway-798958bb55-5lsxv\" (UID: \"56cee210-55b5-4da5-9766-455bb56d7cf8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" May 21 01:02:12.044936 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.044916 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9ms6\" (UniqueName: \"kubernetes.io/projected/56cee210-55b5-4da5-9766-455bb56d7cf8-kube-api-access-m9ms6\") pod \"istiod-openshift-gateway-798958bb55-5lsxv\" (UID: \"56cee210-55b5-4da5-9766-455bb56d7cf8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" May 21 01:02:12.045046 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.044962 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56cee210-55b5-4da5-9766-455bb56d7cf8-istio-token\") pod \"istiod-openshift-gateway-798958bb55-5lsxv\" (UID: \"56cee210-55b5-4da5-9766-455bb56d7cf8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" May 21 01:02:12.045046 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.045007 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/56cee210-55b5-4da5-9766-455bb56d7cf8-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-5lsxv\" (UID: \"56cee210-55b5-4da5-9766-455bb56d7cf8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" May 21 01:02:12.045160 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.045091 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/56cee210-55b5-4da5-9766-455bb56d7cf8-local-certs\") pod \"istiod-openshift-gateway-798958bb55-5lsxv\" (UID: \"56cee210-55b5-4da5-9766-455bb56d7cf8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" May 21 01:02:12.045160 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.045117 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/56cee210-55b5-4da5-9766-455bb56d7cf8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-5lsxv\" (UID: \"56cee210-55b5-4da5-9766-455bb56d7cf8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" May 21 01:02:12.046837 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.046814 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-qrwh4" May 21 01:02:12.146277 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.146245 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/56cee210-55b5-4da5-9766-455bb56d7cf8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-5lsxv\" (UID: \"56cee210-55b5-4da5-9766-455bb56d7cf8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" May 21 01:02:12.146474 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.146285 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/56cee210-55b5-4da5-9766-455bb56d7cf8-cacerts\") pod \"istiod-openshift-gateway-798958bb55-5lsxv\" (UID: \"56cee210-55b5-4da5-9766-455bb56d7cf8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" May 21 01:02:12.146474 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.146348 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m9ms6\" (UniqueName: \"kubernetes.io/projected/56cee210-55b5-4da5-9766-455bb56d7cf8-kube-api-access-m9ms6\") pod \"istiod-openshift-gateway-798958bb55-5lsxv\" (UID: \"56cee210-55b5-4da5-9766-455bb56d7cf8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" May 21 01:02:12.146474 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.146387 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56cee210-55b5-4da5-9766-455bb56d7cf8-istio-token\") pod \"istiod-openshift-gateway-798958bb55-5lsxv\" (UID: \"56cee210-55b5-4da5-9766-455bb56d7cf8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" May 21 01:02:12.146474 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.146421 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/56cee210-55b5-4da5-9766-455bb56d7cf8-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-5lsxv\" (UID: \"56cee210-55b5-4da5-9766-455bb56d7cf8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" May 21 01:02:12.146697 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.146493 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/56cee210-55b5-4da5-9766-455bb56d7cf8-local-certs\") pod \"istiod-openshift-gateway-798958bb55-5lsxv\" (UID: \"56cee210-55b5-4da5-9766-455bb56d7cf8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" May 21 01:02:12.146697 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.146517 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/56cee210-55b5-4da5-9766-455bb56d7cf8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-5lsxv\" (UID: \"56cee210-55b5-4da5-9766-455bb56d7cf8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" May 21 01:02:12.146889 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.146865 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/56cee210-55b5-4da5-9766-455bb56d7cf8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-5lsxv\" (UID: \"56cee210-55b5-4da5-9766-455bb56d7cf8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" May 21 01:02:12.148999 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.148970 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/56cee210-55b5-4da5-9766-455bb56d7cf8-local-certs\") pod \"istiod-openshift-gateway-798958bb55-5lsxv\" (UID: \"56cee210-55b5-4da5-9766-455bb56d7cf8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" May 21 01:02:12.149161 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.149074 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/56cee210-55b5-4da5-9766-455bb56d7cf8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-5lsxv\" (UID: \"56cee210-55b5-4da5-9766-455bb56d7cf8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" May 21 01:02:12.149161 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.149093 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/56cee210-55b5-4da5-9766-455bb56d7cf8-cacerts\") pod \"istiod-openshift-gateway-798958bb55-5lsxv\" (UID: \"56cee210-55b5-4da5-9766-455bb56d7cf8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" May 21 01:02:12.149359 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.149338 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/56cee210-55b5-4da5-9766-455bb56d7cf8-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-5lsxv\" (UID: \"56cee210-55b5-4da5-9766-455bb56d7cf8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" May 21 01:02:12.154647 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.154626 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9ms6\" (UniqueName: \"kubernetes.io/projected/56cee210-55b5-4da5-9766-455bb56d7cf8-kube-api-access-m9ms6\") pod \"istiod-openshift-gateway-798958bb55-5lsxv\" (UID: \"56cee210-55b5-4da5-9766-455bb56d7cf8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" May 21 01:02:12.154817 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.154798 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56cee210-55b5-4da5-9766-455bb56d7cf8-istio-token\") pod \"istiod-openshift-gateway-798958bb55-5lsxv\" (UID: \"56cee210-55b5-4da5-9766-455bb56d7cf8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" May 21 01:02:12.322018 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.321928 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" May 21 01:02:12.466796 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:12.466770 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv"] May 21 01:02:12.470430 ip-10-0-138-101 kubenswrapper[2549]: W0521 01:02:12.470399 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56cee210_55b5_4da5_9766_455bb56d7cf8.slice/crio-179ec8c759aab00c46ca371ffda642c3240255658324383fa587b310ba30ad51 WatchSource:0}: Error finding container 179ec8c759aab00c46ca371ffda642c3240255658324383fa587b310ba30ad51: Status 404 returned error can't find the container with id 179ec8c759aab00c46ca371ffda642c3240255658324383fa587b310ba30ad51 May 21 01:02:13.083099 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:13.083038 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" event={"ID":"56cee210-55b5-4da5-9766-455bb56d7cf8","Type":"ContainerStarted","Data":"179ec8c759aab00c46ca371ffda642c3240255658324383fa587b310ba30ad51"} May 21 01:02:17.053382 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:17.053334 2549 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} May 21 01:02:17.053652 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:17.053426 2549 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} May 21 01:02:17.063585 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:17.063562 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-858dbf95b8-ghjff" May 21 01:02:18.104206 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:18.104174 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" event={"ID":"56cee210-55b5-4da5-9766-455bb56d7cf8","Type":"ContainerStarted","Data":"5c1379bfa1ee2486c7ff8d3cbc10f4022a13ea933b2dba0d784b1df360c72d22"} May 21 01:02:18.104590 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:18.104383 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" May 21 01:02:18.106009 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:18.105982 2549 patch_prober.go:28] interesting pod/istiod-openshift-gateway-798958bb55-5lsxv container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= May 21 01:02:18.106133 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:18.106042 2549 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" podUID="56cee210-55b5-4da5-9766-455bb56d7cf8" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" May 21 01:02:18.123957 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:18.123904 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" podStartSLOduration=2.543774705 podStartE2EDuration="7.123890179s" podCreationTimestamp="2026-05-21 01:02:11 +0000 UTC" firstStartedPulling="2026-05-21 01:02:12.472973316 +0000 UTC m=+451.369918610" lastFinishedPulling="2026-05-21 01:02:17.053088781 +0000 UTC m=+455.950034084" observedRunningTime="2026-05-21 01:02:18.122367684 +0000 UTC m=+457.019312995" watchObservedRunningTime="2026-05-21 01:02:18.123890179 +0000 UTC m=+457.020835489" May 21 01:02:19.108611 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:19.108581 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-5lsxv" May 21 01:02:26.026433 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:02:26.026403 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-856948b99f-fjfvj" May 21 01:03:12.086328 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:12.086298 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bc75v"] May 21 01:03:12.088732 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:12.088716 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bc75v" May 21 01:03:12.091053 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:12.091029 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" May 21 01:03:12.091238 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:12.091154 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-kfg79\"" May 21 01:03:12.092134 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:12.092118 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" May 21 01:03:12.102245 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:12.102227 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bc75v"] May 21 01:03:12.178521 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:12.178490 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h6pv\" (UniqueName: \"kubernetes.io/projected/fe3a8a08-5278-4c77-8954-37a7504f46dd-kube-api-access-4h6pv\") pod \"limitador-operator-controller-manager-85c4996f8c-bc75v\" (UID: \"fe3a8a08-5278-4c77-8954-37a7504f46dd\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bc75v" May 21 01:03:12.279297 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:12.279261 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4h6pv\" (UniqueName: \"kubernetes.io/projected/fe3a8a08-5278-4c77-8954-37a7504f46dd-kube-api-access-4h6pv\") pod \"limitador-operator-controller-manager-85c4996f8c-bc75v\" (UID: \"fe3a8a08-5278-4c77-8954-37a7504f46dd\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bc75v" May 21 01:03:12.287591 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:12.287564 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h6pv\" (UniqueName: \"kubernetes.io/projected/fe3a8a08-5278-4c77-8954-37a7504f46dd-kube-api-access-4h6pv\") pod \"limitador-operator-controller-manager-85c4996f8c-bc75v\" (UID: \"fe3a8a08-5278-4c77-8954-37a7504f46dd\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bc75v" May 21 01:03:12.398810 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:12.398744 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bc75v" May 21 01:03:12.521514 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:12.521486 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bc75v"] May 21 01:03:12.523722 ip-10-0-138-101 kubenswrapper[2549]: W0521 01:03:12.523690 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe3a8a08_5278_4c77_8954_37a7504f46dd.slice/crio-ee894a43f582e4776de454b07010210a70c590d56668cedffed7f582eb7a7fb7 WatchSource:0}: Error finding container ee894a43f582e4776de454b07010210a70c590d56668cedffed7f582eb7a7fb7: Status 404 returned error can't find the container with id ee894a43f582e4776de454b07010210a70c590d56668cedffed7f582eb7a7fb7 May 21 01:03:13.302135 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:13.302093 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bc75v" event={"ID":"fe3a8a08-5278-4c77-8954-37a7504f46dd","Type":"ContainerStarted","Data":"ee894a43f582e4776de454b07010210a70c590d56668cedffed7f582eb7a7fb7"} May 21 01:03:15.310327 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:15.310295 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bc75v" event={"ID":"fe3a8a08-5278-4c77-8954-37a7504f46dd","Type":"ContainerStarted","Data":"dc1b97ea24a40cc05c81871059d930b9391eb8a96a57f2f8a07601a1620f3f14"} May 21 01:03:15.310616 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:15.310416 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bc75v" May 21 01:03:15.327483 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:15.327443 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bc75v" podStartSLOduration=0.641672755 podStartE2EDuration="3.327430891s" podCreationTimestamp="2026-05-21 01:03:12 +0000 UTC" firstStartedPulling="2026-05-21 01:03:12.525639821 +0000 UTC m=+511.422585109" lastFinishedPulling="2026-05-21 01:03:15.211397951 +0000 UTC m=+514.108343245" observedRunningTime="2026-05-21 01:03:15.32617471 +0000 UTC m=+514.223120021" watchObservedRunningTime="2026-05-21 01:03:15.327430891 +0000 UTC m=+514.224376202" May 21 01:03:23.509726 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:23.509699 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-rr2dx"] May 21 01:03:23.513290 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:23.513270 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-rr2dx" May 21 01:03:23.515766 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:23.515747 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-xn4j2\"" May 21 01:03:23.515832 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:23.515791 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" May 21 01:03:23.522572 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:23.522548 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-rr2dx"] May 21 01:03:23.689710 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:23.689682 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdxqd\" (UniqueName: \"kubernetes.io/projected/a25595d3-06ff-459a-9773-9204fa9e29d9-kube-api-access-pdxqd\") pod \"dns-operator-controller-manager-648d5c98bc-rr2dx\" (UID: \"a25595d3-06ff-459a-9773-9204fa9e29d9\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-rr2dx" May 21 01:03:23.790815 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:23.790749 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pdxqd\" (UniqueName: \"kubernetes.io/projected/a25595d3-06ff-459a-9773-9204fa9e29d9-kube-api-access-pdxqd\") pod \"dns-operator-controller-manager-648d5c98bc-rr2dx\" (UID: \"a25595d3-06ff-459a-9773-9204fa9e29d9\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-rr2dx" May 21 01:03:23.800543 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:23.800517 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdxqd\" (UniqueName: \"kubernetes.io/projected/a25595d3-06ff-459a-9773-9204fa9e29d9-kube-api-access-pdxqd\") pod \"dns-operator-controller-manager-648d5c98bc-rr2dx\" (UID: \"a25595d3-06ff-459a-9773-9204fa9e29d9\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-rr2dx" May 21 01:03:23.823916 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:23.823894 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-rr2dx" May 21 01:03:23.976270 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:23.976197 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-rr2dx"] May 21 01:03:23.980967 ip-10-0-138-101 kubenswrapper[2549]: W0521 01:03:23.980929 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda25595d3_06ff_459a_9773_9204fa9e29d9.slice/crio-d9b5582f3d6f983c10b1397cc08bfcbb404039ef8aef7540a11997a3ace98463 WatchSource:0}: Error finding container d9b5582f3d6f983c10b1397cc08bfcbb404039ef8aef7540a11997a3ace98463: Status 404 returned error can't find the container with id d9b5582f3d6f983c10b1397cc08bfcbb404039ef8aef7540a11997a3ace98463 May 21 01:03:24.343799 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:24.343757 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-rr2dx" event={"ID":"a25595d3-06ff-459a-9773-9204fa9e29d9","Type":"ContainerStarted","Data":"d9b5582f3d6f983c10b1397cc08bfcbb404039ef8aef7540a11997a3ace98463"} May 21 01:03:26.317250 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:26.317151 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bc75v" May 21 01:03:27.363048 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:27.363011 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-rr2dx" event={"ID":"a25595d3-06ff-459a-9773-9204fa9e29d9","Type":"ContainerStarted","Data":"9ee01b63158961357b7424a4eeacf09c743b9323b69cd7e87439a689627aaa57"} May 21 01:03:27.363515 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:27.363128 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-rr2dx" May 21 01:03:27.383296 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:27.383251 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-rr2dx" podStartSLOduration=1.910026916 podStartE2EDuration="4.383234603s" podCreationTimestamp="2026-05-21 01:03:23 +0000 UTC" firstStartedPulling="2026-05-21 01:03:23.983738689 +0000 UTC m=+522.880683978" lastFinishedPulling="2026-05-21 01:03:26.456946363 +0000 UTC m=+525.353891665" observedRunningTime="2026-05-21 01:03:27.381693948 +0000 UTC m=+526.278639260" watchObservedRunningTime="2026-05-21 01:03:27.383234603 +0000 UTC m=+526.280179916" May 21 01:03:35.828513 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:35.828480 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-qkmkq"] May 21 01:03:35.832204 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:35.832189 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-qkmkq" May 21 01:03:35.835967 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:35.835944 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hklcs\"" May 21 01:03:35.848154 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:35.848132 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-qkmkq"] May 21 01:03:35.896359 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:35.896328 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dbf88b72-ea42-4991-a1b4-69bdb2ca75af-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-qkmkq\" (UID: \"dbf88b72-ea42-4991-a1b4-69bdb2ca75af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-qkmkq" May 21 01:03:35.896510 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:35.896393 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmg7z\" (UniqueName: \"kubernetes.io/projected/dbf88b72-ea42-4991-a1b4-69bdb2ca75af-kube-api-access-pmg7z\") pod \"kuadrant-operator-controller-manager-84b657d985-qkmkq\" (UID: \"dbf88b72-ea42-4991-a1b4-69bdb2ca75af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-qkmkq" May 21 01:03:35.996843 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:35.996815 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dbf88b72-ea42-4991-a1b4-69bdb2ca75af-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-qkmkq\" (UID: \"dbf88b72-ea42-4991-a1b4-69bdb2ca75af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-qkmkq" May 21 01:03:35.996976 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:35.996864 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pmg7z\" (UniqueName: \"kubernetes.io/projected/dbf88b72-ea42-4991-a1b4-69bdb2ca75af-kube-api-access-pmg7z\") pod \"kuadrant-operator-controller-manager-84b657d985-qkmkq\" (UID: \"dbf88b72-ea42-4991-a1b4-69bdb2ca75af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-qkmkq" May 21 01:03:35.997254 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:35.997231 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dbf88b72-ea42-4991-a1b4-69bdb2ca75af-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-qkmkq\" (UID: \"dbf88b72-ea42-4991-a1b4-69bdb2ca75af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-qkmkq" May 21 01:03:36.004690 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:36.004669 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmg7z\" (UniqueName: \"kubernetes.io/projected/dbf88b72-ea42-4991-a1b4-69bdb2ca75af-kube-api-access-pmg7z\") pod \"kuadrant-operator-controller-manager-84b657d985-qkmkq\" (UID: \"dbf88b72-ea42-4991-a1b4-69bdb2ca75af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-qkmkq" May 21 01:03:36.147627 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:36.147543 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-qkmkq" May 21 01:03:36.279784 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:36.279761 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-qkmkq"] May 21 01:03:36.282676 ip-10-0-138-101 kubenswrapper[2549]: W0521 01:03:36.282648 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbf88b72_ea42_4991_a1b4_69bdb2ca75af.slice/crio-3e6770e988de289358b1b3caee4fc063165ec2c0e35a0e68b63aa3fb624ecc6e WatchSource:0}: Error finding container 3e6770e988de289358b1b3caee4fc063165ec2c0e35a0e68b63aa3fb624ecc6e: Status 404 returned error can't find the container with id 3e6770e988de289358b1b3caee4fc063165ec2c0e35a0e68b63aa3fb624ecc6e May 21 01:03:36.398651 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:36.398591 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-qkmkq" event={"ID":"dbf88b72-ea42-4991-a1b4-69bdb2ca75af","Type":"ContainerStarted","Data":"3e6770e988de289358b1b3caee4fc063165ec2c0e35a0e68b63aa3fb624ecc6e"} May 21 01:03:36.875905 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:36.875870 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-qkmkq"] May 21 01:03:36.891520 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:36.891456 2549 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-qkmkq"] May 21 01:03:36.903233 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:36.903203 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cldj6"] May 21 01:03:36.909043 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:36.909020 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bc75v"] May 21 01:03:36.909242 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:36.909183 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cldj6" May 21 01:03:36.909330 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:36.909303 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bc75v" podUID="fe3a8a08-5278-4c77-8954-37a7504f46dd" containerName="manager" containerID="cri-o://dc1b97ea24a40cc05c81871059d930b9391eb8a96a57f2f8a07601a1620f3f14" gracePeriod=2 May 21 01:03:36.927002 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:36.926968 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cldj6"] May 21 01:03:36.928659 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:36.928641 2549 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bc75v"] May 21 01:03:36.943393 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:36.943370 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2b9zq"] May 21 01:03:36.944875 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:36.944686 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe3a8a08-5278-4c77-8954-37a7504f46dd" containerName="manager" May 21 01:03:36.944875 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:36.944714 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe3a8a08-5278-4c77-8954-37a7504f46dd" containerName="manager" May 21 01:03:36.945037 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:36.944932 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="fe3a8a08-5278-4c77-8954-37a7504f46dd" containerName="manager" May 21 01:03:36.946952 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:36.946921 2549 status_manager.go:895] "Failed to get status for pod" podUID="fe3a8a08-5278-4c77-8954-37a7504f46dd" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bc75v" err="pods \"limitador-operator-controller-manager-85c4996f8c-bc75v\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" May 21 01:03:36.948392 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:36.948370 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2b9zq" May 21 01:03:36.949261 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:36.949212 2549 status_manager.go:895] "Failed to get status for pod" podUID="fe3a8a08-5278-4c77-8954-37a7504f46dd" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bc75v" err="pods \"limitador-operator-controller-manager-85c4996f8c-bc75v\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" May 21 01:03:36.959299 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:36.959262 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2b9zq"] May 21 01:03:36.975797 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:36.975757 2549 status_manager.go:895] "Failed to get status for pod" podUID="fe3a8a08-5278-4c77-8954-37a7504f46dd" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bc75v" err="pods \"limitador-operator-controller-manager-85c4996f8c-bc75v\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" May 21 01:03:37.006981 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:37.006953 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-948jd\" (UniqueName: \"kubernetes.io/projected/b7aa0746-9adf-4210-9a48-efc85cfde172-kube-api-access-948jd\") pod \"limitador-operator-controller-manager-85c4996f8c-2b9zq\" (UID: \"b7aa0746-9adf-4210-9a48-efc85cfde172\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2b9zq" May 21 01:03:37.007133 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:37.006990 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8ac94676-bdec-4ccc-96a0-d62de714064d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-cldj6\" (UID: \"8ac94676-bdec-4ccc-96a0-d62de714064d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cldj6" May 21 01:03:37.007133 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:37.007018 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr792\" (UniqueName: \"kubernetes.io/projected/8ac94676-bdec-4ccc-96a0-d62de714064d-kube-api-access-cr792\") pod \"kuadrant-operator-controller-manager-84b657d985-cldj6\" (UID: \"8ac94676-bdec-4ccc-96a0-d62de714064d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cldj6" May 21 01:03:37.109158 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:37.108336 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-948jd\" (UniqueName: \"kubernetes.io/projected/b7aa0746-9adf-4210-9a48-efc85cfde172-kube-api-access-948jd\") pod \"limitador-operator-controller-manager-85c4996f8c-2b9zq\" (UID: \"b7aa0746-9adf-4210-9a48-efc85cfde172\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2b9zq" May 21 01:03:37.109305 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:37.109201 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8ac94676-bdec-4ccc-96a0-d62de714064d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-cldj6\" (UID: \"8ac94676-bdec-4ccc-96a0-d62de714064d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cldj6" May 21 01:03:37.109305 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:37.109247 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cr792\" (UniqueName: \"kubernetes.io/projected/8ac94676-bdec-4ccc-96a0-d62de714064d-kube-api-access-cr792\") pod \"kuadrant-operator-controller-manager-84b657d985-cldj6\" (UID: \"8ac94676-bdec-4ccc-96a0-d62de714064d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cldj6" May 21 01:03:37.109666 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:37.109614 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8ac94676-bdec-4ccc-96a0-d62de714064d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-cldj6\" (UID: \"8ac94676-bdec-4ccc-96a0-d62de714064d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cldj6" May 21 01:03:37.119686 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:37.119636 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr792\" (UniqueName: \"kubernetes.io/projected/8ac94676-bdec-4ccc-96a0-d62de714064d-kube-api-access-cr792\") pod \"kuadrant-operator-controller-manager-84b657d985-cldj6\" (UID: \"8ac94676-bdec-4ccc-96a0-d62de714064d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cldj6" May 21 01:03:37.119829 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:37.119811 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-948jd\" (UniqueName: \"kubernetes.io/projected/b7aa0746-9adf-4210-9a48-efc85cfde172-kube-api-access-948jd\") pod \"limitador-operator-controller-manager-85c4996f8c-2b9zq\" (UID: \"b7aa0746-9adf-4210-9a48-efc85cfde172\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2b9zq" May 21 01:03:37.164693 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:37.164671 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bc75v" May 21 01:03:37.167044 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:37.167017 2549 status_manager.go:895] "Failed to get status for pod" podUID="fe3a8a08-5278-4c77-8954-37a7504f46dd" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bc75v" err="pods \"limitador-operator-controller-manager-85c4996f8c-bc75v\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" May 21 01:03:37.210990 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:37.210944 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4h6pv\" (UniqueName: \"kubernetes.io/projected/fe3a8a08-5278-4c77-8954-37a7504f46dd-kube-api-access-4h6pv\") pod \"fe3a8a08-5278-4c77-8954-37a7504f46dd\" (UID: \"fe3a8a08-5278-4c77-8954-37a7504f46dd\") " May 21 01:03:37.213457 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:37.213425 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe3a8a08-5278-4c77-8954-37a7504f46dd-kube-api-access-4h6pv" (OuterVolumeSpecName: "kube-api-access-4h6pv") pod "fe3a8a08-5278-4c77-8954-37a7504f46dd" (UID: "fe3a8a08-5278-4c77-8954-37a7504f46dd"). InnerVolumeSpecName "kube-api-access-4h6pv". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 21 01:03:37.269878 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:37.269845 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cldj6" May 21 01:03:37.279185 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:37.279160 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2b9zq" May 21 01:03:37.312304 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:37.312278 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4h6pv\" (UniqueName: \"kubernetes.io/projected/fe3a8a08-5278-4c77-8954-37a7504f46dd-kube-api-access-4h6pv\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 01:03:37.407905 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:37.407835 2549 generic.go:358] "Generic (PLEG): container finished" podID="fe3a8a08-5278-4c77-8954-37a7504f46dd" containerID="dc1b97ea24a40cc05c81871059d930b9391eb8a96a57f2f8a07601a1620f3f14" exitCode=0 May 21 01:03:37.407905 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:37.407896 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bc75v" May 21 01:03:37.408048 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:37.407942 2549 scope.go:117] "RemoveContainer" containerID="dc1b97ea24a40cc05c81871059d930b9391eb8a96a57f2f8a07601a1620f3f14" May 21 01:03:37.410302 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:37.410241 2549 status_manager.go:895] "Failed to get status for pod" podUID="fe3a8a08-5278-4c77-8954-37a7504f46dd" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bc75v" err="pods \"limitador-operator-controller-manager-85c4996f8c-bc75v\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" May 21 01:03:37.421043 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:37.421025 2549 scope.go:117] "RemoveContainer" containerID="dc1b97ea24a40cc05c81871059d930b9391eb8a96a57f2f8a07601a1620f3f14" May 21 01:03:37.422215 ip-10-0-138-101 kubenswrapper[2549]: E0521 01:03:37.422137 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc1b97ea24a40cc05c81871059d930b9391eb8a96a57f2f8a07601a1620f3f14\": container with ID starting with dc1b97ea24a40cc05c81871059d930b9391eb8a96a57f2f8a07601a1620f3f14 not found: ID does not exist" containerID="dc1b97ea24a40cc05c81871059d930b9391eb8a96a57f2f8a07601a1620f3f14" May 21 01:03:37.422215 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:37.422179 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc1b97ea24a40cc05c81871059d930b9391eb8a96a57f2f8a07601a1620f3f14"} err="failed to get container status \"dc1b97ea24a40cc05c81871059d930b9391eb8a96a57f2f8a07601a1620f3f14\": rpc error: code = NotFound desc = could not find container \"dc1b97ea24a40cc05c81871059d930b9391eb8a96a57f2f8a07601a1620f3f14\": container with ID starting with dc1b97ea24a40cc05c81871059d930b9391eb8a96a57f2f8a07601a1620f3f14 not found: ID does not exist" May 21 01:03:37.423265 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:37.423232 2549 status_manager.go:895] "Failed to get status for pod" podUID="fe3a8a08-5278-4c77-8954-37a7504f46dd" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bc75v" err="pods \"limitador-operator-controller-manager-85c4996f8c-bc75v\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" May 21 01:03:37.498362 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:37.498285 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cldj6"] May 21 01:03:37.501258 ip-10-0-138-101 kubenswrapper[2549]: W0521 01:03:37.501168 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ac94676_bdec_4ccc_96a0_d62de714064d.slice/crio-9ee894b193f746c03d16cce1e538faf188fa824e1dda98d55025e4daa860ec8c WatchSource:0}: Error finding container 9ee894b193f746c03d16cce1e538faf188fa824e1dda98d55025e4daa860ec8c: Status 404 returned error can't find the container with id 9ee894b193f746c03d16cce1e538faf188fa824e1dda98d55025e4daa860ec8c May 21 01:03:37.513205 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:37.513178 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2b9zq"] May 21 01:03:37.515425 ip-10-0-138-101 kubenswrapper[2549]: W0521 01:03:37.515398 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7aa0746_9adf_4210_9a48_efc85cfde172.slice/crio-7635f0d73b00c69db7a87e40910029b90a1087c6e7183ed559d3782d9f4626d1 WatchSource:0}: Error finding container 7635f0d73b00c69db7a87e40910029b90a1087c6e7183ed559d3782d9f4626d1: Status 404 returned error can't find the container with id 7635f0d73b00c69db7a87e40910029b90a1087c6e7183ed559d3782d9f4626d1 May 21 01:03:37.672282 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:37.672213 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe3a8a08-5278-4c77-8954-37a7504f46dd" path="/var/lib/kubelet/pods/fe3a8a08-5278-4c77-8954-37a7504f46dd/volumes" May 21 01:03:38.370746 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:38.370455 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-rr2dx" May 21 01:03:38.414414 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:38.414346 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2b9zq" event={"ID":"b7aa0746-9adf-4210-9a48-efc85cfde172","Type":"ContainerStarted","Data":"3191a0d3bae59492743af58ff8629587a24ae4ec3f8f001d2bb0bf08b5b23ec2"} May 21 01:03:38.414414 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:38.414388 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2b9zq" event={"ID":"b7aa0746-9adf-4210-9a48-efc85cfde172","Type":"ContainerStarted","Data":"7635f0d73b00c69db7a87e40910029b90a1087c6e7183ed559d3782d9f4626d1"} May 21 01:03:38.414654 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:38.414454 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2b9zq" May 21 01:03:38.417261 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:38.417224 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cldj6" event={"ID":"8ac94676-bdec-4ccc-96a0-d62de714064d","Type":"ContainerStarted","Data":"9ee894b193f746c03d16cce1e538faf188fa824e1dda98d55025e4daa860ec8c"} May 21 01:03:38.434242 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:38.434192 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2b9zq" podStartSLOduration=2.434176922 podStartE2EDuration="2.434176922s" podCreationTimestamp="2026-05-21 01:03:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 01:03:38.432812158 +0000 UTC m=+537.329757494" watchObservedRunningTime="2026-05-21 01:03:38.434176922 +0000 UTC m=+537.331122234" May 21 01:03:41.430853 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:41.430780 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-qkmkq" podUID="dbf88b72-ea42-4991-a1b4-69bdb2ca75af" containerName="manager" containerID="cri-o://fed46b01e3671039d21b341fead94878e889932b7befef09efa16e3ceb4acdad" gracePeriod=2 May 21 01:03:41.432507 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:41.432461 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cldj6" event={"ID":"8ac94676-bdec-4ccc-96a0-d62de714064d","Type":"ContainerStarted","Data":"9e9e445d89b6fe833989fc5c763b1c9e3a0dcaf0579a35e1779b6540f6e14e22"} May 21 01:03:41.432657 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:41.432584 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cldj6" May 21 01:03:41.452393 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:41.452336 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cldj6" podStartSLOduration=2.398133419 podStartE2EDuration="5.45232045s" podCreationTimestamp="2026-05-21 01:03:36 +0000 UTC" firstStartedPulling="2026-05-21 01:03:37.504176067 +0000 UTC m=+536.401121356" lastFinishedPulling="2026-05-21 01:03:40.558363089 +0000 UTC m=+539.455308387" observedRunningTime="2026-05-21 01:03:41.451268308 +0000 UTC m=+540.348213621" watchObservedRunningTime="2026-05-21 01:03:41.45232045 +0000 UTC m=+540.349265763" May 21 01:03:41.662291 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:41.662266 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-qkmkq" May 21 01:03:41.664295 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:41.664261 2549 status_manager.go:895] "Failed to get status for pod" podUID="dbf88b72-ea42-4991-a1b4-69bdb2ca75af" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-qkmkq" err="pods \"kuadrant-operator-controller-manager-84b657d985-qkmkq\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" May 21 01:03:41.672023 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:41.671996 2549 status_manager.go:895] "Failed to get status for pod" podUID="dbf88b72-ea42-4991-a1b4-69bdb2ca75af" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-qkmkq" err="pods \"kuadrant-operator-controller-manager-84b657d985-qkmkq\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" May 21 01:03:41.748850 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:41.748827 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dbf88b72-ea42-4991-a1b4-69bdb2ca75af-extensions-socket-volume\") pod \"dbf88b72-ea42-4991-a1b4-69bdb2ca75af\" (UID: \"dbf88b72-ea42-4991-a1b4-69bdb2ca75af\") " May 21 01:03:41.748955 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:41.748876 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmg7z\" (UniqueName: \"kubernetes.io/projected/dbf88b72-ea42-4991-a1b4-69bdb2ca75af-kube-api-access-pmg7z\") pod \"dbf88b72-ea42-4991-a1b4-69bdb2ca75af\" (UID: \"dbf88b72-ea42-4991-a1b4-69bdb2ca75af\") " May 21 01:03:41.749131 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:41.749097 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbf88b72-ea42-4991-a1b4-69bdb2ca75af-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "dbf88b72-ea42-4991-a1b4-69bdb2ca75af" (UID: "dbf88b72-ea42-4991-a1b4-69bdb2ca75af"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 21 01:03:41.749449 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:41.749402 2549 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dbf88b72-ea42-4991-a1b4-69bdb2ca75af-extensions-socket-volume\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 01:03:41.751399 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:41.751378 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbf88b72-ea42-4991-a1b4-69bdb2ca75af-kube-api-access-pmg7z" (OuterVolumeSpecName: "kube-api-access-pmg7z") pod "dbf88b72-ea42-4991-a1b4-69bdb2ca75af" (UID: "dbf88b72-ea42-4991-a1b4-69bdb2ca75af"). InnerVolumeSpecName "kube-api-access-pmg7z". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 21 01:03:41.850533 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:41.850510 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmg7z\" (UniqueName: \"kubernetes.io/projected/dbf88b72-ea42-4991-a1b4-69bdb2ca75af-kube-api-access-pmg7z\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 01:03:42.437082 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:42.437033 2549 generic.go:358] "Generic (PLEG): container finished" podID="dbf88b72-ea42-4991-a1b4-69bdb2ca75af" containerID="fed46b01e3671039d21b341fead94878e889932b7befef09efa16e3ceb4acdad" exitCode=2 May 21 01:03:42.437547 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:42.437098 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-qkmkq" May 21 01:03:42.437547 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:42.437129 2549 scope.go:117] "RemoveContainer" containerID="fed46b01e3671039d21b341fead94878e889932b7befef09efa16e3ceb4acdad" May 21 01:03:42.445588 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:42.445568 2549 scope.go:117] "RemoveContainer" containerID="fed46b01e3671039d21b341fead94878e889932b7befef09efa16e3ceb4acdad" May 21 01:03:42.445838 ip-10-0-138-101 kubenswrapper[2549]: E0521 01:03:42.445818 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fed46b01e3671039d21b341fead94878e889932b7befef09efa16e3ceb4acdad\": container with ID starting with fed46b01e3671039d21b341fead94878e889932b7befef09efa16e3ceb4acdad not found: ID does not exist" containerID="fed46b01e3671039d21b341fead94878e889932b7befef09efa16e3ceb4acdad" May 21 01:03:42.445881 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:42.445849 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fed46b01e3671039d21b341fead94878e889932b7befef09efa16e3ceb4acdad"} err="failed to get container status \"fed46b01e3671039d21b341fead94878e889932b7befef09efa16e3ceb4acdad\": rpc error: code = NotFound desc = could not find container \"fed46b01e3671039d21b341fead94878e889932b7befef09efa16e3ceb4acdad\": container with ID starting with fed46b01e3671039d21b341fead94878e889932b7befef09efa16e3ceb4acdad not found: ID does not exist" May 21 01:03:42.446912 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:42.446890 2549 status_manager.go:895] "Failed to get status for pod" podUID="dbf88b72-ea42-4991-a1b4-69bdb2ca75af" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-qkmkq" err="pods \"kuadrant-operator-controller-manager-84b657d985-qkmkq\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" May 21 01:03:43.671934 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:43.671899 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbf88b72-ea42-4991-a1b4-69bdb2ca75af" path="/var/lib/kubelet/pods/dbf88b72-ea42-4991-a1b4-69bdb2ca75af/volumes" May 21 01:03:49.423538 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:49.423503 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-2b9zq" May 21 01:03:52.439847 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:52.439807 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cldj6" May 21 01:03:52.758638 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:52.758585 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qdhqk"] May 21 01:03:52.759113 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:52.759094 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbf88b72-ea42-4991-a1b4-69bdb2ca75af" containerName="manager" May 21 01:03:52.759113 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:52.759113 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbf88b72-ea42-4991-a1b4-69bdb2ca75af" containerName="manager" May 21 01:03:52.759342 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:52.759222 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="dbf88b72-ea42-4991-a1b4-69bdb2ca75af" containerName="manager" May 21 01:03:52.764948 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:52.764926 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qdhqk" May 21 01:03:52.774503 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:52.774471 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qdhqk"] May 21 01:03:52.840572 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:52.840543 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czkf8\" (UniqueName: \"kubernetes.io/projected/7d65159b-310f-4c22-8d17-42a0a42a6e5a-kube-api-access-czkf8\") pod \"kuadrant-operator-controller-manager-55c7f4c975-qdhqk\" (UID: \"7d65159b-310f-4c22-8d17-42a0a42a6e5a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qdhqk" May 21 01:03:52.840712 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:52.840607 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d65159b-310f-4c22-8d17-42a0a42a6e5a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-qdhqk\" (UID: \"7d65159b-310f-4c22-8d17-42a0a42a6e5a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qdhqk" May 21 01:03:52.941809 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:52.941783 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d65159b-310f-4c22-8d17-42a0a42a6e5a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-qdhqk\" (UID: \"7d65159b-310f-4c22-8d17-42a0a42a6e5a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qdhqk" May 21 01:03:52.941901 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:52.941856 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-czkf8\" (UniqueName: \"kubernetes.io/projected/7d65159b-310f-4c22-8d17-42a0a42a6e5a-kube-api-access-czkf8\") pod \"kuadrant-operator-controller-manager-55c7f4c975-qdhqk\" (UID: \"7d65159b-310f-4c22-8d17-42a0a42a6e5a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qdhqk" May 21 01:03:52.942245 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:52.942229 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d65159b-310f-4c22-8d17-42a0a42a6e5a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-qdhqk\" (UID: \"7d65159b-310f-4c22-8d17-42a0a42a6e5a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qdhqk" May 21 01:03:52.953211 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:52.953158 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czkf8\" (UniqueName: \"kubernetes.io/projected/7d65159b-310f-4c22-8d17-42a0a42a6e5a-kube-api-access-czkf8\") pod \"kuadrant-operator-controller-manager-55c7f4c975-qdhqk\" (UID: \"7d65159b-310f-4c22-8d17-42a0a42a6e5a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qdhqk" May 21 01:03:53.076863 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:53.076835 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qdhqk" May 21 01:03:53.203318 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:53.203267 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qdhqk"] May 21 01:03:53.205187 ip-10-0-138-101 kubenswrapper[2549]: W0521 01:03:53.205154 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d65159b_310f_4c22_8d17_42a0a42a6e5a.slice/crio-bfda9aaa978ca8b20cfc06f59609c904abd572a1abc19c3ab540bf11e3ae6fe6 WatchSource:0}: Error finding container bfda9aaa978ca8b20cfc06f59609c904abd572a1abc19c3ab540bf11e3ae6fe6: Status 404 returned error can't find the container with id bfda9aaa978ca8b20cfc06f59609c904abd572a1abc19c3ab540bf11e3ae6fe6 May 21 01:03:53.483542 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:53.483449 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qdhqk" event={"ID":"7d65159b-310f-4c22-8d17-42a0a42a6e5a","Type":"ContainerStarted","Data":"6b6aa3bb5dd614b33cd2536b29476c6c28db29bb6ce58bfabf61c24ea7cfe2b5"} May 21 01:03:53.483542 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:53.483491 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qdhqk" event={"ID":"7d65159b-310f-4c22-8d17-42a0a42a6e5a","Type":"ContainerStarted","Data":"bfda9aaa978ca8b20cfc06f59609c904abd572a1abc19c3ab540bf11e3ae6fe6"} May 21 01:03:53.483542 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:53.483531 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qdhqk" May 21 01:03:53.500520 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:03:53.500464 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qdhqk" podStartSLOduration=1.5004480359999999 podStartE2EDuration="1.500448036s" podCreationTimestamp="2026-05-21 01:03:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 01:03:53.49952356 +0000 UTC m=+552.396468873" watchObservedRunningTime="2026-05-21 01:03:53.500448036 +0000 UTC m=+552.397393350" May 21 01:04:04.489710 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:04.489676 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qdhqk" May 21 01:04:04.560834 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:04.560800 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cldj6"] May 21 01:04:04.561043 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:04.561021 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cldj6" podUID="8ac94676-bdec-4ccc-96a0-d62de714064d" containerName="manager" containerID="cri-o://9e9e445d89b6fe833989fc5c763b1c9e3a0dcaf0579a35e1779b6540f6e14e22" gracePeriod=10 May 21 01:04:04.821325 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:04.821303 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cldj6" May 21 01:04:04.939859 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:04.939829 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cr792\" (UniqueName: \"kubernetes.io/projected/8ac94676-bdec-4ccc-96a0-d62de714064d-kube-api-access-cr792\") pod \"8ac94676-bdec-4ccc-96a0-d62de714064d\" (UID: \"8ac94676-bdec-4ccc-96a0-d62de714064d\") " May 21 01:04:04.939989 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:04.939901 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8ac94676-bdec-4ccc-96a0-d62de714064d-extensions-socket-volume\") pod \"8ac94676-bdec-4ccc-96a0-d62de714064d\" (UID: \"8ac94676-bdec-4ccc-96a0-d62de714064d\") " May 21 01:04:04.940312 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:04.940289 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ac94676-bdec-4ccc-96a0-d62de714064d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8ac94676-bdec-4ccc-96a0-d62de714064d" (UID: "8ac94676-bdec-4ccc-96a0-d62de714064d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 21 01:04:04.941950 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:04.941925 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ac94676-bdec-4ccc-96a0-d62de714064d-kube-api-access-cr792" (OuterVolumeSpecName: "kube-api-access-cr792") pod "8ac94676-bdec-4ccc-96a0-d62de714064d" (UID: "8ac94676-bdec-4ccc-96a0-d62de714064d"). InnerVolumeSpecName "kube-api-access-cr792". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 21 01:04:05.041452 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:05.041422 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cr792\" (UniqueName: \"kubernetes.io/projected/8ac94676-bdec-4ccc-96a0-d62de714064d-kube-api-access-cr792\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 01:04:05.041452 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:05.041448 2549 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8ac94676-bdec-4ccc-96a0-d62de714064d-extensions-socket-volume\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" May 21 01:04:05.528353 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:05.528322 2549 generic.go:358] "Generic (PLEG): container finished" podID="8ac94676-bdec-4ccc-96a0-d62de714064d" containerID="9e9e445d89b6fe833989fc5c763b1c9e3a0dcaf0579a35e1779b6540f6e14e22" exitCode=0 May 21 01:04:05.528718 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:05.528384 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cldj6" May 21 01:04:05.528718 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:05.528395 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cldj6" event={"ID":"8ac94676-bdec-4ccc-96a0-d62de714064d","Type":"ContainerDied","Data":"9e9e445d89b6fe833989fc5c763b1c9e3a0dcaf0579a35e1779b6540f6e14e22"} May 21 01:04:05.528718 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:05.528420 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cldj6" event={"ID":"8ac94676-bdec-4ccc-96a0-d62de714064d","Type":"ContainerDied","Data":"9ee894b193f746c03d16cce1e538faf188fa824e1dda98d55025e4daa860ec8c"} May 21 01:04:05.528718 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:05.528434 2549 scope.go:117] "RemoveContainer" containerID="9e9e445d89b6fe833989fc5c763b1c9e3a0dcaf0579a35e1779b6540f6e14e22" May 21 01:04:05.537527 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:05.537508 2549 scope.go:117] "RemoveContainer" containerID="9e9e445d89b6fe833989fc5c763b1c9e3a0dcaf0579a35e1779b6540f6e14e22" May 21 01:04:05.537753 ip-10-0-138-101 kubenswrapper[2549]: E0521 01:04:05.537737 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e9e445d89b6fe833989fc5c763b1c9e3a0dcaf0579a35e1779b6540f6e14e22\": container with ID starting with 9e9e445d89b6fe833989fc5c763b1c9e3a0dcaf0579a35e1779b6540f6e14e22 not found: ID does not exist" containerID="9e9e445d89b6fe833989fc5c763b1c9e3a0dcaf0579a35e1779b6540f6e14e22" May 21 01:04:05.537811 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:05.537763 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e9e445d89b6fe833989fc5c763b1c9e3a0dcaf0579a35e1779b6540f6e14e22"} err="failed to get container status \"9e9e445d89b6fe833989fc5c763b1c9e3a0dcaf0579a35e1779b6540f6e14e22\": rpc error: code = NotFound desc = could not find container \"9e9e445d89b6fe833989fc5c763b1c9e3a0dcaf0579a35e1779b6540f6e14e22\": container with ID starting with 9e9e445d89b6fe833989fc5c763b1c9e3a0dcaf0579a35e1779b6540f6e14e22 not found: ID does not exist" May 21 01:04:05.550743 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:05.550719 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cldj6"] May 21 01:04:05.553745 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:05.553723 2549 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cldj6"] May 21 01:04:05.678484 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:05.678457 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ac94676-bdec-4ccc-96a0-d62de714064d" path="/var/lib/kubelet/pods/8ac94676-bdec-4ccc-96a0-d62de714064d/volumes" May 21 01:04:23.472375 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:23.472343 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-7b2fp"] May 21 01:04:23.472876 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:23.472837 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ac94676-bdec-4ccc-96a0-d62de714064d" containerName="manager" May 21 01:04:23.472876 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:23.472857 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ac94676-bdec-4ccc-96a0-d62de714064d" containerName="manager" May 21 01:04:23.473024 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:23.473001 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="8ac94676-bdec-4ccc-96a0-d62de714064d" containerName="manager" May 21 01:04:23.477860 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:23.477841 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-7b2fp" May 21 01:04:23.480821 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:23.480798 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" May 21 01:04:23.480968 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:23.480881 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-s8wn4\"" May 21 01:04:23.482648 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:23.482628 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-7b2fp"] May 21 01:04:23.502641 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:23.502614 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqrwx\" (UniqueName: \"kubernetes.io/projected/15174b01-e736-482c-835b-8668c5c79e0d-kube-api-access-hqrwx\") pod \"limitador-limitador-78c99df468-7b2fp\" (UID: \"15174b01-e736-482c-835b-8668c5c79e0d\") " pod="kuadrant-system/limitador-limitador-78c99df468-7b2fp" May 21 01:04:23.502829 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:23.502810 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/15174b01-e736-482c-835b-8668c5c79e0d-config-file\") pod \"limitador-limitador-78c99df468-7b2fp\" (UID: \"15174b01-e736-482c-835b-8668c5c79e0d\") " pod="kuadrant-system/limitador-limitador-78c99df468-7b2fp" May 21 01:04:23.504640 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:23.504621 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-7b2fp"] May 21 01:04:23.603858 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:23.603830 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/15174b01-e736-482c-835b-8668c5c79e0d-config-file\") pod \"limitador-limitador-78c99df468-7b2fp\" (UID: \"15174b01-e736-482c-835b-8668c5c79e0d\") " pod="kuadrant-system/limitador-limitador-78c99df468-7b2fp" May 21 01:04:23.603983 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:23.603940 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hqrwx\" (UniqueName: \"kubernetes.io/projected/15174b01-e736-482c-835b-8668c5c79e0d-kube-api-access-hqrwx\") pod \"limitador-limitador-78c99df468-7b2fp\" (UID: \"15174b01-e736-482c-835b-8668c5c79e0d\") " pod="kuadrant-system/limitador-limitador-78c99df468-7b2fp" May 21 01:04:23.604458 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:23.604439 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/15174b01-e736-482c-835b-8668c5c79e0d-config-file\") pod \"limitador-limitador-78c99df468-7b2fp\" (UID: \"15174b01-e736-482c-835b-8668c5c79e0d\") " pod="kuadrant-system/limitador-limitador-78c99df468-7b2fp" May 21 01:04:23.611516 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:23.611487 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqrwx\" (UniqueName: \"kubernetes.io/projected/15174b01-e736-482c-835b-8668c5c79e0d-kube-api-access-hqrwx\") pod \"limitador-limitador-78c99df468-7b2fp\" (UID: \"15174b01-e736-482c-835b-8668c5c79e0d\") " pod="kuadrant-system/limitador-limitador-78c99df468-7b2fp" May 21 01:04:23.790102 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:23.790076 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-7b2fp" May 21 01:04:23.922103 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:23.922078 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-7b2fp"] May 21 01:04:23.924420 ip-10-0-138-101 kubenswrapper[2549]: W0521 01:04:23.924390 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15174b01_e736_482c_835b_8668c5c79e0d.slice/crio-1ccb483b943f9d188d723e0e7d35ad07545fa378d1ee70de4b37080bfa442502 WatchSource:0}: Error finding container 1ccb483b943f9d188d723e0e7d35ad07545fa378d1ee70de4b37080bfa442502: Status 404 returned error can't find the container with id 1ccb483b943f9d188d723e0e7d35ad07545fa378d1ee70de4b37080bfa442502 May 21 01:04:24.603637 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:24.603598 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-7b2fp" event={"ID":"15174b01-e736-482c-835b-8668c5c79e0d","Type":"ContainerStarted","Data":"1ccb483b943f9d188d723e0e7d35ad07545fa378d1ee70de4b37080bfa442502"} May 21 01:04:26.612290 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:26.612209 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-7b2fp" event={"ID":"15174b01-e736-482c-835b-8668c5c79e0d","Type":"ContainerStarted","Data":"720a7c9e10538484acf49d1f76989ad70fce7277415355cfd58aefffb3e508fe"} May 21 01:04:26.612290 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:26.612285 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-78c99df468-7b2fp" May 21 01:04:26.628180 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:26.628135 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-78c99df468-7b2fp" podStartSLOduration=1.224291846 podStartE2EDuration="3.628124357s" podCreationTimestamp="2026-05-21 01:04:23 +0000 UTC" firstStartedPulling="2026-05-21 01:04:23.926162187 +0000 UTC m=+582.823107475" lastFinishedPulling="2026-05-21 01:04:26.329994693 +0000 UTC m=+585.226939986" observedRunningTime="2026-05-21 01:04:26.627007178 +0000 UTC m=+585.523952493" watchObservedRunningTime="2026-05-21 01:04:26.628124357 +0000 UTC m=+585.525069666" May 21 01:04:37.617121 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:37.617091 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-78c99df468-7b2fp" May 21 01:04:41.625046 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:41.625019 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgxp9_12a9408a-b799-4b3c-a790-a4d6e6d67cbd/ovn-acl-logging/0.log" May 21 01:04:41.625502 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:04:41.625484 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgxp9_12a9408a-b799-4b3c-a790-a4d6e6d67cbd/ovn-acl-logging/0.log" May 21 01:05:45.289815 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:05:45.289783 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-7b2fp"] May 21 01:06:05.580310 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:06:05.580275 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-7b2fp"] May 21 01:06:14.088434 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:06:14.088396 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-7b2fp"] May 21 01:06:31.480750 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:06:31.480719 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-7b2fp"] May 21 01:06:37.382179 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:06:37.382141 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-7b2fp"] May 21 01:06:46.583554 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:06:46.583524 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-7b2fp"] May 21 01:06:56.683462 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:06:56.683429 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-7b2fp"] May 21 01:07:49.879545 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:07:49.879516 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-7b2fp"] May 21 01:08:00.478223 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:08:00.478143 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-7b2fp"] May 21 01:08:09.286224 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:08:09.286185 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-7b2fp"] May 21 01:08:19.686755 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:08:19.686720 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-7b2fp"] May 21 01:08:28.083932 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:08:28.083899 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-7b2fp"] May 21 01:08:38.177948 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:08:38.177916 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-7b2fp"] May 21 01:09:40.381110 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:09:40.381080 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-7b2fp"] May 21 01:09:41.651908 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:09:41.651878 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgxp9_12a9408a-b799-4b3c-a790-a4d6e6d67cbd/ovn-acl-logging/0.log" May 21 01:09:41.653715 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:09:41.653695 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgxp9_12a9408a-b799-4b3c-a790-a4d6e6d67cbd/ovn-acl-logging/0.log" May 21 01:09:55.581016 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:09:55.580985 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-7b2fp"] May 21 01:10:34.279619 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:10:34.279584 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-7b2fp"] May 21 01:10:51.177240 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:10:51.177205 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-7b2fp"] May 21 01:11:04.288045 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:04.287963 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-7b2fp"] May 21 01:11:20.987733 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:20.987696 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-7b2fp"] May 21 01:11:38.013608 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:38.013577 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-fjfvj_45ed8e6a-dec5-4cf2-8b2c-b9d88aa53d0d/manager/0.log" May 21 01:11:38.352044 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:38.351963 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-ghjff_984a1d4b-7192-4318-accb-f02807829edb/manager/2.log" May 21 01:11:38.679743 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:38.679624 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6575999789-q7qn8_10b7eb4e-4524-4504-ad24-f5028c8a865c/manager/0.log" May 21 01:11:40.183958 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:40.183929 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-rr2dx_a25595d3-06ff-459a-9773-9204fa9e29d9/manager/0.log" May 21 01:11:40.507735 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:40.507706 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-qdhqk_7d65159b-310f-4c22-8d17-42a0a42a6e5a/manager/0.log" May 21 01:11:40.614174 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:40.614138 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-7b2fp_15174b01-e736-482c-835b-8668c5c79e0d/limitador/0.log" May 21 01:11:40.726422 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:40.726334 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-2b9zq_b7aa0746-9adf-4210-9a48-efc85cfde172/manager/0.log" May 21 01:11:41.166586 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:41.166557 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-5lsxv_56cee210-55b5-4da5-9766-455bb56d7cf8/discovery/0.log" May 21 01:11:41.372947 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:41.372920 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-857f4b55dc-tjqfl_d65b98da-1e62-477a-8bd9-9f5154fe16aa/kube-auth-proxy/0.log" May 21 01:11:41.695916 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:41.695883 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-685d6d5876-58bgk_6891f821-6e83-4a11-80c9-34bf3144c9ed/router/0.log" May 21 01:11:49.791389 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:49.791356 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-n4j7l_8f8b563f-d153-48f5-ab5e-836807f76cad/global-pull-secret-syncer/0.log" May 21 01:11:49.865596 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:49.865567 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-h7nwx_9ff27421-0ba9-40f6-98db-150fbcca1a5e/konnectivity-agent/0.log" May 21 01:11:49.955181 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:49.955146 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-101.ec2.internal_97c57dc15067251bbf5d4474732b6483/haproxy/0.log" May 21 01:11:54.004218 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:54.004180 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-rr2dx_a25595d3-06ff-459a-9773-9204fa9e29d9/manager/0.log" May 21 01:11:54.119792 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:54.119756 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-qdhqk_7d65159b-310f-4c22-8d17-42a0a42a6e5a/manager/0.log" May 21 01:11:54.137556 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:54.137534 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-7b2fp_15174b01-e736-482c-835b-8668c5c79e0d/limitador/0.log" May 21 01:11:54.173678 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:54.173641 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-2b9zq_b7aa0746-9adf-4210-9a48-efc85cfde172/manager/0.log" May 21 01:11:55.446723 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:55.446698 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f/alertmanager/0.log" May 21 01:11:55.472728 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:55.472703 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f/config-reloader/0.log" May 21 01:11:55.494130 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:55.494101 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f/kube-rbac-proxy-web/0.log" May 21 01:11:55.518483 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:55.518458 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f/kube-rbac-proxy/0.log" May 21 01:11:55.538070 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:55.538038 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f/kube-rbac-proxy-metric/0.log" May 21 01:11:55.556571 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:55.556553 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f/prom-label-proxy/0.log" May 21 01:11:55.575661 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:55.575642 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ada28e6c-5ee6-4f94-8e9b-63c1c527ab5f/init-config-reloader/0.log" May 21 01:11:55.633884 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:55.633856 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-m5nnw_ac23c315-d0c1-42b9-bb31-de85ad519f57/kube-state-metrics/0.log" May 21 01:11:55.650395 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:55.650371 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-m5nnw_ac23c315-d0c1-42b9-bb31-de85ad519f57/kube-rbac-proxy-main/0.log" May 21 01:11:55.669227 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:55.669200 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-m5nnw_ac23c315-d0c1-42b9-bb31-de85ad519f57/kube-rbac-proxy-self/0.log" May 21 01:11:55.697300 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:55.697248 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-bc594fcb8-5w7qf_f70c7d64-1725-43ff-8841-59691b5bf34d/metrics-server/0.log" May 21 01:11:55.835244 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:55.835214 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lwhqz_d20f6a20-17df-4d0a-ad91-379c6a7625b9/node-exporter/0.log" May 21 01:11:55.852450 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:55.852427 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lwhqz_d20f6a20-17df-4d0a-ad91-379c6a7625b9/kube-rbac-proxy/0.log" May 21 01:11:55.871427 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:55.871412 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lwhqz_d20f6a20-17df-4d0a-ad91-379c6a7625b9/init-textfile/0.log" May 21 01:11:56.097227 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:56.097201 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ff098155-6649-42d8-a482-ba283cbc30a3/prometheus/0.log" May 21 01:11:56.116428 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:56.116407 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ff098155-6649-42d8-a482-ba283cbc30a3/config-reloader/0.log" May 21 01:11:56.140152 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:56.140131 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ff098155-6649-42d8-a482-ba283cbc30a3/thanos-sidecar/0.log" May 21 01:11:56.164552 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:56.164532 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ff098155-6649-42d8-a482-ba283cbc30a3/kube-rbac-proxy-web/0.log" May 21 01:11:56.192120 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:56.192095 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ff098155-6649-42d8-a482-ba283cbc30a3/kube-rbac-proxy/0.log" May 21 01:11:56.216321 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:56.216303 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ff098155-6649-42d8-a482-ba283cbc30a3/kube-rbac-proxy-thanos/0.log" May 21 01:11:56.240168 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:56.240146 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ff098155-6649-42d8-a482-ba283cbc30a3/init-config-reloader/0.log" May 21 01:11:56.353970 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:56.353894 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6f9975d58-ddfdp_5402b5c0-1943-46b8-949f-009634baffb3/telemeter-client/0.log" May 21 01:11:56.374382 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:56.374358 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6f9975d58-ddfdp_5402b5c0-1943-46b8-949f-009634baffb3/reload/0.log" May 21 01:11:56.395400 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:56.395383 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6f9975d58-ddfdp_5402b5c0-1943-46b8-949f-009634baffb3/kube-rbac-proxy/0.log" May 21 01:11:56.425732 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:56.425710 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ff586c7c5-5ps4g_2d540d00-7ee2-42d1-b163-8344925e6439/thanos-query/0.log" May 21 01:11:56.446153 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:56.446123 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ff586c7c5-5ps4g_2d540d00-7ee2-42d1-b163-8344925e6439/kube-rbac-proxy-web/0.log" May 21 01:11:56.464835 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:56.464800 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ff586c7c5-5ps4g_2d540d00-7ee2-42d1-b163-8344925e6439/kube-rbac-proxy/0.log" May 21 01:11:56.483341 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:56.483325 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ff586c7c5-5ps4g_2d540d00-7ee2-42d1-b163-8344925e6439/prom-label-proxy/0.log" May 21 01:11:56.513527 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:56.513507 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ff586c7c5-5ps4g_2d540d00-7ee2-42d1-b163-8344925e6439/kube-rbac-proxy-rules/0.log" May 21 01:11:56.536899 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:56.536872 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ff586c7c5-5ps4g_2d540d00-7ee2-42d1-b163-8344925e6439/kube-rbac-proxy-metrics/0.log" May 21 01:11:57.765705 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:57.765674 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d4d8f94f9-cqdsk_86999352-f51c-47ff-9bc5-a62d9939a6ba/networking-console-plugin/0.log" May 21 01:11:58.813794 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:58.813766 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7z5t2/perf-node-gather-daemonset-wnj4w"] May 21 01:11:58.816647 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:58.816627 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7z5t2/perf-node-gather-daemonset-wnj4w" May 21 01:11:58.818998 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:58.818978 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7z5t2\"/\"kube-root-ca.crt\"" May 21 01:11:58.819863 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:58.819843 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7z5t2\"/\"openshift-service-ca.crt\"" May 21 01:11:58.819968 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:58.819862 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7z5t2\"/\"default-dockercfg-pjvps\"" May 21 01:11:58.823593 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:58.823573 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7z5t2/perf-node-gather-daemonset-wnj4w"] May 21 01:11:58.920755 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:58.920724 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nb8d\" (UniqueName: \"kubernetes.io/projected/016769f4-8bd8-4666-9ec2-9cddf184d6cc-kube-api-access-8nb8d\") pod \"perf-node-gather-daemonset-wnj4w\" (UID: \"016769f4-8bd8-4666-9ec2-9cddf184d6cc\") " pod="openshift-must-gather-7z5t2/perf-node-gather-daemonset-wnj4w" May 21 01:11:58.920896 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:58.920760 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/016769f4-8bd8-4666-9ec2-9cddf184d6cc-proc\") pod \"perf-node-gather-daemonset-wnj4w\" (UID: \"016769f4-8bd8-4666-9ec2-9cddf184d6cc\") " pod="openshift-must-gather-7z5t2/perf-node-gather-daemonset-wnj4w" May 21 01:11:58.920896 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:58.920801 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/016769f4-8bd8-4666-9ec2-9cddf184d6cc-podres\") pod \"perf-node-gather-daemonset-wnj4w\" (UID: \"016769f4-8bd8-4666-9ec2-9cddf184d6cc\") " pod="openshift-must-gather-7z5t2/perf-node-gather-daemonset-wnj4w" May 21 01:11:58.920896 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:58.920839 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/016769f4-8bd8-4666-9ec2-9cddf184d6cc-sys\") pod \"perf-node-gather-daemonset-wnj4w\" (UID: \"016769f4-8bd8-4666-9ec2-9cddf184d6cc\") " pod="openshift-must-gather-7z5t2/perf-node-gather-daemonset-wnj4w" May 21 01:11:58.920998 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:58.920920 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/016769f4-8bd8-4666-9ec2-9cddf184d6cc-lib-modules\") pod \"perf-node-gather-daemonset-wnj4w\" (UID: \"016769f4-8bd8-4666-9ec2-9cddf184d6cc\") " pod="openshift-must-gather-7z5t2/perf-node-gather-daemonset-wnj4w" May 21 01:11:59.021740 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:59.021711 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/016769f4-8bd8-4666-9ec2-9cddf184d6cc-sys\") pod \"perf-node-gather-daemonset-wnj4w\" (UID: \"016769f4-8bd8-4666-9ec2-9cddf184d6cc\") " pod="openshift-must-gather-7z5t2/perf-node-gather-daemonset-wnj4w" May 21 01:11:59.021882 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:59.021788 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/016769f4-8bd8-4666-9ec2-9cddf184d6cc-lib-modules\") pod \"perf-node-gather-daemonset-wnj4w\" (UID: \"016769f4-8bd8-4666-9ec2-9cddf184d6cc\") " pod="openshift-must-gather-7z5t2/perf-node-gather-daemonset-wnj4w" May 21 01:11:59.021882 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:59.021826 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/016769f4-8bd8-4666-9ec2-9cddf184d6cc-sys\") pod \"perf-node-gather-daemonset-wnj4w\" (UID: \"016769f4-8bd8-4666-9ec2-9cddf184d6cc\") " pod="openshift-must-gather-7z5t2/perf-node-gather-daemonset-wnj4w" May 21 01:11:59.021882 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:59.021841 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8nb8d\" (UniqueName: \"kubernetes.io/projected/016769f4-8bd8-4666-9ec2-9cddf184d6cc-kube-api-access-8nb8d\") pod \"perf-node-gather-daemonset-wnj4w\" (UID: \"016769f4-8bd8-4666-9ec2-9cddf184d6cc\") " pod="openshift-must-gather-7z5t2/perf-node-gather-daemonset-wnj4w" May 21 01:11:59.022035 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:59.021929 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/016769f4-8bd8-4666-9ec2-9cddf184d6cc-proc\") pod \"perf-node-gather-daemonset-wnj4w\" (UID: \"016769f4-8bd8-4666-9ec2-9cddf184d6cc\") " pod="openshift-must-gather-7z5t2/perf-node-gather-daemonset-wnj4w" May 21 01:11:59.022035 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:59.021976 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/016769f4-8bd8-4666-9ec2-9cddf184d6cc-podres\") pod \"perf-node-gather-daemonset-wnj4w\" (UID: \"016769f4-8bd8-4666-9ec2-9cddf184d6cc\") " pod="openshift-must-gather-7z5t2/perf-node-gather-daemonset-wnj4w" May 21 01:11:59.022035 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:59.021984 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/016769f4-8bd8-4666-9ec2-9cddf184d6cc-lib-modules\") pod \"perf-node-gather-daemonset-wnj4w\" (UID: \"016769f4-8bd8-4666-9ec2-9cddf184d6cc\") " pod="openshift-must-gather-7z5t2/perf-node-gather-daemonset-wnj4w" May 21 01:11:59.022221 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:59.022039 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/016769f4-8bd8-4666-9ec2-9cddf184d6cc-proc\") pod \"perf-node-gather-daemonset-wnj4w\" (UID: \"016769f4-8bd8-4666-9ec2-9cddf184d6cc\") " pod="openshift-must-gather-7z5t2/perf-node-gather-daemonset-wnj4w" May 21 01:11:59.022221 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:59.022154 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/016769f4-8bd8-4666-9ec2-9cddf184d6cc-podres\") pod \"perf-node-gather-daemonset-wnj4w\" (UID: \"016769f4-8bd8-4666-9ec2-9cddf184d6cc\") " pod="openshift-must-gather-7z5t2/perf-node-gather-daemonset-wnj4w" May 21 01:11:59.029431 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:59.029415 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nb8d\" (UniqueName: \"kubernetes.io/projected/016769f4-8bd8-4666-9ec2-9cddf184d6cc-kube-api-access-8nb8d\") pod \"perf-node-gather-daemonset-wnj4w\" (UID: \"016769f4-8bd8-4666-9ec2-9cddf184d6cc\") " pod="openshift-must-gather-7z5t2/perf-node-gather-daemonset-wnj4w" May 21 01:11:59.128744 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:59.128678 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7z5t2/perf-node-gather-daemonset-wnj4w" May 21 01:11:59.254540 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:59.254330 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7z5t2/perf-node-gather-daemonset-wnj4w"] May 21 01:11:59.256938 ip-10-0-138-101 kubenswrapper[2549]: W0521 01:11:59.256905 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod016769f4_8bd8_4666_9ec2_9cddf184d6cc.slice/crio-184a756d27dcefef8cd82c1bf572a4f5dc17d4e6cf9166669c8f8a394a1eaea0 WatchSource:0}: Error finding container 184a756d27dcefef8cd82c1bf572a4f5dc17d4e6cf9166669c8f8a394a1eaea0: Status 404 returned error can't find the container with id 184a756d27dcefef8cd82c1bf572a4f5dc17d4e6cf9166669c8f8a394a1eaea0 May 21 01:11:59.258796 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:11:59.258776 2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 21 01:12:00.231836 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:00.231808 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5mlhs_776a4785-ccb6-42c4-b6d8-050c42ad5541/dns/0.log" May 21 01:12:00.244469 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:00.244439 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7z5t2/perf-node-gather-daemonset-wnj4w" event={"ID":"016769f4-8bd8-4666-9ec2-9cddf184d6cc","Type":"ContainerStarted","Data":"8cd443b149347db51d0537ab36fe02d08b69a7835939a1cf9d311ec194a8ee0d"} May 21 01:12:00.244469 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:00.244472 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7z5t2/perf-node-gather-daemonset-wnj4w" event={"ID":"016769f4-8bd8-4666-9ec2-9cddf184d6cc","Type":"ContainerStarted","Data":"184a756d27dcefef8cd82c1bf572a4f5dc17d4e6cf9166669c8f8a394a1eaea0"} May 21 01:12:00.244661 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:00.244541 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7z5t2/perf-node-gather-daemonset-wnj4w" May 21 01:12:00.250544 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:00.250525 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5mlhs_776a4785-ccb6-42c4-b6d8-050c42ad5541/kube-rbac-proxy/0.log" May 21 01:12:00.258997 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:00.258958 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-7z5t2/perf-node-gather-daemonset-wnj4w" podStartSLOduration=2.258946244 podStartE2EDuration="2.258946244s" podCreationTimestamp="2026-05-21 01:11:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 01:12:00.257674854 +0000 UTC m=+1039.154620165" watchObservedRunningTime="2026-05-21 01:12:00.258946244 +0000 UTC m=+1039.155891556" May 21 01:12:00.385560 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:00.385534 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-96x9m_31374768-4e56-404b-8d22-7e2136138fa4/dns-node-resolver/0.log" May 21 01:12:00.898823 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:00.898791 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sfsvt_0a1071e0-9724-4546-8b54-44bce04c35c2/node-ca/0.log" May 21 01:12:01.765312 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:01.765255 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-5lsxv_56cee210-55b5-4da5-9766-455bb56d7cf8/discovery/0.log" May 21 01:12:01.806473 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:01.806443 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-857f4b55dc-tjqfl_d65b98da-1e62-477a-8bd9-9f5154fe16aa/kube-auth-proxy/0.log" May 21 01:12:01.883512 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:01.883487 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-685d6d5876-58bgk_6891f821-6e83-4a11-80c9-34bf3144c9ed/router/0.log" May 21 01:12:02.396217 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:02.396177 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2lhq8_877e95c2-b86f-49d3-a207-eb91f732bfff/serve-healthcheck-canary/0.log" May 21 01:12:02.942670 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:02.942644 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dpnq5_151cfde9-ff22-4152-9278-6971c624a383/kube-rbac-proxy/0.log" May 21 01:12:02.960447 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:02.960418 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dpnq5_151cfde9-ff22-4152-9278-6971c624a383/exporter/0.log" May 21 01:12:02.979313 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:02.979289 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dpnq5_151cfde9-ff22-4152-9278-6971c624a383/extractor/0.log" May 21 01:12:04.848883 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:04.848844 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-fjfvj_45ed8e6a-dec5-4cf2-8b2c-b9d88aa53d0d/manager/0.log" May 21 01:12:04.915612 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:04.915584 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-ghjff_984a1d4b-7192-4318-accb-f02807829edb/manager/1.log" May 21 01:12:04.935966 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:04.935943 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-ghjff_984a1d4b-7192-4318-accb-f02807829edb/manager/2.log" May 21 01:12:05.032116 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:05.032090 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6575999789-q7qn8_10b7eb4e-4524-4504-ad24-f5028c8a865c/manager/0.log" May 21 01:12:06.258662 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:06.258637 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7z5t2/perf-node-gather-daemonset-wnj4w" May 21 01:12:10.613938 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:10.613904 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-d8ttl_02d56ea6-c685-4f09-b951-40d453ba0a31/migrator/0.log" May 21 01:12:10.632445 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:10.632419 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-d8ttl_02d56ea6-c685-4f09-b951-40d453ba0a31/graceful-termination/0.log" May 21 01:12:12.198911 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:12.198883 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-spsr8_5da3aae9-eaf5-4328-afed-6595832fc2e1/kube-multus-additional-cni-plugins/0.log" May 21 01:12:12.217801 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:12.217776 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-spsr8_5da3aae9-eaf5-4328-afed-6595832fc2e1/egress-router-binary-copy/0.log" May 21 01:12:12.237475 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:12.237451 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-spsr8_5da3aae9-eaf5-4328-afed-6595832fc2e1/cni-plugins/0.log" May 21 01:12:12.256045 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:12.256028 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-spsr8_5da3aae9-eaf5-4328-afed-6595832fc2e1/bond-cni-plugin/0.log" May 21 01:12:12.273584 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:12.273569 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-spsr8_5da3aae9-eaf5-4328-afed-6595832fc2e1/routeoverride-cni/0.log" May 21 01:12:12.291719 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:12.291700 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-spsr8_5da3aae9-eaf5-4328-afed-6595832fc2e1/whereabouts-cni-bincopy/0.log" May 21 01:12:12.309877 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:12.309857 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-spsr8_5da3aae9-eaf5-4328-afed-6595832fc2e1/whereabouts-cni/0.log" May 21 01:12:12.369342 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:12.369321 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-sh44w_acd173d4-75da-4564-ac8e-84996bb1742d/kube-multus/0.log" May 21 01:12:12.434794 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:12.434773 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-skbch_5ee0c301-12eb-4c68-9990-6ceafb2c90fa/network-metrics-daemon/0.log" May 21 01:12:12.451426 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:12.451370 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-skbch_5ee0c301-12eb-4c68-9990-6ceafb2c90fa/kube-rbac-proxy/0.log" May 21 01:12:13.530643 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:13.530578 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgxp9_12a9408a-b799-4b3c-a790-a4d6e6d67cbd/ovn-controller/0.log" May 21 01:12:13.545838 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:13.545813 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgxp9_12a9408a-b799-4b3c-a790-a4d6e6d67cbd/ovn-acl-logging/0.log" May 21 01:12:13.555353 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:13.555326 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgxp9_12a9408a-b799-4b3c-a790-a4d6e6d67cbd/ovn-acl-logging/1.log" May 21 01:12:13.573464 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:13.573438 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgxp9_12a9408a-b799-4b3c-a790-a4d6e6d67cbd/kube-rbac-proxy-node/0.log" May 21 01:12:13.591881 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:13.591856 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgxp9_12a9408a-b799-4b3c-a790-a4d6e6d67cbd/kube-rbac-proxy-ovn-metrics/0.log" May 21 01:12:13.607483 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:13.607468 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgxp9_12a9408a-b799-4b3c-a790-a4d6e6d67cbd/northd/0.log" May 21 01:12:13.625275 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:13.625253 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgxp9_12a9408a-b799-4b3c-a790-a4d6e6d67cbd/nbdb/0.log" May 21 01:12:13.643913 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:13.643894 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgxp9_12a9408a-b799-4b3c-a790-a4d6e6d67cbd/sbdb/0.log" May 21 01:12:13.805568 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:13.805492 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgxp9_12a9408a-b799-4b3c-a790-a4d6e6d67cbd/ovnkube-controller/0.log" May 21 01:12:15.109776 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:15.109740 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7f44b6dd87-xvxlc_88201346-80c9-4e71-8c22-d42722339e1f/check-endpoints/0.log" May 21 01:12:15.175663 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:15.175638 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-rxzxp_3b778703-acf8-4977-a1b1-2f5f9042d2b3/network-check-target-container/0.log" May 21 01:12:16.176173 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:16.176145 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-qdnd9_7d5e55de-c445-48f6-8911-e4f1a11104de/iptables-alerter/0.log" May 21 01:12:16.834285 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:16.834241 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-gvmxz_96547212-630f-43e2-9662-5938454d35ec/tuned/0.log" May 21 01:12:19.953363 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:19.953334 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca_service-ca-67d667c7d-pbjgz_99084a94-3345-4362-86d2-f9936139c30f/service-ca-controller/0.log" May 21 01:12:20.486890 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:20.486852 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-vvqd9_8adbdb60-a625-47a1-8252-779cd35e6a35/csi-driver/0.log" May 21 01:12:20.505286 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:20.505259 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-vvqd9_8adbdb60-a625-47a1-8252-779cd35e6a35/csi-node-driver-registrar/0.log" May 21 01:12:20.522989 ip-10-0-138-101 kubenswrapper[2549]: I0521 01:12:20.522967 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-vvqd9_8adbdb60-a625-47a1-8252-779cd35e6a35/csi-liveness-probe/0.log"