Sep 08 03:53:36.791856 ip-10-0-128-141 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 08 03:53:36.791868 ip-10-0-128-141 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 08 03:53:36.791877 ip-10-0-128-141 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 08 03:53:36.792061 ip-10-0-128-141 systemd[1]: Failed to start Kubernetes Kubelet. Sep 08 03:53:46.999673 ip-10-0-128-141 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 08 03:53:46.999692 ip-10-0-128-141 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 1910edc6b2194d589ca27a2287b42494 -- Sep 08 03:55:38.333054 ip-10-0-128-141 systemd[1]: Starting Kubernetes Kubelet... Sep 08 03:55:38.727270 ip-10-0-128-141 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 08 03:55:38.727270 ip-10-0-128-141 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 08 03:55:38.727270 ip-10-0-128-141 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 08 03:55:38.727270 ip-10-0-128-141 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 08 03:55:38.727270 ip-10-0-128-141 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 08 03:55:38.730085 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.730000 2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 08 03:55:38.733707 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733688 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 03:55:38.733707 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733704 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 03:55:38.733707 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733709 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 03:55:38.733912 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733714 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 03:55:38.733912 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733719 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 03:55:38.733912 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733724 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 03:55:38.733912 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733728 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 03:55:38.733912 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733732 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 03:55:38.733912 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733736 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 03:55:38.733912 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733740 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 03:55:38.733912 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733745 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 03:55:38.733912 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733748 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 03:55:38.733912 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733752 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 03:55:38.733912 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733757 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 03:55:38.733912 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733761 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 03:55:38.733912 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733778 2579 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 03:55:38.733912 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733783 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 03:55:38.733912 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733786 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 03:55:38.733912 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733790 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 03:55:38.733912 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733794 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 03:55:38.733912 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733798 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 03:55:38.733912 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733802 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 03:55:38.733912 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733805 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 03:55:38.734542 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733810 2579 feature_gate.go:328] unrecognized feature gate: Example Sep 08 03:55:38.734542 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733815 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 03:55:38.734542 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733819 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 03:55:38.734542 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733823 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 03:55:38.734542 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733829 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 03:55:38.734542 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733835 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 03:55:38.734542 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733840 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 03:55:38.734542 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733844 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 03:55:38.734542 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733850 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 03:55:38.734542 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733856 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 03:55:38.734542 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733861 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 03:55:38.734542 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733866 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 03:55:38.734542 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733871 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 03:55:38.734542 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733876 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 03:55:38.734542 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733880 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 03:55:38.734542 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733885 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 03:55:38.734542 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733890 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 03:55:38.734542 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733895 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 03:55:38.734542 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733900 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 03:55:38.735020 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733904 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 03:55:38.735020 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733908 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 03:55:38.735020 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733913 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 03:55:38.735020 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733917 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 03:55:38.735020 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733921 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 03:55:38.735020 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733925 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 03:55:38.735020 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733931 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 03:55:38.735020 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733935 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 03:55:38.735020 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733939 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 03:55:38.735020 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733943 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 03:55:38.735020 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733947 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 03:55:38.735020 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733952 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 03:55:38.735020 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733956 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 03:55:38.735020 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733960 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 03:55:38.735020 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733964 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 03:55:38.735020 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733968 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 03:55:38.735020 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733972 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 03:55:38.735020 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733976 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 03:55:38.735020 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733980 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 03:55:38.735020 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733984 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 03:55:38.735826 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733988 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 03:55:38.735826 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733992 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 03:55:38.735826 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.733997 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 03:55:38.735826 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734001 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 03:55:38.735826 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734005 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 03:55:38.735826 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734009 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 03:55:38.735826 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734014 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 03:55:38.735826 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734018 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 03:55:38.735826 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734022 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 03:55:38.735826 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734027 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 03:55:38.735826 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734032 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 03:55:38.735826 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734036 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 03:55:38.735826 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734041 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 03:55:38.735826 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734045 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 03:55:38.735826 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734049 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 03:55:38.735826 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734053 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 03:55:38.735826 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734058 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 03:55:38.735826 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734062 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 03:55:38.735826 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734066 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 03:55:38.735826 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734070 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 03:55:38.736664 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734074 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 03:55:38.736664 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734081 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 03:55:38.736664 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734089 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 03:55:38.736664 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734094 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 03:55:38.736664 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734098 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 03:55:38.736664 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734102 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 03:55:38.736664 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734107 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 03:55:38.736664 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734111 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 03:55:38.736664 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734115 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 03:55:38.736664 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734120 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 03:55:38.736664 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734124 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 03:55:38.736664 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734128 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 03:55:38.736664 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734132 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 03:55:38.736664 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734137 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 03:55:38.736664 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734141 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 03:55:38.736664 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734144 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 03:55:38.736664 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.734149 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 03:55:38.736664 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736285 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 03:55:38.736664 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736297 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 03:55:38.737475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736302 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 03:55:38.737475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736307 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 03:55:38.737475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736311 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 03:55:38.737475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736316 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 03:55:38.737475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736320 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 03:55:38.737475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736324 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 03:55:38.737475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736329 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 03:55:38.737475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736333 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 03:55:38.737475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736337 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 03:55:38.737475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736341 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 03:55:38.737475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736346 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 03:55:38.737475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736350 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 03:55:38.737475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736355 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 03:55:38.737475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736359 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 03:55:38.737475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736363 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 03:55:38.737475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736366 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 03:55:38.737475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736370 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 03:55:38.737475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736374 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 03:55:38.737475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736379 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 03:55:38.737475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736383 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 03:55:38.738369 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736387 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 03:55:38.738369 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736392 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 03:55:38.738369 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736396 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 03:55:38.738369 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736401 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 03:55:38.738369 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736405 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 03:55:38.738369 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736409 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 03:55:38.738369 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736414 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 03:55:38.738369 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736418 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 03:55:38.738369 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736423 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 03:55:38.738369 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736427 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 03:55:38.738369 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736431 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 03:55:38.738369 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736436 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 03:55:38.738369 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736440 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 03:55:38.738369 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736444 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 03:55:38.738369 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736449 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 03:55:38.738369 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736452 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 03:55:38.738369 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736457 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 03:55:38.738369 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736462 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 03:55:38.738369 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736466 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 03:55:38.738369 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736471 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 03:55:38.739249 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736475 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 03:55:38.739249 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736479 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 03:55:38.739249 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736484 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 03:55:38.739249 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736489 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 03:55:38.739249 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736493 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 03:55:38.739249 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736497 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 03:55:38.739249 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736502 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 03:55:38.739249 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736506 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 03:55:38.739249 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736510 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 03:55:38.739249 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736515 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 03:55:38.739249 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736519 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 03:55:38.739249 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736523 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 03:55:38.739249 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736530 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 03:55:38.739249 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736536 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 03:55:38.739249 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736541 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 03:55:38.739249 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736545 2579 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 03:55:38.739249 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736550 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 03:55:38.739249 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736554 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 03:55:38.739249 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736559 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 03:55:38.739872 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736563 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 03:55:38.739872 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736567 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 03:55:38.739872 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736572 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 03:55:38.739872 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736576 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 03:55:38.739872 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736580 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 03:55:38.739872 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736584 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 03:55:38.739872 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736588 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 03:55:38.739872 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736592 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 03:55:38.739872 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736596 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 03:55:38.739872 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736600 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 03:55:38.739872 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736605 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 03:55:38.739872 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736612 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 03:55:38.739872 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736616 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 03:55:38.739872 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736620 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 03:55:38.739872 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736624 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 03:55:38.739872 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736628 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 03:55:38.739872 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736632 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 03:55:38.739872 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736637 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 03:55:38.739872 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736641 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 03:55:38.739872 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736645 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 03:55:38.740402 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736649 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 03:55:38.740402 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736654 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 03:55:38.740402 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736658 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 03:55:38.740402 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736662 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 03:55:38.740402 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736668 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 03:55:38.740402 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736673 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 03:55:38.740402 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736678 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 03:55:38.740402 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736683 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 03:55:38.740402 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736687 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 03:55:38.740402 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736691 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 03:55:38.740402 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736696 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 03:55:38.740402 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736700 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 03:55:38.740402 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736704 2579 feature_gate.go:328] unrecognized feature gate: Example Sep 08 03:55:38.740402 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736709 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 03:55:38.740402 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736715 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 03:55:38.740402 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736721 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 03:55:38.740402 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736725 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 03:55:38.740402 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.736729 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 03:55:38.740402 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737384 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 03:55:38.740402 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737393 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 03:55:38.740994 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737399 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 03:55:38.740994 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737404 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 03:55:38.740994 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737413 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 03:55:38.740994 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737418 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 03:55:38.740994 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737422 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 03:55:38.740994 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737426 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 03:55:38.740994 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737430 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 03:55:38.740994 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737434 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 03:55:38.740994 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737439 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 03:55:38.740994 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737444 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 03:55:38.740994 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737448 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 03:55:38.740994 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737453 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 03:55:38.740994 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737457 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 03:55:38.740994 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737461 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 03:55:38.740994 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737466 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 03:55:38.740994 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737470 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 03:55:38.740994 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737473 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 03:55:38.740994 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737477 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 03:55:38.740994 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737481 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 03:55:38.740994 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737485 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 03:55:38.741504 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737489 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 03:55:38.741504 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737493 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 03:55:38.741504 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737498 2579 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 03:55:38.741504 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737502 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 03:55:38.741504 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737506 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 03:55:38.741504 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737510 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 03:55:38.741504 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737515 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 03:55:38.741504 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737519 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 03:55:38.741504 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737539 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 03:55:38.741504 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737544 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 03:55:38.741504 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737548 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 03:55:38.741504 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737553 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 03:55:38.741504 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737557 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 03:55:38.741504 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737561 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 03:55:38.741504 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737566 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 03:55:38.741504 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737571 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 03:55:38.741504 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737575 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 03:55:38.741504 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737579 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 03:55:38.741504 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737583 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 03:55:38.741504 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737587 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 03:55:38.742203 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737592 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 03:55:38.742203 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737597 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 03:55:38.742203 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737601 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 03:55:38.742203 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737605 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 03:55:38.742203 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737609 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 03:55:38.742203 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737614 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 03:55:38.742203 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737618 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 03:55:38.742203 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737622 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 03:55:38.742203 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737625 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 03:55:38.742203 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737630 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 03:55:38.742203 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737634 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 03:55:38.742203 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737638 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 03:55:38.742203 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737643 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 03:55:38.742203 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737647 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 03:55:38.742203 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737650 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 03:55:38.742203 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737655 2579 feature_gate.go:328] unrecognized feature gate: Example Sep 08 03:55:38.742203 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737659 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 03:55:38.742203 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737663 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 03:55:38.742203 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737667 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 03:55:38.742203 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737671 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 03:55:38.742703 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737676 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 03:55:38.742703 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737681 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 03:55:38.742703 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737685 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 03:55:38.742703 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737689 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 03:55:38.742703 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737693 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 03:55:38.742703 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737704 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 03:55:38.742703 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737709 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 03:55:38.742703 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737716 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 03:55:38.742703 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737722 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 03:55:38.742703 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737727 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 03:55:38.742703 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737732 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 03:55:38.742703 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737739 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 03:55:38.742703 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737744 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 03:55:38.742703 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737750 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 03:55:38.742703 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737754 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 03:55:38.742703 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737759 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 03:55:38.742703 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737763 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 03:55:38.742703 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737783 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 03:55:38.742703 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737787 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 03:55:38.743299 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737791 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 03:55:38.743299 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737796 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 03:55:38.743299 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737800 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 03:55:38.743299 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737804 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 03:55:38.743299 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737808 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 03:55:38.743299 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737812 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 03:55:38.743299 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737816 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 03:55:38.743299 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737820 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 03:55:38.743299 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737824 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 03:55:38.743299 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737828 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 03:55:38.743299 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737832 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 03:55:38.743299 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737836 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 03:55:38.743299 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737841 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 03:55:38.743299 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737846 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 03:55:38.743299 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737853 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 03:55:38.743299 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737857 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 03:55:38.743299 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737861 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 03:55:38.743299 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.737865 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 03:55:38.743299 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.737960 2579 flags.go:64] FLAG: --address="0.0.0.0" Sep 08 03:55:38.743299 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738020 2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 08 03:55:38.743832 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738029 2579 flags.go:64] FLAG: --anonymous-auth="true" Sep 08 03:55:38.743832 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738036 2579 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 08 03:55:38.743832 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738044 2579 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 08 03:55:38.743832 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738049 2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 08 03:55:38.743832 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738056 2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 08 03:55:38.743832 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738062 2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 08 03:55:38.743832 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738067 2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 08 03:55:38.743832 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738072 2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 08 03:55:38.743832 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738077 2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 08 03:55:38.743832 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738083 2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 08 03:55:38.743832 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738088 2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 08 03:55:38.743832 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738093 2579 flags.go:64] FLAG: --cgroup-root="" Sep 08 03:55:38.743832 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738098 2579 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 08 03:55:38.743832 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738102 2579 flags.go:64] FLAG: --client-ca-file="" Sep 08 03:55:38.743832 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738107 2579 flags.go:64] FLAG: --cloud-provider="external" Sep 08 03:55:38.743832 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738135 2579 flags.go:64] FLAG: --cluster-dns="[]" Sep 08 03:55:38.743832 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738144 2579 flags.go:64] FLAG: --cluster-domain="" Sep 08 03:55:38.743832 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738149 2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 08 03:55:38.743832 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738154 2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 08 03:55:38.743832 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738159 2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 08 03:55:38.743832 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738164 2579 flags.go:64] FLAG: --container-log-max-files="5" Sep 08 03:55:38.743832 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738170 2579 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 08 03:55:38.743832 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738177 2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 08 03:55:38.744402 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738182 2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 08 03:55:38.744402 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738187 2579 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 08 03:55:38.744402 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738192 2579 flags.go:64] FLAG: --contention-profiling="false" Sep 08 03:55:38.744402 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738196 2579 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 08 03:55:38.744402 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738201 2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 08 03:55:38.744402 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738207 2579 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 08 03:55:38.744402 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738212 2579 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 08 03:55:38.744402 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738220 2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 08 03:55:38.744402 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738225 2579 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 08 03:55:38.744402 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738229 2579 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 08 03:55:38.744402 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738234 2579 flags.go:64] FLAG: --enable-load-reader="false" Sep 08 03:55:38.744402 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738239 2579 flags.go:64] FLAG: --enable-server="true" Sep 08 03:55:38.744402 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738244 2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 08 03:55:38.744402 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738252 2579 flags.go:64] FLAG: --event-burst="100" Sep 08 03:55:38.744402 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738257 2579 flags.go:64] FLAG: --event-qps="50" Sep 08 03:55:38.744402 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738262 2579 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 08 03:55:38.744402 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738267 2579 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 08 03:55:38.744402 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738272 2579 flags.go:64] FLAG: --eviction-hard="" Sep 08 03:55:38.744402 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738278 2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 08 03:55:38.744402 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738282 2579 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 08 03:55:38.744402 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738287 2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 08 03:55:38.744402 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738293 2579 flags.go:64] FLAG: --eviction-soft="" Sep 08 03:55:38.744402 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738297 2579 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 08 03:55:38.744402 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738302 2579 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 08 03:55:38.744402 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738307 2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 08 03:55:38.745138 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738311 2579 flags.go:64] FLAG: --experimental-mounter-path="" Sep 08 03:55:38.745138 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738316 2579 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 08 03:55:38.745138 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738321 2579 flags.go:64] FLAG: --fail-swap-on="true" Sep 08 03:55:38.745138 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738325 2579 flags.go:64] FLAG: --feature-gates="" Sep 08 03:55:38.745138 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738331 2579 flags.go:64] FLAG: --file-check-frequency="20s" Sep 08 03:55:38.745138 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738336 2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 08 03:55:38.745138 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738341 2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 08 03:55:38.745138 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738346 2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 08 03:55:38.745138 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738352 2579 flags.go:64] FLAG: --healthz-port="10248" Sep 08 03:55:38.745138 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738356 2579 flags.go:64] FLAG: --help="false" Sep 08 03:55:38.745138 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738361 2579 flags.go:64] FLAG: --hostname-override="ip-10-0-128-141.ec2.internal" Sep 08 03:55:38.745138 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738366 2579 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 08 03:55:38.745138 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738370 2579 flags.go:64] FLAG: --http-check-frequency="20s" Sep 08 03:55:38.745138 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738375 2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 08 03:55:38.745138 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738382 2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 08 03:55:38.745138 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738388 2579 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 08 03:55:38.745138 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738393 2579 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 08 03:55:38.745138 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738397 2579 flags.go:64] FLAG: --image-service-endpoint="" Sep 08 03:55:38.745138 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738403 2579 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 08 03:55:38.745138 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738408 2579 flags.go:64] FLAG: --kube-api-burst="100" Sep 08 03:55:38.745138 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738415 2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 08 03:55:38.745138 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738421 2579 flags.go:64] FLAG: --kube-api-qps="50" Sep 08 03:55:38.745138 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738425 2579 flags.go:64] FLAG: --kube-reserved="" Sep 08 03:55:38.745138 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738430 2579 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 08 03:55:38.745783 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738435 2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 08 03:55:38.745783 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738440 2579 flags.go:64] FLAG: --kubelet-cgroups="" Sep 08 03:55:38.745783 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738445 2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 08 03:55:38.745783 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738449 2579 flags.go:64] FLAG: --lock-file="" Sep 08 03:55:38.745783 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738454 2579 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 08 03:55:38.745783 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738459 2579 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 08 03:55:38.745783 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738464 2579 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 08 03:55:38.745783 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738473 2579 flags.go:64] FLAG: --log-json-split-stream="false" Sep 08 03:55:38.745783 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738477 2579 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 08 03:55:38.745783 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738482 2579 flags.go:64] FLAG: --log-text-split-stream="false" Sep 08 03:55:38.745783 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738487 2579 flags.go:64] FLAG: --logging-format="text" Sep 08 03:55:38.745783 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738492 2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 08 03:55:38.745783 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738498 2579 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 08 03:55:38.745783 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738502 2579 flags.go:64] FLAG: --manifest-url="" Sep 08 03:55:38.745783 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738507 2579 flags.go:64] FLAG: --manifest-url-header="" Sep 08 03:55:38.745783 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738513 2579 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 08 03:55:38.745783 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738518 2579 flags.go:64] FLAG: --max-open-files="1000000" Sep 08 03:55:38.745783 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738524 2579 flags.go:64] FLAG: --max-pods="110" Sep 08 03:55:38.745783 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738530 2579 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 08 03:55:38.745783 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738535 2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 08 03:55:38.745783 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738539 2579 flags.go:64] FLAG: --memory-manager-policy="None" Sep 08 03:55:38.745783 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738544 2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 08 03:55:38.745783 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738551 2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 08 03:55:38.745783 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738556 2579 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 08 03:55:38.745783 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738560 2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 08 03:55:38.746428 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738571 2579 flags.go:64] FLAG: --node-status-max-images="50" Sep 08 03:55:38.746428 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738576 2579 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 08 03:55:38.746428 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738581 2579 flags.go:64] FLAG: --oom-score-adj="-999" Sep 08 03:55:38.746428 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738587 2579 flags.go:64] FLAG: --pod-cidr="" Sep 08 03:55:38.746428 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738593 2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 08 03:55:38.746428 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738601 2579 flags.go:64] FLAG: --pod-manifest-path="" Sep 08 03:55:38.746428 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738606 2579 flags.go:64] FLAG: --pod-max-pids="-1" Sep 08 03:55:38.746428 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738611 2579 flags.go:64] FLAG: --pods-per-core="0" Sep 08 03:55:38.746428 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738615 2579 flags.go:64] FLAG: --port="10250" Sep 08 03:55:38.746428 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738621 2579 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 08 03:55:38.746428 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738625 2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0340698a28bfde65a" Sep 08 03:55:38.746428 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738630 2579 flags.go:64] FLAG: --qos-reserved="" Sep 08 03:55:38.746428 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738635 2579 flags.go:64] FLAG: --read-only-port="10255" Sep 08 03:55:38.746428 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738640 2579 flags.go:64] FLAG: --register-node="true" Sep 08 03:55:38.746428 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738645 2579 flags.go:64] FLAG: --register-with-taints="" Sep 08 03:55:38.746428 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738650 2579 flags.go:64] FLAG: --registry-burst="10" Sep 08 03:55:38.746428 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738655 2579 flags.go:64] FLAG: --registry-qps="5" Sep 08 03:55:38.746428 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738660 2579 flags.go:64] FLAG: --reserved-cpus="" Sep 08 03:55:38.746428 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738664 2579 flags.go:64] FLAG: --reserved-memory="" Sep 08 03:55:38.746428 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738670 2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 08 03:55:38.746428 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738675 2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 08 03:55:38.746428 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738680 2579 flags.go:64] FLAG: --rotate-certificates="false" Sep 08 03:55:38.746428 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738684 2579 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 08 03:55:38.746428 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738689 2579 flags.go:64] FLAG: --runonce="false" Sep 08 03:55:38.746428 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738694 2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 08 03:55:38.747105 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738699 2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 08 03:55:38.747105 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738704 2579 flags.go:64] FLAG: --seccomp-default="false" Sep 08 03:55:38.747105 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738709 2579 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 08 03:55:38.747105 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738713 2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 08 03:55:38.747105 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738720 2579 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 08 03:55:38.747105 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738725 2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 08 03:55:38.747105 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738729 2579 flags.go:64] FLAG: --storage-driver-password="root" Sep 08 03:55:38.747105 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738734 2579 flags.go:64] FLAG: --storage-driver-secure="false" Sep 08 03:55:38.747105 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738739 2579 flags.go:64] FLAG: --storage-driver-table="stats" Sep 08 03:55:38.747105 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738743 2579 flags.go:64] FLAG: --storage-driver-user="root" Sep 08 03:55:38.747105 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738748 2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 08 03:55:38.747105 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738755 2579 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 08 03:55:38.747105 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738761 2579 flags.go:64] FLAG: --system-cgroups="" Sep 08 03:55:38.747105 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738783 2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 08 03:55:38.747105 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738793 2579 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 08 03:55:38.747105 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738798 2579 flags.go:64] FLAG: --tls-cert-file="" Sep 08 03:55:38.747105 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738802 2579 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 08 03:55:38.747105 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738810 2579 flags.go:64] FLAG: --tls-min-version="" Sep 08 03:55:38.747105 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738815 2579 flags.go:64] FLAG: --tls-private-key-file="" Sep 08 03:55:38.747105 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738819 2579 flags.go:64] FLAG: --topology-manager-policy="none" Sep 08 03:55:38.747105 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738824 2579 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 08 03:55:38.747105 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738829 2579 flags.go:64] FLAG: --topology-manager-scope="container" Sep 08 03:55:38.747105 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738834 2579 flags.go:64] FLAG: --v="2" Sep 08 03:55:38.747105 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738840 2579 flags.go:64] FLAG: --version="false" Sep 08 03:55:38.747105 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738846 2579 flags.go:64] FLAG: --vmodule="" Sep 08 03:55:38.747732 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738852 2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 08 03:55:38.747732 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.738858 2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 08 03:55:38.747732 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739058 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 03:55:38.747732 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739064 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 03:55:38.747732 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739069 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 03:55:38.747732 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739073 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 03:55:38.747732 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739077 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 03:55:38.747732 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739081 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 03:55:38.747732 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739085 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 03:55:38.747732 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739089 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 03:55:38.747732 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739093 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 03:55:38.747732 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739102 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 03:55:38.747732 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739108 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 03:55:38.747732 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739112 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 03:55:38.747732 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739119 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 03:55:38.747732 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739124 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 03:55:38.747732 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739130 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 03:55:38.747732 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739135 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 03:55:38.747732 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739142 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 03:55:38.748224 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739146 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 03:55:38.748224 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739151 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 03:55:38.748224 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739155 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 03:55:38.748224 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739160 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 03:55:38.748224 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739164 2579 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 03:55:38.748224 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739168 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 03:55:38.748224 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739172 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 03:55:38.748224 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739176 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 03:55:38.748224 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739180 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 03:55:38.748224 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739185 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 03:55:38.748224 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739189 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 03:55:38.748224 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739193 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 03:55:38.748224 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739197 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 03:55:38.748224 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739201 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 03:55:38.748224 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739205 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 03:55:38.748224 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739209 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 03:55:38.748224 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739213 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 03:55:38.748224 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739217 2579 feature_gate.go:328] unrecognized feature gate: Example Sep 08 03:55:38.748224 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739221 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 03:55:38.748224 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739225 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 03:55:38.748710 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739230 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 03:55:38.748710 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739234 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 03:55:38.748710 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739238 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 03:55:38.748710 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739242 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 03:55:38.748710 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739250 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 03:55:38.748710 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739255 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 03:55:38.748710 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739260 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 03:55:38.748710 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739264 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 03:55:38.748710 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739268 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 03:55:38.748710 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739273 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 03:55:38.748710 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739277 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 03:55:38.748710 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739283 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 03:55:38.748710 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739288 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 03:55:38.748710 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739293 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 03:55:38.748710 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739298 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 03:55:38.748710 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739302 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 03:55:38.748710 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739306 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 03:55:38.748710 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739311 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 03:55:38.748710 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739315 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 03:55:38.749201 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739319 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 03:55:38.749201 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739323 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 03:55:38.749201 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739328 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 03:55:38.749201 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739332 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 03:55:38.749201 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739336 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 03:55:38.749201 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739340 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 03:55:38.749201 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739344 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 03:55:38.749201 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739349 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 03:55:38.749201 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739353 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 03:55:38.749201 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739357 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 03:55:38.749201 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739360 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 03:55:38.749201 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739365 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 03:55:38.749201 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739369 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 03:55:38.749201 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739374 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 03:55:38.749201 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739377 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 03:55:38.749201 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739382 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 03:55:38.749201 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739386 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 03:55:38.749201 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739391 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 03:55:38.749201 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739395 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 03:55:38.749201 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739399 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 03:55:38.749692 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739403 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 03:55:38.749692 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739407 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 03:55:38.749692 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739411 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 03:55:38.749692 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739415 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 03:55:38.749692 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739422 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 03:55:38.749692 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739426 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 03:55:38.749692 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739430 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 03:55:38.749692 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739434 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 03:55:38.749692 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739438 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 03:55:38.749692 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739443 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 03:55:38.749692 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739447 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 03:55:38.749692 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739450 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 03:55:38.749692 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739455 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 03:55:38.749692 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739459 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 03:55:38.749692 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739463 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 03:55:38.749692 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739467 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 03:55:38.749692 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739471 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 03:55:38.749692 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739475 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 03:55:38.749692 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739479 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 03:55:38.749692 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739483 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 03:55:38.750269 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739487 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 03:55:38.750269 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739491 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 03:55:38.750269 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.739496 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 03:55:38.750269 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.740344 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 08 03:55:38.750269 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.747688 2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 08 03:55:38.750269 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.747707 2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 08 03:55:38.750269 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.747723 2579 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 08 03:55:38.750269 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.747728 2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 08 03:55:38.750269 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747821 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 03:55:38.750269 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747827 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 03:55:38.750269 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747830 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 03:55:38.750269 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747833 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 03:55:38.750269 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747836 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 03:55:38.750269 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747839 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 03:55:38.750269 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747841 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 03:55:38.750653 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747844 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 03:55:38.750653 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747847 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 03:55:38.750653 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747849 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 03:55:38.750653 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747852 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 03:55:38.750653 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747854 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 03:55:38.750653 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747857 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 03:55:38.750653 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747860 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 03:55:38.750653 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747862 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 03:55:38.750653 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747865 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 03:55:38.750653 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747867 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 03:55:38.750653 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747870 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 03:55:38.750653 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747872 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 03:55:38.750653 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747875 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 03:55:38.750653 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747878 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 03:55:38.750653 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747880 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 03:55:38.750653 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747883 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 03:55:38.750653 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747885 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 03:55:38.750653 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747888 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 03:55:38.750653 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747890 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 03:55:38.750653 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747893 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 03:55:38.751191 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747896 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 03:55:38.751191 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747899 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 03:55:38.751191 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747908 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 03:55:38.751191 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747910 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 03:55:38.751191 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747913 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 03:55:38.751191 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747915 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 03:55:38.751191 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747918 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 03:55:38.751191 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747921 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 03:55:38.751191 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747923 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 03:55:38.751191 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747926 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 03:55:38.751191 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747928 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 03:55:38.751191 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747931 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 03:55:38.751191 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747933 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 03:55:38.751191 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747936 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 03:55:38.751191 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747938 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 03:55:38.751191 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747941 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 03:55:38.751191 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747943 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 03:55:38.751191 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747946 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 03:55:38.751191 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747948 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 03:55:38.751649 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747951 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 03:55:38.751649 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747953 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 03:55:38.751649 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747956 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 03:55:38.751649 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747958 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 03:55:38.751649 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747961 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 03:55:38.751649 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747964 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 03:55:38.751649 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747968 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 03:55:38.751649 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747972 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 03:55:38.751649 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747974 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 03:55:38.751649 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747977 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 03:55:38.751649 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747980 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 03:55:38.751649 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747983 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 03:55:38.751649 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747986 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 03:55:38.751649 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747989 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 03:55:38.751649 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747992 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 03:55:38.751649 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.747995 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 03:55:38.751649 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748003 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 03:55:38.751649 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748006 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 03:55:38.751649 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748009 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 03:55:38.751649 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748011 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 03:55:38.752152 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748014 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 03:55:38.752152 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748017 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 03:55:38.752152 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748019 2579 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 03:55:38.752152 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748022 2579 feature_gate.go:328] unrecognized feature gate: Example Sep 08 03:55:38.752152 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748024 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 03:55:38.752152 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748027 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 03:55:38.752152 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748030 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 03:55:38.752152 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748032 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 03:55:38.752152 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748035 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 03:55:38.752152 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748037 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 03:55:38.752152 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748040 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 03:55:38.752152 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748042 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 03:55:38.752152 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748045 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 03:55:38.752152 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748049 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 03:55:38.752152 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748052 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 03:55:38.752152 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748055 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 03:55:38.752152 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748057 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 03:55:38.752152 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748060 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 03:55:38.752152 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748063 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 03:55:38.752152 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748065 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 03:55:38.752647 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748068 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 03:55:38.752647 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748071 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 03:55:38.752647 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748073 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 03:55:38.752647 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748075 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 03:55:38.752647 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748078 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 03:55:38.752647 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748081 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 03:55:38.752647 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748083 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 03:55:38.752647 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748086 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 03:55:38.752647 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748089 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 03:55:38.752647 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748098 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 03:55:38.752647 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748101 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 03:55:38.752647 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748104 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 03:55:38.752647 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748107 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 03:55:38.752647 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.748112 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 08 03:55:38.753011 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748264 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 03:55:38.753011 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748270 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 03:55:38.753011 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748274 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 03:55:38.753011 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748277 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 03:55:38.753011 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748280 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 03:55:38.753011 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748284 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 03:55:38.753011 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748286 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 03:55:38.753011 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748289 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 03:55:38.753011 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748292 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 03:55:38.753011 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748294 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 03:55:38.753011 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748297 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 03:55:38.753011 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748299 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 03:55:38.753011 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748302 2579 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 03:55:38.753011 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748304 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 03:55:38.753011 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748307 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 03:55:38.753011 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748310 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 03:55:38.753011 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748312 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 03:55:38.753011 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748315 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 03:55:38.753011 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748317 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 03:55:38.753475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748319 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 03:55:38.753475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748322 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 03:55:38.753475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748324 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 03:55:38.753475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748327 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 03:55:38.753475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748329 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 03:55:38.753475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748332 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 03:55:38.753475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748335 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 03:55:38.753475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748337 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 03:55:38.753475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748341 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 03:55:38.753475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748351 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 03:55:38.753475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748354 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 03:55:38.753475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748357 2579 feature_gate.go:328] unrecognized feature gate: Example Sep 08 03:55:38.753475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748359 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 03:55:38.753475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748362 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 03:55:38.753475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748365 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 03:55:38.753475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748367 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 03:55:38.753475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748370 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 03:55:38.753475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748372 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 03:55:38.753475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748375 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 03:55:38.753475 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748377 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 03:55:38.753980 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748380 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 03:55:38.753980 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748382 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 03:55:38.753980 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748385 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 03:55:38.753980 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748387 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 03:55:38.753980 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748390 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 03:55:38.753980 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748393 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 03:55:38.753980 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748395 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 03:55:38.753980 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748398 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 03:55:38.753980 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748400 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 03:55:38.753980 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748403 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 03:55:38.753980 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748405 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 03:55:38.753980 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748408 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 03:55:38.753980 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748410 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 03:55:38.753980 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748413 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 03:55:38.753980 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748415 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 03:55:38.753980 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748418 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 03:55:38.753980 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748420 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 03:55:38.753980 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748423 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 03:55:38.753980 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748425 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 03:55:38.753980 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748427 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 03:55:38.754447 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748430 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 03:55:38.754447 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748432 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 03:55:38.754447 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748440 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 03:55:38.754447 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748443 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 03:55:38.754447 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748445 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 03:55:38.754447 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748448 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 03:55:38.754447 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748450 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 03:55:38.754447 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748453 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 03:55:38.754447 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748456 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 03:55:38.754447 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748459 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 03:55:38.754447 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748463 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 03:55:38.754447 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748465 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 03:55:38.754447 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748468 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 03:55:38.754447 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748471 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 03:55:38.754447 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748473 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 03:55:38.754447 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748476 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 03:55:38.754447 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748478 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 03:55:38.754447 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748481 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 03:55:38.754447 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748483 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 03:55:38.754927 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748486 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 03:55:38.754927 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748488 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 03:55:38.754927 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748491 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 03:55:38.754927 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748493 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 03:55:38.754927 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748496 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 03:55:38.754927 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748498 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 03:55:38.754927 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748501 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 03:55:38.754927 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748503 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 03:55:38.754927 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748506 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 03:55:38.754927 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748508 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 03:55:38.754927 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748511 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 03:55:38.754927 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748513 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 03:55:38.754927 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748515 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 03:55:38.754927 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748518 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 03:55:38.754927 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748520 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 03:55:38.754927 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748522 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 03:55:38.754927 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748530 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 03:55:38.754927 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748533 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 03:55:38.754927 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748535 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 03:55:38.754927 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748538 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 03:55:38.755415 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:38.748540 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 03:55:38.755415 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.748545 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 08 03:55:38.755415 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.748658 2579 server.go:962] "Client rotation is on, will bootstrap in background" Sep 08 03:55:38.755415 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.751440 2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 08 03:55:38.755415 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.752367 2579 server.go:1019] "Starting client certificate rotation" Sep 08 03:55:38.755415 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.752461 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 08 03:55:38.755415 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.752500 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 08 03:55:38.755415 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.754967 2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 08 03:55:38.760989 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.760968 2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 08 03:55:38.774879 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.774856 2579 log.go:25] "Validated CRI v1 runtime API" Sep 08 03:55:38.777832 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.777816 2579 log.go:25] "Validated CRI v1 image API" Sep 08 03:55:38.780191 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.780174 2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 08 03:55:38.781806 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.781790 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 08 03:55:38.783469 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.783442 2579 fs.go:135] Filesystem UUIDs: map[587294d4-c75f-4e07-8d29-1403212ddfb0:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f33e5c9a-630f-4197-807a-34a88914fd20:/dev/nvme0n1p4] Sep 08 03:55:38.783541 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.783469 2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 08 03:55:38.788761 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.788644 2579 manager.go:216] Machine: {Timestamp:2026-09-08 03:55:38.787177032 +0000 UTC m=+0.341903483 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3122489 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28642d4d75e3ecd3d0517c19ae68b5 SystemUUID:ec28642d-4d75-e3ec-d3d0-517c19ae68b5 BootID:1910edc6-b219-4d58-9ca2-7a2287b42494 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:52:73:0c:9e:ff Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:52:73:0c:9e:ff Speed:0 Mtu:9001} {Name:ovs-system MacAddress:02:ce:3a:e5:ee:7f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 08 03:55:38.788761 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.788753 2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 08 03:55:38.788937 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.788907 2579 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 08 03:55:38.792259 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.792237 2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 08 03:55:38.792404 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.792260 2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-141.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 08 03:55:38.792984 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.792974 2579 topology_manager.go:138] "Creating topology manager with none policy" Sep 08 03:55:38.793023 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.792987 2579 container_manager_linux.go:309] "Creating device plugin manager" Sep 08 03:55:38.793023 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.792994 2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 08 03:55:38.793691 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.793681 2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 08 03:55:38.795073 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.795063 2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 08 03:55:38.797759 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.797748 2579 state_mem.go:36] "Initialized new in-memory state store" Sep 08 03:55:38.797883 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.797873 2579 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 08 03:55:38.797919 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.797905 2579 kubelet.go:486] "Attempting to sync node with API server" Sep 08 03:55:38.797919 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.797919 2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 08 03:55:38.798039 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.797932 2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 08 03:55:38.798039 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.797941 2579 kubelet.go:398] "Adding apiserver pod source" Sep 08 03:55:38.798039 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.797950 2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 08 03:55:38.798269 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.798257 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 08 03:55:38.798299 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.798282 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 08 03:55:38.800559 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.800537 2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 08 03:55:38.802826 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.802812 2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 08 03:55:38.802890 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.802842 2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 08 03:55:38.803033 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.803023 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 08 03:55:38.803073 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.803038 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 08 03:55:38.803073 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.803044 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 08 03:55:38.803073 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.803049 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 08 03:55:38.803073 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.803057 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 08 03:55:38.803073 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.803063 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 08 03:55:38.803583 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.803574 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 08 03:55:38.803583 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.803583 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 08 03:55:38.803645 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.803591 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 08 03:55:38.803645 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.803598 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 08 03:55:38.803645 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.803616 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 08 03:55:38.803645 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.803625 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 08 03:55:38.804497 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.804486 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 08 03:55:38.804534 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.804498 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 08 03:55:38.806827 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:38.806744 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 08 03:55:38.806935 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:38.806895 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-141.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 08 03:55:38.808374 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.808352 2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-141.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 08 03:55:38.809393 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.809380 2579 server.go:1268] "Started kubelet" Sep 08 03:55:38.809464 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.809426 2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 08 03:55:38.809522 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.809494 2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 08 03:55:38.809568 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.809543 2579 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 08 03:55:38.810185 ip-10-0-128-141 systemd[1]: Started Kubernetes Kubelet. Sep 08 03:55:38.811181 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.811148 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-r4btq" Sep 08 03:55:38.811266 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.811201 2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 08 03:55:38.811569 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.811554 2579 server.go:310] "Adding debug handlers to kubelet server" Sep 08 03:55:38.813908 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:38.813877 2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 08 03:55:38.816036 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.816019 2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 08 03:55:38.816036 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.816031 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 08 03:55:38.816163 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:38.816128 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-141.ec2.internal\" not found" Sep 08 03:55:38.816625 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.816609 2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 08 03:55:38.816710 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.816698 2579 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 08 03:55:38.816787 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.816717 2579 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 08 03:55:38.816869 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.816856 2579 reconstruct.go:97] "Volume reconstruction finished" Sep 08 03:55:38.816869 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.816868 2579 reconciler.go:29] "Reconciler: start to sync state" Sep 08 03:55:38.816952 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.816913 2579 factory.go:55] Registering systemd factory Sep 08 03:55:38.816952 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.816928 2579 factory.go:223] Registration of the systemd container factory successfully Sep 08 03:55:38.817078 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:38.816132 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-141.ec2.internal.18d33c43c7d5590a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-141.ec2.internal,UID:ip-10-0-128-141.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-141.ec2.internal,},FirstTimestamp:2026-09-08 03:55:38.809342218 +0000 UTC m=+0.364068672,LastTimestamp:2026-09-08 03:55:38.809342218 +0000 UTC m=+0.364068672,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-141.ec2.internal,}" Sep 08 03:55:38.817192 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.817159 2579 factory.go:153] Registering CRI-O factory Sep 08 03:55:38.817258 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.817197 2579 factory.go:223] Registration of the crio container factory successfully Sep 08 03:55:38.817309 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.817257 2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 08 03:55:38.817309 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.817276 2579 factory.go:103] Registering Raw factory Sep 08 03:55:38.817309 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.817294 2579 manager.go:1196] Started watching for new ooms in manager Sep 08 03:55:38.817670 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.817655 2579 manager.go:349] Starting recovery of all containers Sep 08 03:55:38.819213 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.819191 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-r4btq" Sep 08 03:55:38.827003 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.826042 2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 03:55:38.828120 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.828104 2579 manager.go:354] Recovery completed Sep 08 03:55:38.828430 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:38.828402 2579 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-141.ec2.internal\" not found" node="ip-10-0-128-141.ec2.internal" Sep 08 03:55:38.833605 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.833586 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 03:55:38.835490 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.835474 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasSufficientMemory" Sep 08 03:55:38.835594 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.835503 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 03:55:38.835594 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.835513 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasSufficientPID" Sep 08 03:55:38.836113 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.836096 2579 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 08 03:55:38.836113 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.836110 2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 08 03:55:38.836239 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.836121 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 03:55:38.836239 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.836130 2579 state_mem.go:36] "Initialized new in-memory state store" Sep 08 03:55:38.836846 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.836831 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasSufficientMemory" Sep 08 03:55:38.836916 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.836862 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 03:55:38.836916 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.836876 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasSufficientPID" Sep 08 03:55:38.837202 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.837190 2579 policy_none.go:49] "None policy: Start" Sep 08 03:55:38.837263 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.837205 2579 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 08 03:55:38.837263 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.837215 2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 08 03:55:38.838219 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.838209 2579 policy_none.go:47] "Start" Sep 08 03:55:38.861342 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.861328 2579 manager.go:337] "Starting Device Plugin manager" Sep 08 03:55:38.876403 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:38.861360 2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 08 03:55:38.876403 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.861369 2579 server.go:87] "Starting device plugin registration server" Sep 08 03:55:38.876403 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.861596 2579 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 08 03:55:38.876403 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.861609 2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 08 03:55:38.876403 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.861941 2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 08 03:55:38.876403 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.862025 2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 08 03:55:38.876403 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.862034 2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 08 03:55:38.876403 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:38.862296 2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 08 03:55:38.876403 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:38.862366 2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-141.ec2.internal\" not found" Sep 08 03:55:38.934203 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.934171 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 08 03:55:38.935477 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.935461 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 08 03:55:38.935545 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.935509 2579 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 08 03:55:38.935545 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.935531 2579 kubelet.go:2448] "Starting kubelet main sync loop" Sep 08 03:55:38.935629 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:38.935573 2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 08 03:55:38.937494 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.937469 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 03:55:38.962254 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.962228 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 03:55:38.963422 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.963406 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasSufficientMemory" Sep 08 03:55:38.963495 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.963437 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 03:55:38.963495 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.963450 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasSufficientPID" Sep 08 03:55:38.963495 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.963471 2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-141.ec2.internal" Sep 08 03:55:38.971405 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:38.971392 2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-141.ec2.internal" Sep 08 03:55:38.971468 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:38.971412 2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-141.ec2.internal\": node \"ip-10-0-128-141.ec2.internal\" not found" Sep 08 03:55:38.986407 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:38.986342 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-141.ec2.internal\" not found" Sep 08 03:55:39.035966 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.035940 2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-141.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal"] Sep 08 03:55:39.036066 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.036006 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 03:55:39.036977 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.036955 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasSufficientMemory" Sep 08 03:55:39.037051 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.036992 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 03:55:39.037051 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.037005 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasSufficientPID" Sep 08 03:55:39.038213 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.038202 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 03:55:39.038335 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.038321 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-141.ec2.internal" Sep 08 03:55:39.038376 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.038349 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 03:55:39.038920 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.038900 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasSufficientMemory" Sep 08 03:55:39.039002 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.038923 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasSufficientMemory" Sep 08 03:55:39.039002 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.038930 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 03:55:39.039002 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.038945 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasSufficientPID" Sep 08 03:55:39.039002 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.038955 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 03:55:39.039002 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.038965 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasSufficientPID" Sep 08 03:55:39.040092 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.040077 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal" Sep 08 03:55:39.040160 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.040107 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 03:55:39.040716 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.040698 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasSufficientMemory" Sep 08 03:55:39.040836 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.040723 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 03:55:39.040836 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.040735 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasSufficientPID" Sep 08 03:55:39.065420 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:39.065398 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-141.ec2.internal\" not found" node="ip-10-0-128-141.ec2.internal" Sep 08 03:55:39.069818 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:39.069802 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-141.ec2.internal\" not found" node="ip-10-0-128-141.ec2.internal" Sep 08 03:55:39.086780 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:39.086750 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-141.ec2.internal\" not found" Sep 08 03:55:39.118398 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.118374 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f6f3f87e6fd344b94ef9c0dd58f8052f-config\") pod \"kube-apiserver-proxy-ip-10-0-128-141.ec2.internal\" (UID: \"f6f3f87e6fd344b94ef9c0dd58f8052f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-141.ec2.internal" Sep 08 03:55:39.118490 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.118398 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ed2033b059035428790cef4fd0e866ec-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal\" (UID: \"ed2033b059035428790cef4fd0e866ec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal" Sep 08 03:55:39.118490 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.118416 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed2033b059035428790cef4fd0e866ec-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal\" (UID: \"ed2033b059035428790cef4fd0e866ec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal" Sep 08 03:55:39.187623 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:39.187602 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-141.ec2.internal\" not found" Sep 08 03:55:39.218962 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.218941 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f6f3f87e6fd344b94ef9c0dd58f8052f-config\") pod \"kube-apiserver-proxy-ip-10-0-128-141.ec2.internal\" (UID: \"f6f3f87e6fd344b94ef9c0dd58f8052f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-141.ec2.internal" Sep 08 03:55:39.219043 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.218966 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ed2033b059035428790cef4fd0e866ec-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal\" (UID: \"ed2033b059035428790cef4fd0e866ec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal" Sep 08 03:55:39.219043 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.218983 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed2033b059035428790cef4fd0e866ec-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal\" (UID: \"ed2033b059035428790cef4fd0e866ec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal" Sep 08 03:55:39.219043 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.219008 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed2033b059035428790cef4fd0e866ec-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal\" (UID: \"ed2033b059035428790cef4fd0e866ec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal" Sep 08 03:55:39.219153 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.219044 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f6f3f87e6fd344b94ef9c0dd58f8052f-config\") pod \"kube-apiserver-proxy-ip-10-0-128-141.ec2.internal\" (UID: \"f6f3f87e6fd344b94ef9c0dd58f8052f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-141.ec2.internal" Sep 08 03:55:39.219153 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.219055 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ed2033b059035428790cef4fd0e866ec-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal\" (UID: \"ed2033b059035428790cef4fd0e866ec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal" Sep 08 03:55:39.288286 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:39.288202 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-141.ec2.internal\" not found" Sep 08 03:55:39.367954 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.367931 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-141.ec2.internal" Sep 08 03:55:39.372833 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:39.372818 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal" Sep 08 03:55:39.389254 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:39.389225 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-141.ec2.internal\" not found" Sep 08 03:55:41.423441 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:41.423402 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-141.ec2.internal\" not found" Sep 08 03:55:41.524002 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:41.523920 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-141.ec2.internal\" not found" Sep 08 03:55:41.565074 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.565046 2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 03:55:41.650818 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.650781 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-141.ec2.internal" Sep 08 03:55:41.658323 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.658307 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 08 03:55:41.660146 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.660133 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal" Sep 08 03:55:41.671560 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.671545 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 08 03:55:41.685708 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.685687 2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 08 03:55:41.685853 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.685831 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 08 03:55:41.685899 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.685855 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 08 03:55:41.685899 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.685885 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 08 03:55:41.731981 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.731957 2579 apiserver.go:52] "Watching apiserver" Sep 08 03:55:41.750015 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.749989 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 08 03:55:41.750638 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.750619 2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 08 03:55:41.752596 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.752575 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-nft7t","openshift-multus/multus-l47nd","openshift-network-operator/iptables-alerter-492b5","openshift-ovn-kubernetes/ovnkube-node-tg7hb","kube-system/konnectivity-agent-2rrvn","kube-system/kube-apiserver-proxy-ip-10-0-128-141.ec2.internal","openshift-cluster-node-tuning-operator/tuned-h8sh5","openshift-image-registry/node-ca-wmrvz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal","openshift-multus/network-metrics-daemon-r4xfl","openshift-network-diagnostics/network-check-target-xj4pw","kube-system/global-pull-secret-syncer-hrmrn","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd","openshift-dns/node-resolver-djmk2"] Sep 08 03:55:41.754099 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.754080 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nft7t" Sep 08 03:55:41.755064 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.755041 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-07 03:50:40 +0000 UTC" deadline="2028-02-03 23:01:25.157768756 +0000 UTC" Sep 08 03:55:41.755064 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.755064 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12331h5m43.402707368s" Sep 08 03:55:41.755165 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.755148 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.757901 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.756562 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 08 03:55:41.757901 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.756778 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 08 03:55:41.757901 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.756897 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 08 03:55:41.757901 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.757197 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.757901 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.757021 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 08 03:55:41.757901 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.757076 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 08 03:55:41.758220 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.758153 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 08 03:55:41.758539 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.758517 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wv6xc\"" Sep 08 03:55:41.758850 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.758819 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rzlv4\"" Sep 08 03:55:41.759203 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.759188 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.760356 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.760338 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 08 03:55:41.760477 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.760409 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9s8nj\"" Sep 08 03:55:41.760758 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.760740 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2rrvn" Sep 08 03:55:41.760856 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.760839 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 08 03:55:41.762102 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.762087 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 08 03:55:41.762210 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.762146 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-h9wdd\"" Sep 08 03:55:41.762275 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.762215 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-492b5" Sep 08 03:55:41.762275 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.762240 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 08 03:55:41.763157 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.763138 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 08 03:55:41.763232 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.763139 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 08 03:55:41.763232 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.763148 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 08 03:55:41.763961 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.763733 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wmrvz" Sep 08 03:55:41.763961 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.763768 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 08 03:55:41.763961 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.763786 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-njw5q\"" Sep 08 03:55:41.763961 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.763788 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 08 03:55:41.764167 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.763982 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 08 03:55:41.764816 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.764802 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 08 03:55:41.764886 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.764823 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 08 03:55:41.765103 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.765089 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:55:41.765186 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:41.765165 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xfl" podUID="b7239679-8052-415f-ae9a-1b20d61d1223" Sep 08 03:55:41.765248 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.765238 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 08 03:55:41.765346 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.765326 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pzq2s\"" Sep 08 03:55:41.766386 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.766364 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 08 03:55:41.766641 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.766623 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2h5cd\"" Sep 08 03:55:41.766732 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.766684 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:55:41.766732 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.766628 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 08 03:55:41.766732 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.766719 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 08 03:55:41.766889 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:41.766735 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj4pw" podUID="c9c565ac-fd53-499d-b60a-e0af77499433" Sep 08 03:55:41.767235 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.767219 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 08 03:55:41.768178 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.768162 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:55:41.768241 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:41.768209 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrmrn" podUID="26877828-beec-4ff6-a322-79cdf54e0039" Sep 08 03:55:41.769793 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.769777 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" Sep 08 03:55:41.771391 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.771374 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-djmk2" Sep 08 03:55:41.772172 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.772155 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 08 03:55:41.772233 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.772195 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 08 03:55:41.772276 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.772232 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2mkbx\"" Sep 08 03:55:41.772471 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.772448 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 08 03:55:41.773908 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.773891 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 08 03:55:41.774532 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.774493 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-f2rl6\"" Sep 08 03:55:41.775583 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.775569 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 08 03:55:41.797974 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.797950 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8d87z" Sep 08 03:55:41.805139 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.805118 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8d87z" Sep 08 03:55:41.851505 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.851488 2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 08 03:55:41.856613 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.856597 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-var-lib-kubelet\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.856701 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.856620 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bvn7\" (UniqueName: \"kubernetes.io/projected/308e3330-d662-4672-8e7f-9892c04a5491-kube-api-access-4bvn7\") pod \"node-ca-wmrvz\" (UID: \"308e3330-d662-4672-8e7f-9892c04a5491\") " pod="openshift-image-registry/node-ca-wmrvz" Sep 08 03:55:41.856701 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.856640 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26877828-beec-4ff6-a322-79cdf54e0039-original-pull-secret\") pod \"global-pull-secret-syncer-hrmrn\" (UID: \"26877828-beec-4ff6-a322-79cdf54e0039\") " pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:55:41.856701 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.856654 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-etc-openvswitch\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.856701 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.856669 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.856831 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.856701 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-etc-sysctl-conf\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.856831 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.856749 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cc5a49e7-1980-4418-acf0-0be7ad18e2f5-agent-certs\") pod \"konnectivity-agent-2rrvn\" (UID: \"cc5a49e7-1980-4418-acf0-0be7ad18e2f5\") " pod="kube-system/konnectivity-agent-2rrvn" Sep 08 03:55:41.856831 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.856764 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-etc-kubernetes\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.856831 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.856780 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-systemd-units\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.856831 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.856814 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-node-log\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.856990 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.856837 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-log-socket\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.856990 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.856859 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs\") pod \"network-metrics-daemon-r4xfl\" (UID: \"b7239679-8052-415f-ae9a-1b20d61d1223\") " pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:55:41.856990 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.856882 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbmfx\" (UniqueName: \"kubernetes.io/projected/9a603e06-e5e6-4bc5-b352-42bb6dcdbb85-kube-api-access-sbmfx\") pod \"multus-additional-cni-plugins-nft7t\" (UID: \"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85\") " pod="openshift-multus/multus-additional-cni-plugins-nft7t" Sep 08 03:55:41.856990 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.856915 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-host-slash\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.856990 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.856941 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-lib-modules\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.856990 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.856975 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-etc-sysctl-d\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.857209 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857003 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-host-kubelet\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.857209 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857023 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e7e94ee9-2c58-4da9-9fa9-0364a55475a1-host-slash\") pod \"iptables-alerter-492b5\" (UID: \"e7e94ee9-2c58-4da9-9fa9-0364a55475a1\") " pod="openshift-network-operator/iptables-alerter-492b5" Sep 08 03:55:41.857209 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857036 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-etc-systemd\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.857209 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857049 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/85872ada-b39c-4c47-9286-d77ceca6615d-etc-tuned\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.857209 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857087 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-cnibin\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.857209 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857103 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-host-var-lib-kubelet\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.857209 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857117 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-host-run-multus-certs\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.857209 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857132 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-var-lib-openvswitch\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.857209 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857154 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-host-run-ovn-kubernetes\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.857209 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857171 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/32fbd6c2-64d8-4c05-9ae1-ad267d8f8150-registration-dir\") pod \"aws-ebs-csi-driver-node-47grd\" (UID: \"32fbd6c2-64d8-4c05-9ae1-ad267d8f8150\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" Sep 08 03:55:41.857209 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857188 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a603e06-e5e6-4bc5-b352-42bb6dcdbb85-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nft7t\" (UID: \"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85\") " pod="openshift-multus/multus-additional-cni-plugins-nft7t" Sep 08 03:55:41.857209 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857201 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a60305a7-deaa-4247-b229-9ff28b1ad55b-cni-binary-copy\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.857707 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857224 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-run-openvswitch\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.857707 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857254 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-host-cni-bin\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.857707 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857273 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-host-cni-netd\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.857707 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857286 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/45ae9afd-2dea-43b8-aa31-169b11e60fd2-ovn-node-metrics-cert\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.857707 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857310 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-host\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.857707 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857331 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-run-systemd\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.857707 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857345 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-etc-sysconfig\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.857707 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857361 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/32fbd6c2-64d8-4c05-9ae1-ad267d8f8150-sys-fs\") pod \"aws-ebs-csi-driver-node-47grd\" (UID: \"32fbd6c2-64d8-4c05-9ae1-ad267d8f8150\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" Sep 08 03:55:41.857707 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857377 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-etc-modprobe-d\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.857707 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857397 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec702c02-1c7e-471b-9783-5593eefc92a3-tmp-dir\") pod \"node-resolver-djmk2\" (UID: \"ec702c02-1c7e-471b-9783-5593eefc92a3\") " pod="openshift-dns/node-resolver-djmk2" Sep 08 03:55:41.857707 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857430 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcbch\" (UniqueName: \"kubernetes.io/projected/ec702c02-1c7e-471b-9783-5593eefc92a3-kube-api-access-gcbch\") pod \"node-resolver-djmk2\" (UID: \"ec702c02-1c7e-471b-9783-5593eefc92a3\") " pod="openshift-dns/node-resolver-djmk2" Sep 08 03:55:41.857707 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857481 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-multus-cni-dir\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.857707 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857509 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-host-var-lib-cni-bin\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.857707 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857531 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-run\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.857707 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857555 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85872ada-b39c-4c47-9286-d77ceca6615d-tmp\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.857707 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857586 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9a603e06-e5e6-4bc5-b352-42bb6dcdbb85-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nft7t\" (UID: \"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85\") " pod="openshift-multus/multus-additional-cni-plugins-nft7t" Sep 08 03:55:41.857707 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857611 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a60305a7-deaa-4247-b229-9ff28b1ad55b-multus-daemon-config\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.858319 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857633 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-host-run-netns\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.858319 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857696 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e7e94ee9-2c58-4da9-9fa9-0364a55475a1-iptables-alerter-script\") pod \"iptables-alerter-492b5\" (UID: \"e7e94ee9-2c58-4da9-9fa9-0364a55475a1\") " pod="openshift-network-operator/iptables-alerter-492b5" Sep 08 03:55:41.858319 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857723 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cwgq\" (UniqueName: \"kubernetes.io/projected/b7239679-8052-415f-ae9a-1b20d61d1223-kube-api-access-7cwgq\") pod \"network-metrics-daemon-r4xfl\" (UID: \"b7239679-8052-415f-ae9a-1b20d61d1223\") " pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:55:41.858319 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857747 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/45ae9afd-2dea-43b8-aa31-169b11e60fd2-env-overrides\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.858319 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857771 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b86xg\" (UniqueName: \"kubernetes.io/projected/e7e94ee9-2c58-4da9-9fa9-0364a55475a1-kube-api-access-b86xg\") pod \"iptables-alerter-492b5\" (UID: \"e7e94ee9-2c58-4da9-9fa9-0364a55475a1\") " pod="openshift-network-operator/iptables-alerter-492b5" Sep 08 03:55:41.858319 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857795 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/32fbd6c2-64d8-4c05-9ae1-ad267d8f8150-kubelet-dir\") pod \"aws-ebs-csi-driver-node-47grd\" (UID: \"32fbd6c2-64d8-4c05-9ae1-ad267d8f8150\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" Sep 08 03:55:41.858319 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857817 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/32fbd6c2-64d8-4c05-9ae1-ad267d8f8150-socket-dir\") pod \"aws-ebs-csi-driver-node-47grd\" (UID: \"32fbd6c2-64d8-4c05-9ae1-ad267d8f8150\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" Sep 08 03:55:41.858319 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857841 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-etc-kubernetes\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.858319 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857863 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjmgb\" (UniqueName: \"kubernetes.io/projected/c9c565ac-fd53-499d-b60a-e0af77499433-kube-api-access-wjmgb\") pod \"network-check-target-xj4pw\" (UID: \"c9c565ac-fd53-499d-b60a-e0af77499433\") " pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:55:41.858319 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857889 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/26877828-beec-4ff6-a322-79cdf54e0039-kubelet-config\") pod \"global-pull-secret-syncer-hrmrn\" (UID: \"26877828-beec-4ff6-a322-79cdf54e0039\") " pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:55:41.858319 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857926 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-system-cni-dir\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.858319 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857948 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a603e06-e5e6-4bc5-b352-42bb6dcdbb85-cnibin\") pod \"multus-additional-cni-plugins-nft7t\" (UID: \"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85\") " pod="openshift-multus/multus-additional-cni-plugins-nft7t" Sep 08 03:55:41.858319 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857971 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-host-run-netns\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.858319 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.857989 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brbwg\" (UniqueName: \"kubernetes.io/projected/85872ada-b39c-4c47-9286-d77ceca6615d-kube-api-access-brbwg\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.858319 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.858015 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cc5a49e7-1980-4418-acf0-0be7ad18e2f5-konnectivity-ca\") pod \"konnectivity-agent-2rrvn\" (UID: \"cc5a49e7-1980-4418-acf0-0be7ad18e2f5\") " pod="kube-system/konnectivity-agent-2rrvn" Sep 08 03:55:41.858319 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.858037 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/308e3330-d662-4672-8e7f-9892c04a5491-serviceca\") pod \"node-ca-wmrvz\" (UID: \"308e3330-d662-4672-8e7f-9892c04a5491\") " pod="openshift-image-registry/node-ca-wmrvz" Sep 08 03:55:41.858869 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.858051 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/32fbd6c2-64d8-4c05-9ae1-ad267d8f8150-device-dir\") pod \"aws-ebs-csi-driver-node-47grd\" (UID: \"32fbd6c2-64d8-4c05-9ae1-ad267d8f8150\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" Sep 08 03:55:41.858869 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.858077 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/32fbd6c2-64d8-4c05-9ae1-ad267d8f8150-etc-selinux\") pod \"aws-ebs-csi-driver-node-47grd\" (UID: \"32fbd6c2-64d8-4c05-9ae1-ad267d8f8150\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" Sep 08 03:55:41.858869 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.858093 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a603e06-e5e6-4bc5-b352-42bb6dcdbb85-system-cni-dir\") pod \"multus-additional-cni-plugins-nft7t\" (UID: \"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85\") " pod="openshift-multus/multus-additional-cni-plugins-nft7t" Sep 08 03:55:41.858869 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.858112 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/26877828-beec-4ff6-a322-79cdf54e0039-dbus\") pod \"global-pull-secret-syncer-hrmrn\" (UID: \"26877828-beec-4ff6-a322-79cdf54e0039\") " pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:55:41.858869 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.858128 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ec702c02-1c7e-471b-9783-5593eefc92a3-hosts-file\") pod \"node-resolver-djmk2\" (UID: \"ec702c02-1c7e-471b-9783-5593eefc92a3\") " pod="openshift-dns/node-resolver-djmk2" Sep 08 03:55:41.858869 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.858169 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/45ae9afd-2dea-43b8-aa31-169b11e60fd2-ovnkube-script-lib\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.858869 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.858196 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts67s\" (UniqueName: \"kubernetes.io/projected/45ae9afd-2dea-43b8-aa31-169b11e60fd2-kube-api-access-ts67s\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.858869 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.858216 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/308e3330-d662-4672-8e7f-9892c04a5491-host\") pod \"node-ca-wmrvz\" (UID: \"308e3330-d662-4672-8e7f-9892c04a5491\") " pod="openshift-image-registry/node-ca-wmrvz" Sep 08 03:55:41.858869 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.858237 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a603e06-e5e6-4bc5-b352-42bb6dcdbb85-cni-binary-copy\") pod \"multus-additional-cni-plugins-nft7t\" (UID: \"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85\") " pod="openshift-multus/multus-additional-cni-plugins-nft7t" Sep 08 03:55:41.858869 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.858253 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-host-run-k8s-cni-cncf-io\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.858869 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.858271 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-hostroot\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.858869 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.858295 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-multus-conf-dir\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.858869 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.858317 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-run-ovn\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.858869 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.858335 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/45ae9afd-2dea-43b8-aa31-169b11e60fd2-ovnkube-config\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.858869 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.858371 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9a603e06-e5e6-4bc5-b352-42bb6dcdbb85-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nft7t\" (UID: \"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85\") " pod="openshift-multus/multus-additional-cni-plugins-nft7t" Sep 08 03:55:41.858869 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.858393 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-multus-socket-dir-parent\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.859352 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.858414 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gbpf\" (UniqueName: \"kubernetes.io/projected/a60305a7-deaa-4247-b229-9ff28b1ad55b-kube-api-access-7gbpf\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.859352 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.858448 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-host-var-lib-cni-multus\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.859352 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.858490 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x92h9\" (UniqueName: \"kubernetes.io/projected/32fbd6c2-64d8-4c05-9ae1-ad267d8f8150-kube-api-access-x92h9\") pod \"aws-ebs-csi-driver-node-47grd\" (UID: \"32fbd6c2-64d8-4c05-9ae1-ad267d8f8150\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" Sep 08 03:55:41.859352 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.858512 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-sys\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.910919 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.910893 2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 03:55:41.945825 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:41.945793 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded2033b059035428790cef4fd0e866ec.slice/crio-795a97ca3878f1219f2ed8cb197ddf748e6e86bb6e63f7fe1eeb605061437ae4 WatchSource:0}: Error finding container 795a97ca3878f1219f2ed8cb197ddf748e6e86bb6e63f7fe1eeb605061437ae4: Status 404 returned error can't find the container with id 795a97ca3878f1219f2ed8cb197ddf748e6e86bb6e63f7fe1eeb605061437ae4 Sep 08 03:55:41.946055 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:41.946033 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6f3f87e6fd344b94ef9c0dd58f8052f.slice/crio-c5412b6190610b6ac687f0cdf08dc29e4dc241714802129837cb8006de5769db WatchSource:0}: Error finding container c5412b6190610b6ac687f0cdf08dc29e4dc241714802129837cb8006de5769db: Status 404 returned error can't find the container with id c5412b6190610b6ac687f0cdf08dc29e4dc241714802129837cb8006de5769db Sep 08 03:55:41.951750 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.951730 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 03:55:41.959242 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959221 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9a603e06-e5e6-4bc5-b352-42bb6dcdbb85-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nft7t\" (UID: \"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85\") " pod="openshift-multus/multus-additional-cni-plugins-nft7t" Sep 08 03:55:41.959317 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959255 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-multus-socket-dir-parent\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.959317 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959278 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gbpf\" (UniqueName: \"kubernetes.io/projected/a60305a7-deaa-4247-b229-9ff28b1ad55b-kube-api-access-7gbpf\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.959317 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959301 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-host-var-lib-cni-multus\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.959474 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959358 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-host-var-lib-cni-multus\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.959474 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959413 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-multus-socket-dir-parent\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.959565 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959483 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x92h9\" (UniqueName: \"kubernetes.io/projected/32fbd6c2-64d8-4c05-9ae1-ad267d8f8150-kube-api-access-x92h9\") pod \"aws-ebs-csi-driver-node-47grd\" (UID: \"32fbd6c2-64d8-4c05-9ae1-ad267d8f8150\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" Sep 08 03:55:41.959565 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959522 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-sys\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.959565 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959541 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-var-lib-kubelet\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.959565 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959557 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bvn7\" (UniqueName: \"kubernetes.io/projected/308e3330-d662-4672-8e7f-9892c04a5491-kube-api-access-4bvn7\") pod \"node-ca-wmrvz\" (UID: \"308e3330-d662-4672-8e7f-9892c04a5491\") " pod="openshift-image-registry/node-ca-wmrvz" Sep 08 03:55:41.959756 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959574 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26877828-beec-4ff6-a322-79cdf54e0039-original-pull-secret\") pod \"global-pull-secret-syncer-hrmrn\" (UID: \"26877828-beec-4ff6-a322-79cdf54e0039\") " pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:55:41.959756 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959598 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-etc-openvswitch\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.959756 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959622 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.959756 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959624 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-var-lib-kubelet\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.959756 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959646 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-etc-sysctl-conf\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.959756 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959661 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-sys\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.959756 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959669 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cc5a49e7-1980-4418-acf0-0be7ad18e2f5-agent-certs\") pod \"konnectivity-agent-2rrvn\" (UID: \"cc5a49e7-1980-4418-acf0-0be7ad18e2f5\") " pod="kube-system/konnectivity-agent-2rrvn" Sep 08 03:55:41.959756 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959686 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-etc-openvswitch\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.959756 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959704 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-etc-kubernetes\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.959756 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959713 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.959756 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959731 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-systemd-units\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.959756 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959754 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-node-log\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.959756 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:41.959756 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 03:55:41.960377 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959763 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-etc-kubernetes\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.960377 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959777 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-log-socket\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.960377 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959805 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs\") pod \"network-metrics-daemon-r4xfl\" (UID: \"b7239679-8052-415f-ae9a-1b20d61d1223\") " pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:55:41.960377 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959811 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-node-log\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.960377 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959811 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-etc-sysctl-conf\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.960377 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959811 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-systemd-units\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.960377 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959832 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbmfx\" (UniqueName: \"kubernetes.io/projected/9a603e06-e5e6-4bc5-b352-42bb6dcdbb85-kube-api-access-sbmfx\") pod \"multus-additional-cni-plugins-nft7t\" (UID: \"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85\") " pod="openshift-multus/multus-additional-cni-plugins-nft7t" Sep 08 03:55:41.960377 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959861 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-log-socket\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.960377 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:41.959868 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26877828-beec-4ff6-a322-79cdf54e0039-original-pull-secret podName:26877828-beec-4ff6-a322-79cdf54e0039 nodeName:}" failed. No retries permitted until 2026-09-08 03:55:42.459845951 +0000 UTC m=+2.080881921 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/26877828-beec-4ff6-a322-79cdf54e0039-original-pull-secret") pod "global-pull-secret-syncer-hrmrn" (UID: "26877828-beec-4ff6-a322-79cdf54e0039") : object "kube-system"/"original-pull-secret" not registered Sep 08 03:55:41.960377 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959888 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-host-slash\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.960377 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:41.959899 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 03:55:41.960377 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959917 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-lib-modules\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.960377 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959942 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-etc-sysctl-d\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.960377 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959941 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-host-slash\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.960377 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:41.959954 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs podName:b7239679-8052-415f-ae9a-1b20d61d1223 nodeName:}" failed. No retries permitted until 2026-09-08 03:55:42.459937579 +0000 UTC m=+2.080973538 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs") pod "network-metrics-daemon-r4xfl" (UID: "b7239679-8052-415f-ae9a-1b20d61d1223") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 03:55:41.960377 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959983 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-host-kubelet\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.960377 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.959994 2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 08 03:55:41.961187 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960009 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e7e94ee9-2c58-4da9-9fa9-0364a55475a1-host-slash\") pod \"iptables-alerter-492b5\" (UID: \"e7e94ee9-2c58-4da9-9fa9-0364a55475a1\") " pod="openshift-network-operator/iptables-alerter-492b5" Sep 08 03:55:41.961187 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960033 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-host-kubelet\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.961187 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960033 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-etc-systemd\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.961187 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960049 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-lib-modules\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.961187 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960073 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-etc-systemd\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.961187 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960064 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/85872ada-b39c-4c47-9286-d77ceca6615d-etc-tuned\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.961187 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960079 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-etc-sysctl-d\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.961187 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960105 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-cnibin\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.961187 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960110 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e7e94ee9-2c58-4da9-9fa9-0364a55475a1-host-slash\") pod \"iptables-alerter-492b5\" (UID: \"e7e94ee9-2c58-4da9-9fa9-0364a55475a1\") " pod="openshift-network-operator/iptables-alerter-492b5" Sep 08 03:55:41.961187 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960131 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-host-var-lib-kubelet\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.961187 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960161 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-cnibin\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.961187 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960164 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-host-run-multus-certs\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.961187 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960200 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-host-run-multus-certs\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.961187 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960201 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-host-var-lib-kubelet\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.961187 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960213 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-var-lib-openvswitch\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.961187 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960239 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-host-run-ovn-kubernetes\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.961187 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960263 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/32fbd6c2-64d8-4c05-9ae1-ad267d8f8150-registration-dir\") pod \"aws-ebs-csi-driver-node-47grd\" (UID: \"32fbd6c2-64d8-4c05-9ae1-ad267d8f8150\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" Sep 08 03:55:41.961187 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960286 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a603e06-e5e6-4bc5-b352-42bb6dcdbb85-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nft7t\" (UID: \"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85\") " pod="openshift-multus/multus-additional-cni-plugins-nft7t" Sep 08 03:55:41.961970 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960312 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a60305a7-deaa-4247-b229-9ff28b1ad55b-cni-binary-copy\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.961970 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960320 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-var-lib-openvswitch\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.961970 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960323 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/32fbd6c2-64d8-4c05-9ae1-ad267d8f8150-registration-dir\") pod \"aws-ebs-csi-driver-node-47grd\" (UID: \"32fbd6c2-64d8-4c05-9ae1-ad267d8f8150\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" Sep 08 03:55:41.961970 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960335 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-run-openvswitch\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.961970 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960351 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-host-run-ovn-kubernetes\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.961970 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960369 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-host-cni-bin\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.961970 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960371 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-run-openvswitch\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.961970 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960395 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-host-cni-netd\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.961970 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960418 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-host-cni-bin\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.961970 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960439 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/45ae9afd-2dea-43b8-aa31-169b11e60fd2-ovn-node-metrics-cert\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.961970 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960474 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-host\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.961970 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960474 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-host-cni-netd\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.961970 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960487 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9a603e06-e5e6-4bc5-b352-42bb6dcdbb85-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nft7t\" (UID: \"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85\") " pod="openshift-multus/multus-additional-cni-plugins-nft7t" Sep 08 03:55:41.961970 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960514 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-host\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.961970 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960474 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a603e06-e5e6-4bc5-b352-42bb6dcdbb85-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nft7t\" (UID: \"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85\") " pod="openshift-multus/multus-additional-cni-plugins-nft7t" Sep 08 03:55:41.961970 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960553 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-run-systemd\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.961970 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960597 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-run-systemd\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.962863 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960619 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-etc-sysconfig\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.962863 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960638 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/32fbd6c2-64d8-4c05-9ae1-ad267d8f8150-sys-fs\") pod \"aws-ebs-csi-driver-node-47grd\" (UID: \"32fbd6c2-64d8-4c05-9ae1-ad267d8f8150\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" Sep 08 03:55:41.962863 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960653 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-etc-modprobe-d\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.962863 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960669 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec702c02-1c7e-471b-9783-5593eefc92a3-tmp-dir\") pod \"node-resolver-djmk2\" (UID: \"ec702c02-1c7e-471b-9783-5593eefc92a3\") " pod="openshift-dns/node-resolver-djmk2" Sep 08 03:55:41.962863 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960693 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcbch\" (UniqueName: \"kubernetes.io/projected/ec702c02-1c7e-471b-9783-5593eefc92a3-kube-api-access-gcbch\") pod \"node-resolver-djmk2\" (UID: \"ec702c02-1c7e-471b-9783-5593eefc92a3\") " pod="openshift-dns/node-resolver-djmk2" Sep 08 03:55:41.962863 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960715 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/32fbd6c2-64d8-4c05-9ae1-ad267d8f8150-sys-fs\") pod \"aws-ebs-csi-driver-node-47grd\" (UID: \"32fbd6c2-64d8-4c05-9ae1-ad267d8f8150\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" Sep 08 03:55:41.962863 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960715 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-multus-cni-dir\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.962863 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960751 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-host-var-lib-cni-bin\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.962863 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960753 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-etc-modprobe-d\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.962863 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960770 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-run\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.962863 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960790 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85872ada-b39c-4c47-9286-d77ceca6615d-tmp\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.962863 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960817 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9a603e06-e5e6-4bc5-b352-42bb6dcdbb85-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nft7t\" (UID: \"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85\") " pod="openshift-multus/multus-additional-cni-plugins-nft7t" Sep 08 03:55:41.962863 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960824 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-multus-cni-dir\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.962863 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960839 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a60305a7-deaa-4247-b229-9ff28b1ad55b-cni-binary-copy\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.962863 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960841 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a60305a7-deaa-4247-b229-9ff28b1ad55b-multus-daemon-config\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.962863 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960881 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-host-run-netns\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.962863 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960907 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e7e94ee9-2c58-4da9-9fa9-0364a55475a1-iptables-alerter-script\") pod \"iptables-alerter-492b5\" (UID: \"e7e94ee9-2c58-4da9-9fa9-0364a55475a1\") " pod="openshift-network-operator/iptables-alerter-492b5" Sep 08 03:55:41.963653 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960931 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cwgq\" (UniqueName: \"kubernetes.io/projected/b7239679-8052-415f-ae9a-1b20d61d1223-kube-api-access-7cwgq\") pod \"network-metrics-daemon-r4xfl\" (UID: \"b7239679-8052-415f-ae9a-1b20d61d1223\") " pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:55:41.963653 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960954 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/45ae9afd-2dea-43b8-aa31-169b11e60fd2-env-overrides\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.963653 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.960979 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b86xg\" (UniqueName: \"kubernetes.io/projected/e7e94ee9-2c58-4da9-9fa9-0364a55475a1-kube-api-access-b86xg\") pod \"iptables-alerter-492b5\" (UID: \"e7e94ee9-2c58-4da9-9fa9-0364a55475a1\") " pod="openshift-network-operator/iptables-alerter-492b5" Sep 08 03:55:41.963653 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.961005 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/32fbd6c2-64d8-4c05-9ae1-ad267d8f8150-kubelet-dir\") pod \"aws-ebs-csi-driver-node-47grd\" (UID: \"32fbd6c2-64d8-4c05-9ae1-ad267d8f8150\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" Sep 08 03:55:41.963653 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.961017 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec702c02-1c7e-471b-9783-5593eefc92a3-tmp-dir\") pod \"node-resolver-djmk2\" (UID: \"ec702c02-1c7e-471b-9783-5593eefc92a3\") " pod="openshift-dns/node-resolver-djmk2" Sep 08 03:55:41.963653 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.961028 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/32fbd6c2-64d8-4c05-9ae1-ad267d8f8150-socket-dir\") pod \"aws-ebs-csi-driver-node-47grd\" (UID: \"32fbd6c2-64d8-4c05-9ae1-ad267d8f8150\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" Sep 08 03:55:41.963653 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.961052 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-etc-kubernetes\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.963653 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.961086 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjmgb\" (UniqueName: \"kubernetes.io/projected/c9c565ac-fd53-499d-b60a-e0af77499433-kube-api-access-wjmgb\") pod \"network-check-target-xj4pw\" (UID: \"c9c565ac-fd53-499d-b60a-e0af77499433\") " pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:55:41.963653 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.961112 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/26877828-beec-4ff6-a322-79cdf54e0039-kubelet-config\") pod \"global-pull-secret-syncer-hrmrn\" (UID: \"26877828-beec-4ff6-a322-79cdf54e0039\") " pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:55:41.963653 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.961120 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-etc-kubernetes\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.963653 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.961137 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-system-cni-dir\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.963653 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.961158 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-host-run-netns\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.963653 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.961162 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a603e06-e5e6-4bc5-b352-42bb6dcdbb85-cnibin\") pod \"multus-additional-cni-plugins-nft7t\" (UID: \"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85\") " pod="openshift-multus/multus-additional-cni-plugins-nft7t" Sep 08 03:55:41.963653 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.961185 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-host-run-netns\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.963653 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.961208 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brbwg\" (UniqueName: \"kubernetes.io/projected/85872ada-b39c-4c47-9286-d77ceca6615d-kube-api-access-brbwg\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.963653 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.961231 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cc5a49e7-1980-4418-acf0-0be7ad18e2f5-konnectivity-ca\") pod \"konnectivity-agent-2rrvn\" (UID: \"cc5a49e7-1980-4418-acf0-0be7ad18e2f5\") " pod="kube-system/konnectivity-agent-2rrvn" Sep 08 03:55:41.963653 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.961235 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/26877828-beec-4ff6-a322-79cdf54e0039-kubelet-config\") pod \"global-pull-secret-syncer-hrmrn\" (UID: \"26877828-beec-4ff6-a322-79cdf54e0039\") " pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:55:41.964403 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.961270 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a60305a7-deaa-4247-b229-9ff28b1ad55b-multus-daemon-config\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.964403 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.961318 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-system-cni-dir\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.964403 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.961356 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a603e06-e5e6-4bc5-b352-42bb6dcdbb85-cnibin\") pod \"multus-additional-cni-plugins-nft7t\" (UID: \"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85\") " pod="openshift-multus/multus-additional-cni-plugins-nft7t" Sep 08 03:55:41.964403 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.961391 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-host-run-netns\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.964403 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.961715 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/308e3330-d662-4672-8e7f-9892c04a5491-serviceca\") pod \"node-ca-wmrvz\" (UID: \"308e3330-d662-4672-8e7f-9892c04a5491\") " pod="openshift-image-registry/node-ca-wmrvz" Sep 08 03:55:41.964403 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.961777 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-etc-sysconfig\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.964403 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.961781 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/45ae9afd-2dea-43b8-aa31-169b11e60fd2-env-overrides\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.964403 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.961806 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e7e94ee9-2c58-4da9-9fa9-0364a55475a1-iptables-alerter-script\") pod \"iptables-alerter-492b5\" (UID: \"e7e94ee9-2c58-4da9-9fa9-0364a55475a1\") " pod="openshift-network-operator/iptables-alerter-492b5" Sep 08 03:55:41.964403 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.961827 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/85872ada-b39c-4c47-9286-d77ceca6615d-run\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.964403 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.961831 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-host-var-lib-cni-bin\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.964403 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.961981 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/32fbd6c2-64d8-4c05-9ae1-ad267d8f8150-kubelet-dir\") pod \"aws-ebs-csi-driver-node-47grd\" (UID: \"32fbd6c2-64d8-4c05-9ae1-ad267d8f8150\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" Sep 08 03:55:41.964403 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.962073 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/32fbd6c2-64d8-4c05-9ae1-ad267d8f8150-socket-dir\") pod \"aws-ebs-csi-driver-node-47grd\" (UID: \"32fbd6c2-64d8-4c05-9ae1-ad267d8f8150\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" Sep 08 03:55:41.964403 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.961271 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/308e3330-d662-4672-8e7f-9892c04a5491-serviceca\") pod \"node-ca-wmrvz\" (UID: \"308e3330-d662-4672-8e7f-9892c04a5491\") " pod="openshift-image-registry/node-ca-wmrvz" Sep 08 03:55:41.964403 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.962290 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/32fbd6c2-64d8-4c05-9ae1-ad267d8f8150-device-dir\") pod \"aws-ebs-csi-driver-node-47grd\" (UID: \"32fbd6c2-64d8-4c05-9ae1-ad267d8f8150\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" Sep 08 03:55:41.964403 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.962315 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/32fbd6c2-64d8-4c05-9ae1-ad267d8f8150-etc-selinux\") pod \"aws-ebs-csi-driver-node-47grd\" (UID: \"32fbd6c2-64d8-4c05-9ae1-ad267d8f8150\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" Sep 08 03:55:41.964403 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.962339 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a603e06-e5e6-4bc5-b352-42bb6dcdbb85-system-cni-dir\") pod \"multus-additional-cni-plugins-nft7t\" (UID: \"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85\") " pod="openshift-multus/multus-additional-cni-plugins-nft7t" Sep 08 03:55:41.964403 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.962362 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/26877828-beec-4ff6-a322-79cdf54e0039-dbus\") pod \"global-pull-secret-syncer-hrmrn\" (UID: \"26877828-beec-4ff6-a322-79cdf54e0039\") " pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:55:41.964403 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.962384 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ec702c02-1c7e-471b-9783-5593eefc92a3-hosts-file\") pod \"node-resolver-djmk2\" (UID: \"ec702c02-1c7e-471b-9783-5593eefc92a3\") " pod="openshift-dns/node-resolver-djmk2" Sep 08 03:55:41.964954 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.962408 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/45ae9afd-2dea-43b8-aa31-169b11e60fd2-ovnkube-script-lib\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.964954 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.962432 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts67s\" (UniqueName: \"kubernetes.io/projected/45ae9afd-2dea-43b8-aa31-169b11e60fd2-kube-api-access-ts67s\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.964954 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.962479 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/308e3330-d662-4672-8e7f-9892c04a5491-host\") pod \"node-ca-wmrvz\" (UID: \"308e3330-d662-4672-8e7f-9892c04a5491\") " pod="openshift-image-registry/node-ca-wmrvz" Sep 08 03:55:41.964954 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.962493 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9a603e06-e5e6-4bc5-b352-42bb6dcdbb85-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nft7t\" (UID: \"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85\") " pod="openshift-multus/multus-additional-cni-plugins-nft7t" Sep 08 03:55:41.964954 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.962502 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a603e06-e5e6-4bc5-b352-42bb6dcdbb85-cni-binary-copy\") pod \"multus-additional-cni-plugins-nft7t\" (UID: \"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85\") " pod="openshift-multus/multus-additional-cni-plugins-nft7t" Sep 08 03:55:41.964954 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.962583 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-host-run-k8s-cni-cncf-io\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.964954 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.962609 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-hostroot\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.964954 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.962677 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-multus-conf-dir\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.964954 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.962705 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-run-ovn\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.964954 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.962730 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/45ae9afd-2dea-43b8-aa31-169b11e60fd2-ovnkube-config\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.964954 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.962916 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a603e06-e5e6-4bc5-b352-42bb6dcdbb85-cni-binary-copy\") pod \"multus-additional-cni-plugins-nft7t\" (UID: \"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85\") " pod="openshift-multus/multus-additional-cni-plugins-nft7t" Sep 08 03:55:41.964954 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.962980 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/32fbd6c2-64d8-4c05-9ae1-ad267d8f8150-device-dir\") pod \"aws-ebs-csi-driver-node-47grd\" (UID: \"32fbd6c2-64d8-4c05-9ae1-ad267d8f8150\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" Sep 08 03:55:41.964954 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.963049 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/32fbd6c2-64d8-4c05-9ae1-ad267d8f8150-etc-selinux\") pod \"aws-ebs-csi-driver-node-47grd\" (UID: \"32fbd6c2-64d8-4c05-9ae1-ad267d8f8150\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" Sep 08 03:55:41.964954 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.963080 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-hostroot\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.964954 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.963088 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a603e06-e5e6-4bc5-b352-42bb6dcdbb85-system-cni-dir\") pod \"multus-additional-cni-plugins-nft7t\" (UID: \"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85\") " pod="openshift-multus/multus-additional-cni-plugins-nft7t" Sep 08 03:55:41.964954 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.963125 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-multus-conf-dir\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.964954 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.963169 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ec702c02-1c7e-471b-9783-5593eefc92a3-hosts-file\") pod \"node-resolver-djmk2\" (UID: \"ec702c02-1c7e-471b-9783-5593eefc92a3\") " pod="openshift-dns/node-resolver-djmk2" Sep 08 03:55:41.965416 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.963125 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a60305a7-deaa-4247-b229-9ff28b1ad55b-host-run-k8s-cni-cncf-io\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.965416 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.963186 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/45ae9afd-2dea-43b8-aa31-169b11e60fd2-ovnkube-config\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.965416 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.963225 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/26877828-beec-4ff6-a322-79cdf54e0039-dbus\") pod \"global-pull-secret-syncer-hrmrn\" (UID: \"26877828-beec-4ff6-a322-79cdf54e0039\") " pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:55:41.965416 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.963248 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/45ae9afd-2dea-43b8-aa31-169b11e60fd2-run-ovn\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.965416 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.963302 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/308e3330-d662-4672-8e7f-9892c04a5491-host\") pod \"node-ca-wmrvz\" (UID: \"308e3330-d662-4672-8e7f-9892c04a5491\") " pod="openshift-image-registry/node-ca-wmrvz" Sep 08 03:55:41.965416 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.963635 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/45ae9afd-2dea-43b8-aa31-169b11e60fd2-ovnkube-script-lib\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.965416 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.963906 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cc5a49e7-1980-4418-acf0-0be7ad18e2f5-konnectivity-ca\") pod \"konnectivity-agent-2rrvn\" (UID: \"cc5a49e7-1980-4418-acf0-0be7ad18e2f5\") " pod="kube-system/konnectivity-agent-2rrvn" Sep 08 03:55:41.965416 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.964235 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/85872ada-b39c-4c47-9286-d77ceca6615d-etc-tuned\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.965416 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.964288 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85872ada-b39c-4c47-9286-d77ceca6615d-tmp\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.965416 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.964345 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/45ae9afd-2dea-43b8-aa31-169b11e60fd2-ovn-node-metrics-cert\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.965416 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.964495 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cc5a49e7-1980-4418-acf0-0be7ad18e2f5-agent-certs\") pod \"konnectivity-agent-2rrvn\" (UID: \"cc5a49e7-1980-4418-acf0-0be7ad18e2f5\") " pod="kube-system/konnectivity-agent-2rrvn" Sep 08 03:55:41.970815 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.970791 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x92h9\" (UniqueName: \"kubernetes.io/projected/32fbd6c2-64d8-4c05-9ae1-ad267d8f8150-kube-api-access-x92h9\") pod \"aws-ebs-csi-driver-node-47grd\" (UID: \"32fbd6c2-64d8-4c05-9ae1-ad267d8f8150\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" Sep 08 03:55:41.972447 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.972423 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bvn7\" (UniqueName: \"kubernetes.io/projected/308e3330-d662-4672-8e7f-9892c04a5491-kube-api-access-4bvn7\") pod \"node-ca-wmrvz\" (UID: \"308e3330-d662-4672-8e7f-9892c04a5491\") " pod="openshift-image-registry/node-ca-wmrvz" Sep 08 03:55:41.972629 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.972614 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbmfx\" (UniqueName: \"kubernetes.io/projected/9a603e06-e5e6-4bc5-b352-42bb6dcdbb85-kube-api-access-sbmfx\") pod \"multus-additional-cni-plugins-nft7t\" (UID: \"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85\") " pod="openshift-multus/multus-additional-cni-plugins-nft7t" Sep 08 03:55:41.972774 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.972759 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gbpf\" (UniqueName: \"kubernetes.io/projected/a60305a7-deaa-4247-b229-9ff28b1ad55b-kube-api-access-7gbpf\") pod \"multus-l47nd\" (UID: \"a60305a7-deaa-4247-b229-9ff28b1ad55b\") " pod="openshift-multus/multus-l47nd" Sep 08 03:55:41.973546 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:41.973449 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 03:55:41.973622 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:41.973553 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 03:55:41.973622 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:41.973569 2579 projected.go:196] Error preparing data for projected volume kube-api-access-wjmgb for pod openshift-network-diagnostics/network-check-target-xj4pw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 03:55:41.973717 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:41.973634 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9c565ac-fd53-499d-b60a-e0af77499433-kube-api-access-wjmgb podName:c9c565ac-fd53-499d-b60a-e0af77499433 nodeName:}" failed. No retries permitted until 2026-09-08 03:55:42.473618053 +0000 UTC m=+2.094654021 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wjmgb" (UniqueName: "kubernetes.io/projected/c9c565ac-fd53-499d-b60a-e0af77499433-kube-api-access-wjmgb") pod "network-check-target-xj4pw" (UID: "c9c565ac-fd53-499d-b60a-e0af77499433") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 03:55:41.974809 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.974787 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brbwg\" (UniqueName: \"kubernetes.io/projected/85872ada-b39c-4c47-9286-d77ceca6615d-kube-api-access-brbwg\") pod \"tuned-h8sh5\" (UID: \"85872ada-b39c-4c47-9286-d77ceca6615d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:41.975025 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.975003 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcbch\" (UniqueName: \"kubernetes.io/projected/ec702c02-1c7e-471b-9783-5593eefc92a3-kube-api-access-gcbch\") pod \"node-resolver-djmk2\" (UID: \"ec702c02-1c7e-471b-9783-5593eefc92a3\") " pod="openshift-dns/node-resolver-djmk2" Sep 08 03:55:41.975323 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.975308 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cwgq\" (UniqueName: \"kubernetes.io/projected/b7239679-8052-415f-ae9a-1b20d61d1223-kube-api-access-7cwgq\") pod \"network-metrics-daemon-r4xfl\" (UID: \"b7239679-8052-415f-ae9a-1b20d61d1223\") " pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:55:41.976480 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.976448 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts67s\" (UniqueName: \"kubernetes.io/projected/45ae9afd-2dea-43b8-aa31-169b11e60fd2-kube-api-access-ts67s\") pod \"ovnkube-node-tg7hb\" (UID: \"45ae9afd-2dea-43b8-aa31-169b11e60fd2\") " pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:41.976601 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:41.976585 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b86xg\" (UniqueName: \"kubernetes.io/projected/e7e94ee9-2c58-4da9-9fa9-0364a55475a1-kube-api-access-b86xg\") pod \"iptables-alerter-492b5\" (UID: \"e7e94ee9-2c58-4da9-9fa9-0364a55475a1\") " pod="openshift-network-operator/iptables-alerter-492b5" Sep 08 03:55:42.087699 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:42.087617 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nft7t" Sep 08 03:55:42.098073 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:42.098043 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a603e06_e5e6_4bc5_b352_42bb6dcdbb85.slice/crio-dff3b75efae918dadabedff3e4a9a2553746d068c55f9c3b80f5985f5900376b WatchSource:0}: Error finding container dff3b75efae918dadabedff3e4a9a2553746d068c55f9c3b80f5985f5900376b: Status 404 returned error can't find the container with id dff3b75efae918dadabedff3e4a9a2553746d068c55f9c3b80f5985f5900376b Sep 08 03:55:42.108008 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:42.107992 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-l47nd" Sep 08 03:55:42.115754 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:42.115735 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda60305a7_deaa_4247_b229_9ff28b1ad55b.slice/crio-d3f150fbcc87d99ac1c8da74f565a9e04da75ffe6cf32cd1439c982fab8476d0 WatchSource:0}: Error finding container d3f150fbcc87d99ac1c8da74f565a9e04da75ffe6cf32cd1439c982fab8476d0: Status 404 returned error can't find the container with id d3f150fbcc87d99ac1c8da74f565a9e04da75ffe6cf32cd1439c982fab8476d0 Sep 08 03:55:42.127844 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:42.127827 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" Sep 08 03:55:42.136406 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:42.135676 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85872ada_b39c_4c47_9286_d77ceca6615d.slice/crio-9113bf7b02acdf958842d16bab29b961d77f6b55e1c929874e13c8ce5712076e WatchSource:0}: Error finding container 9113bf7b02acdf958842d16bab29b961d77f6b55e1c929874e13c8ce5712076e: Status 404 returned error can't find the container with id 9113bf7b02acdf958842d16bab29b961d77f6b55e1c929874e13c8ce5712076e Sep 08 03:55:42.140574 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:42.140560 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:55:42.146345 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:42.146325 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2rrvn" Sep 08 03:55:42.148127 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:42.148105 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45ae9afd_2dea_43b8_aa31_169b11e60fd2.slice/crio-5c8dba1590718e7df1a0c5249cd1868973598b031ff758ede699473df21a7938 WatchSource:0}: Error finding container 5c8dba1590718e7df1a0c5249cd1868973598b031ff758ede699473df21a7938: Status 404 returned error can't find the container with id 5c8dba1590718e7df1a0c5249cd1868973598b031ff758ede699473df21a7938 Sep 08 03:55:42.153870 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:42.153837 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc5a49e7_1980_4418_acf0_0be7ad18e2f5.slice/crio-308ac5eea2ab77ad4fad5a0dffba109a195624742425da9f3053f0f21f41cd81 WatchSource:0}: Error finding container 308ac5eea2ab77ad4fad5a0dffba109a195624742425da9f3053f0f21f41cd81: Status 404 returned error can't find the container with id 308ac5eea2ab77ad4fad5a0dffba109a195624742425da9f3053f0f21f41cd81 Sep 08 03:55:42.166194 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:42.166174 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-492b5" Sep 08 03:55:42.173954 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:42.173929 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7e94ee9_2c58_4da9_9fa9_0364a55475a1.slice/crio-b17e2acdb5c51f53c0a8270eb179e7f3364e0952fdee9372d1eccb80359e970b WatchSource:0}: Error finding container b17e2acdb5c51f53c0a8270eb179e7f3364e0952fdee9372d1eccb80359e970b: Status 404 returned error can't find the container with id b17e2acdb5c51f53c0a8270eb179e7f3364e0952fdee9372d1eccb80359e970b Sep 08 03:55:42.180388 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:42.180373 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wmrvz" Sep 08 03:55:42.187884 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:42.187866 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod308e3330_d662_4672_8e7f_9892c04a5491.slice/crio-de9bf2d57920e7c9811e7805c0f9b0a4e3c25a4595bcf32580ee07ab27dd5136 WatchSource:0}: Error finding container de9bf2d57920e7c9811e7805c0f9b0a4e3c25a4595bcf32580ee07ab27dd5136: Status 404 returned error can't find the container with id de9bf2d57920e7c9811e7805c0f9b0a4e3c25a4595bcf32580ee07ab27dd5136 Sep 08 03:55:42.194603 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:42.194576 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" Sep 08 03:55:42.199410 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:42.199394 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-djmk2" Sep 08 03:55:42.202586 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:42.202568 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32fbd6c2_64d8_4c05_9ae1_ad267d8f8150.slice/crio-d2b35b2331325250457f9ca895c14bff83473564da15ee159709eef5987a4afb WatchSource:0}: Error finding container d2b35b2331325250457f9ca895c14bff83473564da15ee159709eef5987a4afb: Status 404 returned error can't find the container with id d2b35b2331325250457f9ca895c14bff83473564da15ee159709eef5987a4afb Sep 08 03:55:42.207221 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:55:42.207202 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec702c02_1c7e_471b_9783_5593eefc92a3.slice/crio-afe725ed28e0c4bee3c9060db51c1fa8b05237318726ba5f374dc9826b41d365 WatchSource:0}: Error finding container afe725ed28e0c4bee3c9060db51c1fa8b05237318726ba5f374dc9826b41d365: Status 404 returned error can't find the container with id afe725ed28e0c4bee3c9060db51c1fa8b05237318726ba5f374dc9826b41d365 Sep 08 03:55:42.467049 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:42.466960 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26877828-beec-4ff6-a322-79cdf54e0039-original-pull-secret\") pod \"global-pull-secret-syncer-hrmrn\" (UID: \"26877828-beec-4ff6-a322-79cdf54e0039\") " pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:55:42.467049 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:42.467008 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs\") pod \"network-metrics-daemon-r4xfl\" (UID: \"b7239679-8052-415f-ae9a-1b20d61d1223\") " pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:55:42.467260 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:42.467131 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 03:55:42.467260 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:42.467192 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs podName:b7239679-8052-415f-ae9a-1b20d61d1223 nodeName:}" failed. No retries permitted until 2026-09-08 03:55:43.467173313 +0000 UTC m=+3.088209264 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs") pod "network-metrics-daemon-r4xfl" (UID: "b7239679-8052-415f-ae9a-1b20d61d1223") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 03:55:42.467545 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:42.467437 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 03:55:42.467545 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:42.467514 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26877828-beec-4ff6-a322-79cdf54e0039-original-pull-secret podName:26877828-beec-4ff6-a322-79cdf54e0039 nodeName:}" failed. No retries permitted until 2026-09-08 03:55:43.467497115 +0000 UTC m=+3.088533066 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/26877828-beec-4ff6-a322-79cdf54e0039-original-pull-secret") pod "global-pull-secret-syncer-hrmrn" (UID: "26877828-beec-4ff6-a322-79cdf54e0039") : object "kube-system"/"original-pull-secret" not registered Sep 08 03:55:42.567860 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:42.567777 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjmgb\" (UniqueName: \"kubernetes.io/projected/c9c565ac-fd53-499d-b60a-e0af77499433-kube-api-access-wjmgb\") pod \"network-check-target-xj4pw\" (UID: \"c9c565ac-fd53-499d-b60a-e0af77499433\") " pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:55:42.568024 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:42.567992 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 03:55:42.568024 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:42.568014 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 03:55:42.568241 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:42.568027 2579 projected.go:196] Error preparing data for projected volume kube-api-access-wjmgb for pod openshift-network-diagnostics/network-check-target-xj4pw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 03:55:42.568241 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:42.568084 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9c565ac-fd53-499d-b60a-e0af77499433-kube-api-access-wjmgb podName:c9c565ac-fd53-499d-b60a-e0af77499433 nodeName:}" failed. No retries permitted until 2026-09-08 03:55:43.568065288 +0000 UTC m=+3.189101237 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wjmgb" (UniqueName: "kubernetes.io/projected/c9c565ac-fd53-499d-b60a-e0af77499433-kube-api-access-wjmgb") pod "network-check-target-xj4pw" (UID: "c9c565ac-fd53-499d-b60a-e0af77499433") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 03:55:42.808519 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:42.808333 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 03:50:41 +0000 UTC" deadline="2028-05-05 06:34:36.979910712 +0000 UTC" Sep 08 03:55:42.808519 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:42.808397 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14522h38m54.171517694s" Sep 08 03:55:42.871911 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:42.871881 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:55:42.872070 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:42.872008 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xfl" podUID="b7239679-8052-415f-ae9a-1b20d61d1223" Sep 08 03:55:42.915308 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:42.915174 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" event={"ID":"32fbd6c2-64d8-4c05-9ae1-ad267d8f8150","Type":"ContainerStarted","Data":"d2b35b2331325250457f9ca895c14bff83473564da15ee159709eef5987a4afb"} Sep 08 03:55:42.922401 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:42.922357 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" event={"ID":"45ae9afd-2dea-43b8-aa31-169b11e60fd2","Type":"ContainerStarted","Data":"5c8dba1590718e7df1a0c5249cd1868973598b031ff758ede699473df21a7938"} Sep 08 03:55:42.952417 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:42.952368 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" event={"ID":"85872ada-b39c-4c47-9286-d77ceca6615d","Type":"ContainerStarted","Data":"9113bf7b02acdf958842d16bab29b961d77f6b55e1c929874e13c8ce5712076e"} Sep 08 03:55:42.955239 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:42.955216 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 03:55:42.967373 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:42.967338 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-l47nd" event={"ID":"a60305a7-deaa-4247-b229-9ff28b1ad55b","Type":"ContainerStarted","Data":"d3f150fbcc87d99ac1c8da74f565a9e04da75ffe6cf32cd1439c982fab8476d0"} Sep 08 03:55:42.988739 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:42.988646 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nft7t" event={"ID":"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85","Type":"ContainerStarted","Data":"dff3b75efae918dadabedff3e4a9a2553746d068c55f9c3b80f5985f5900376b"} Sep 08 03:55:43.006933 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:43.006895 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal" event={"ID":"ed2033b059035428790cef4fd0e866ec","Type":"ContainerStarted","Data":"795a97ca3878f1219f2ed8cb197ddf748e6e86bb6e63f7fe1eeb605061437ae4"} Sep 08 03:55:43.029287 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:43.029248 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-141.ec2.internal" event={"ID":"f6f3f87e6fd344b94ef9c0dd58f8052f","Type":"ContainerStarted","Data":"c5412b6190610b6ac687f0cdf08dc29e4dc241714802129837cb8006de5769db"} Sep 08 03:55:43.056965 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:43.056700 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-djmk2" event={"ID":"ec702c02-1c7e-471b-9783-5593eefc92a3","Type":"ContainerStarted","Data":"afe725ed28e0c4bee3c9060db51c1fa8b05237318726ba5f374dc9826b41d365"} Sep 08 03:55:43.086025 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:43.085945 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wmrvz" event={"ID":"308e3330-d662-4672-8e7f-9892c04a5491","Type":"ContainerStarted","Data":"de9bf2d57920e7c9811e7805c0f9b0a4e3c25a4595bcf32580ee07ab27dd5136"} Sep 08 03:55:43.104904 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:43.104865 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-492b5" event={"ID":"e7e94ee9-2c58-4da9-9fa9-0364a55475a1","Type":"ContainerStarted","Data":"b17e2acdb5c51f53c0a8270eb179e7f3364e0952fdee9372d1eccb80359e970b"} Sep 08 03:55:43.111894 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:43.111862 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2rrvn" event={"ID":"cc5a49e7-1980-4418-acf0-0be7ad18e2f5","Type":"ContainerStarted","Data":"308ac5eea2ab77ad4fad5a0dffba109a195624742425da9f3053f0f21f41cd81"} Sep 08 03:55:43.282158 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:43.282124 2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 03:55:43.477603 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:43.477512 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26877828-beec-4ff6-a322-79cdf54e0039-original-pull-secret\") pod \"global-pull-secret-syncer-hrmrn\" (UID: \"26877828-beec-4ff6-a322-79cdf54e0039\") " pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:55:43.477603 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:43.477568 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs\") pod \"network-metrics-daemon-r4xfl\" (UID: \"b7239679-8052-415f-ae9a-1b20d61d1223\") " pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:55:43.477825 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:43.477686 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 03:55:43.477825 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:43.477739 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 03:55:43.477825 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:43.477764 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26877828-beec-4ff6-a322-79cdf54e0039-original-pull-secret podName:26877828-beec-4ff6-a322-79cdf54e0039 nodeName:}" failed. No retries permitted until 2026-09-08 03:55:45.477742637 +0000 UTC m=+5.098778632 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/26877828-beec-4ff6-a322-79cdf54e0039-original-pull-secret") pod "global-pull-secret-syncer-hrmrn" (UID: "26877828-beec-4ff6-a322-79cdf54e0039") : object "kube-system"/"original-pull-secret" not registered Sep 08 03:55:43.477825 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:43.477793 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs podName:b7239679-8052-415f-ae9a-1b20d61d1223 nodeName:}" failed. No retries permitted until 2026-09-08 03:55:45.477776378 +0000 UTC m=+5.098812328 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs") pod "network-metrics-daemon-r4xfl" (UID: "b7239679-8052-415f-ae9a-1b20d61d1223") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 03:55:43.578371 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:43.578333 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjmgb\" (UniqueName: \"kubernetes.io/projected/c9c565ac-fd53-499d-b60a-e0af77499433-kube-api-access-wjmgb\") pod \"network-check-target-xj4pw\" (UID: \"c9c565ac-fd53-499d-b60a-e0af77499433\") " pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:55:43.578573 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:43.578526 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 03:55:43.578573 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:43.578550 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 03:55:43.578573 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:43.578565 2579 projected.go:196] Error preparing data for projected volume kube-api-access-wjmgb for pod openshift-network-diagnostics/network-check-target-xj4pw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 03:55:43.578731 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:43.578625 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9c565ac-fd53-499d-b60a-e0af77499433-kube-api-access-wjmgb podName:c9c565ac-fd53-499d-b60a-e0af77499433 nodeName:}" failed. No retries permitted until 2026-09-08 03:55:45.578606837 +0000 UTC m=+5.199642801 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wjmgb" (UniqueName: "kubernetes.io/projected/c9c565ac-fd53-499d-b60a-e0af77499433-kube-api-access-wjmgb") pod "network-check-target-xj4pw" (UID: "c9c565ac-fd53-499d-b60a-e0af77499433") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 03:55:43.601868 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:43.601835 2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 03:55:43.808676 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:43.808559 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 03:50:41 +0000 UTC" deadline="2028-05-26 17:34:55.738148474 +0000 UTC" Sep 08 03:55:43.808676 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:43.808627 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15037h39m11.929525964s" Sep 08 03:55:43.869894 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:43.869859 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:55:43.870073 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:43.870000 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrmrn" podUID="26877828-beec-4ff6-a322-79cdf54e0039" Sep 08 03:55:43.870422 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:43.870401 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:55:43.870588 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:43.870568 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj4pw" podUID="c9c565ac-fd53-499d-b60a-e0af77499433" Sep 08 03:55:44.870975 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:44.870471 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:55:44.870975 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:44.870623 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xfl" podUID="b7239679-8052-415f-ae9a-1b20d61d1223" Sep 08 03:55:45.493928 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:45.493883 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26877828-beec-4ff6-a322-79cdf54e0039-original-pull-secret\") pod \"global-pull-secret-syncer-hrmrn\" (UID: \"26877828-beec-4ff6-a322-79cdf54e0039\") " pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:55:45.494114 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:45.493947 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs\") pod \"network-metrics-daemon-r4xfl\" (UID: \"b7239679-8052-415f-ae9a-1b20d61d1223\") " pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:55:45.494114 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:45.494080 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 03:55:45.494226 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:45.494140 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs podName:b7239679-8052-415f-ae9a-1b20d61d1223 nodeName:}" failed. No retries permitted until 2026-09-08 03:55:49.494122837 +0000 UTC m=+9.115158787 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs") pod "network-metrics-daemon-r4xfl" (UID: "b7239679-8052-415f-ae9a-1b20d61d1223") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 03:55:45.494851 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:45.494827 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 03:55:45.494964 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:45.494895 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26877828-beec-4ff6-a322-79cdf54e0039-original-pull-secret podName:26877828-beec-4ff6-a322-79cdf54e0039 nodeName:}" failed. No retries permitted until 2026-09-08 03:55:49.494875146 +0000 UTC m=+9.115911097 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/26877828-beec-4ff6-a322-79cdf54e0039-original-pull-secret") pod "global-pull-secret-syncer-hrmrn" (UID: "26877828-beec-4ff6-a322-79cdf54e0039") : object "kube-system"/"original-pull-secret" not registered Sep 08 03:55:45.595920 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:45.595270 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjmgb\" (UniqueName: \"kubernetes.io/projected/c9c565ac-fd53-499d-b60a-e0af77499433-kube-api-access-wjmgb\") pod \"network-check-target-xj4pw\" (UID: \"c9c565ac-fd53-499d-b60a-e0af77499433\") " pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:55:45.595920 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:45.595498 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 03:55:45.595920 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:45.595526 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 03:55:45.595920 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:45.595538 2579 projected.go:196] Error preparing data for projected volume kube-api-access-wjmgb for pod openshift-network-diagnostics/network-check-target-xj4pw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 03:55:45.595920 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:45.595596 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9c565ac-fd53-499d-b60a-e0af77499433-kube-api-access-wjmgb podName:c9c565ac-fd53-499d-b60a-e0af77499433 nodeName:}" failed. No retries permitted until 2026-09-08 03:55:49.595578099 +0000 UTC m=+9.216614044 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wjmgb" (UniqueName: "kubernetes.io/projected/c9c565ac-fd53-499d-b60a-e0af77499433-kube-api-access-wjmgb") pod "network-check-target-xj4pw" (UID: "c9c565ac-fd53-499d-b60a-e0af77499433") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 03:55:45.869874 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:45.869836 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:55:45.869874 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:45.869862 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:55:45.870121 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:45.869983 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrmrn" podUID="26877828-beec-4ff6-a322-79cdf54e0039" Sep 08 03:55:45.870294 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:45.870258 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj4pw" podUID="c9c565ac-fd53-499d-b60a-e0af77499433" Sep 08 03:55:46.871055 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:46.871026 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:55:46.871676 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:46.871143 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xfl" podUID="b7239679-8052-415f-ae9a-1b20d61d1223" Sep 08 03:55:47.870502 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:47.870452 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:55:47.870687 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:47.870595 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrmrn" podUID="26877828-beec-4ff6-a322-79cdf54e0039" Sep 08 03:55:47.871065 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:47.870865 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:55:47.871065 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:47.870974 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj4pw" podUID="c9c565ac-fd53-499d-b60a-e0af77499433" Sep 08 03:55:48.871005 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:48.870970 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:55:48.871194 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:48.871115 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xfl" podUID="b7239679-8052-415f-ae9a-1b20d61d1223" Sep 08 03:55:49.531225 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:49.531179 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26877828-beec-4ff6-a322-79cdf54e0039-original-pull-secret\") pod \"global-pull-secret-syncer-hrmrn\" (UID: \"26877828-beec-4ff6-a322-79cdf54e0039\") " pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:55:49.531395 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:49.531236 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs\") pod \"network-metrics-daemon-r4xfl\" (UID: \"b7239679-8052-415f-ae9a-1b20d61d1223\") " pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:55:49.531395 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:49.531379 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 03:55:49.531550 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:49.531449 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26877828-beec-4ff6-a322-79cdf54e0039-original-pull-secret podName:26877828-beec-4ff6-a322-79cdf54e0039 nodeName:}" failed. No retries permitted until 2026-09-08 03:55:57.531433738 +0000 UTC m=+17.152469688 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/26877828-beec-4ff6-a322-79cdf54e0039-original-pull-secret") pod "global-pull-secret-syncer-hrmrn" (UID: "26877828-beec-4ff6-a322-79cdf54e0039") : object "kube-system"/"original-pull-secret" not registered Sep 08 03:55:49.531550 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:49.531384 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 03:55:49.531550 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:49.531531 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs podName:b7239679-8052-415f-ae9a-1b20d61d1223 nodeName:}" failed. No retries permitted until 2026-09-08 03:55:57.53151346 +0000 UTC m=+17.152549410 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs") pod "network-metrics-daemon-r4xfl" (UID: "b7239679-8052-415f-ae9a-1b20d61d1223") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 03:55:49.632942 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:49.632244 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjmgb\" (UniqueName: \"kubernetes.io/projected/c9c565ac-fd53-499d-b60a-e0af77499433-kube-api-access-wjmgb\") pod \"network-check-target-xj4pw\" (UID: \"c9c565ac-fd53-499d-b60a-e0af77499433\") " pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:55:49.632942 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:49.632424 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 03:55:49.632942 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:49.632442 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 03:55:49.632942 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:49.632455 2579 projected.go:196] Error preparing data for projected volume kube-api-access-wjmgb for pod openshift-network-diagnostics/network-check-target-xj4pw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 03:55:49.632942 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:49.632538 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9c565ac-fd53-499d-b60a-e0af77499433-kube-api-access-wjmgb podName:c9c565ac-fd53-499d-b60a-e0af77499433 nodeName:}" failed. No retries permitted until 2026-09-08 03:55:57.632519949 +0000 UTC m=+17.253555910 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wjmgb" (UniqueName: "kubernetes.io/projected/c9c565ac-fd53-499d-b60a-e0af77499433-kube-api-access-wjmgb") pod "network-check-target-xj4pw" (UID: "c9c565ac-fd53-499d-b60a-e0af77499433") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 03:55:49.869834 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:49.869801 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:55:49.869997 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:49.869931 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrmrn" podUID="26877828-beec-4ff6-a322-79cdf54e0039" Sep 08 03:55:49.870323 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:49.870169 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:55:49.870323 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:49.870285 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj4pw" podUID="c9c565ac-fd53-499d-b60a-e0af77499433" Sep 08 03:55:50.871305 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:50.870823 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:55:50.871305 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:50.870950 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xfl" podUID="b7239679-8052-415f-ae9a-1b20d61d1223" Sep 08 03:55:51.870206 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:51.870165 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:55:51.870206 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:51.870203 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:55:51.870506 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:51.870303 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrmrn" podUID="26877828-beec-4ff6-a322-79cdf54e0039" Sep 08 03:55:51.870506 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:51.870430 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj4pw" podUID="c9c565ac-fd53-499d-b60a-e0af77499433" Sep 08 03:55:52.869513 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:52.869485 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:55:52.869978 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:52.869619 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xfl" podUID="b7239679-8052-415f-ae9a-1b20d61d1223" Sep 08 03:55:53.869466 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:53.869418 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:55:53.869628 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:53.869420 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:55:53.869628 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:53.869580 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrmrn" podUID="26877828-beec-4ff6-a322-79cdf54e0039" Sep 08 03:55:53.869973 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:53.869657 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj4pw" podUID="c9c565ac-fd53-499d-b60a-e0af77499433" Sep 08 03:55:54.869695 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:54.869661 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:55:54.870100 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:54.869813 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xfl" podUID="b7239679-8052-415f-ae9a-1b20d61d1223" Sep 08 03:55:55.869920 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:55.869885 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:55:55.870397 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:55.869886 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:55:55.870397 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:55.870029 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj4pw" podUID="c9c565ac-fd53-499d-b60a-e0af77499433" Sep 08 03:55:55.870397 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:55.870097 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrmrn" podUID="26877828-beec-4ff6-a322-79cdf54e0039" Sep 08 03:55:56.870050 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:56.870022 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:55:56.870406 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:56.870171 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xfl" podUID="b7239679-8052-415f-ae9a-1b20d61d1223" Sep 08 03:55:57.589069 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:57.589028 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26877828-beec-4ff6-a322-79cdf54e0039-original-pull-secret\") pod \"global-pull-secret-syncer-hrmrn\" (UID: \"26877828-beec-4ff6-a322-79cdf54e0039\") " pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:55:57.589069 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:57.589074 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs\") pod \"network-metrics-daemon-r4xfl\" (UID: \"b7239679-8052-415f-ae9a-1b20d61d1223\") " pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:55:57.589394 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:57.589184 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 03:55:57.589394 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:57.589249 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs podName:b7239679-8052-415f-ae9a-1b20d61d1223 nodeName:}" failed. No retries permitted until 2026-09-08 03:56:13.589233372 +0000 UTC m=+33.210269317 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs") pod "network-metrics-daemon-r4xfl" (UID: "b7239679-8052-415f-ae9a-1b20d61d1223") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 03:55:57.589394 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:57.589184 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 03:55:57.589394 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:57.589314 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26877828-beec-4ff6-a322-79cdf54e0039-original-pull-secret podName:26877828-beec-4ff6-a322-79cdf54e0039 nodeName:}" failed. No retries permitted until 2026-09-08 03:56:13.589302768 +0000 UTC m=+33.210338715 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/26877828-beec-4ff6-a322-79cdf54e0039-original-pull-secret") pod "global-pull-secret-syncer-hrmrn" (UID: "26877828-beec-4ff6-a322-79cdf54e0039") : object "kube-system"/"original-pull-secret" not registered Sep 08 03:55:57.689442 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:57.689400 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjmgb\" (UniqueName: \"kubernetes.io/projected/c9c565ac-fd53-499d-b60a-e0af77499433-kube-api-access-wjmgb\") pod \"network-check-target-xj4pw\" (UID: \"c9c565ac-fd53-499d-b60a-e0af77499433\") " pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:55:57.689616 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:57.689594 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 03:55:57.689686 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:57.689628 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 03:55:57.689686 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:57.689642 2579 projected.go:196] Error preparing data for projected volume kube-api-access-wjmgb for pod openshift-network-diagnostics/network-check-target-xj4pw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 03:55:57.689766 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:57.689710 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9c565ac-fd53-499d-b60a-e0af77499433-kube-api-access-wjmgb podName:c9c565ac-fd53-499d-b60a-e0af77499433 nodeName:}" failed. No retries permitted until 2026-09-08 03:56:13.689689268 +0000 UTC m=+33.310725213 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wjmgb" (UniqueName: "kubernetes.io/projected/c9c565ac-fd53-499d-b60a-e0af77499433-kube-api-access-wjmgb") pod "network-check-target-xj4pw" (UID: "c9c565ac-fd53-499d-b60a-e0af77499433") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 03:55:57.869654 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:57.869573 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:55:57.869819 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:57.869573 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:55:57.869819 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:57.869699 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrmrn" podUID="26877828-beec-4ff6-a322-79cdf54e0039" Sep 08 03:55:57.869819 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:57.869776 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj4pw" podUID="c9c565ac-fd53-499d-b60a-e0af77499433" Sep 08 03:55:58.869937 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:58.869902 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:55:58.870362 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:58.870052 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xfl" podUID="b7239679-8052-415f-ae9a-1b20d61d1223" Sep 08 03:55:59.869592 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:59.869551 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:55:59.869771 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:55:59.869551 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:55:59.869771 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:59.869699 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrmrn" podUID="26877828-beec-4ff6-a322-79cdf54e0039" Sep 08 03:55:59.869771 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:55:59.869729 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj4pw" podUID="c9c565ac-fd53-499d-b60a-e0af77499433" Sep 08 03:56:00.870471 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:00.870423 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:56:00.870935 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:00.870545 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xfl" podUID="b7239679-8052-415f-ae9a-1b20d61d1223" Sep 08 03:56:01.152876 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:01.152834 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" event={"ID":"45ae9afd-2dea-43b8-aa31-169b11e60fd2","Type":"ContainerStarted","Data":"955e1b348b2ca46ecf8b94b7d24e806b278b311fdb751e9e7821acd11dfc0512"} Sep 08 03:56:01.155245 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:01.155219 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" event={"ID":"85872ada-b39c-4c47-9286-d77ceca6615d","Type":"ContainerStarted","Data":"3986ffd2eb4b32b07e94b15ffa3b7c9c4fb0df4fe60eb520cab621e23274352a"} Sep 08 03:56:01.156905 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:01.156882 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-141.ec2.internal" event={"ID":"f6f3f87e6fd344b94ef9c0dd58f8052f","Type":"ContainerStarted","Data":"3da40cfbc83d792fdbcf2de53e05aa1cc12d03061fbfe8051a077da734d7adc8"} Sep 08 03:56:01.212565 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:01.212516 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-h8sh5" podStartSLOduration=2.581148457 podStartE2EDuration="21.212498664s" totalImagesPullingTime="18.631350207s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 03:55:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 03:56:01.187934785 +0000 UTC m=+20.808970751" watchObservedRunningTime="2026-09-08 03:56:01.212498664 +0000 UTC m=+20.833534635" Sep 08 03:56:01.870805 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:01.870586 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:56:01.871437 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:01.870911 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj4pw" podUID="c9c565ac-fd53-499d-b60a-e0af77499433" Sep 08 03:56:01.871437 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:01.871027 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:56:01.871437 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:01.871110 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrmrn" podUID="26877828-beec-4ff6-a322-79cdf54e0039" Sep 08 03:56:02.161111 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:02.161071 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" event={"ID":"32fbd6c2-64d8-4c05-9ae1-ad267d8f8150","Type":"ContainerStarted","Data":"b67bc9deaf2ec68f475ce8415e50a40c2dd54a45a3c6212c943532c94cd898bd"} Sep 08 03:56:02.163791 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:02.163762 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tg7hb_45ae9afd-2dea-43b8-aa31-169b11e60fd2/ovn-acl-logging/0.log" Sep 08 03:56:02.164144 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:02.164124 2579 generic.go:356] "Generic (PLEG): container finished" podID="45ae9afd-2dea-43b8-aa31-169b11e60fd2" containerID="e8f3a3bea7c1522d96b43504b71d5b73ac2d421ee171c8310e2df4edb71caaaa" exitCode=1 Sep 08 03:56:02.164219 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:02.164200 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" event={"ID":"45ae9afd-2dea-43b8-aa31-169b11e60fd2","Type":"ContainerStarted","Data":"15829916ebb470e4bee31f71a73114fb8439fb41cdd0e13d66c851de3f5fffc1"} Sep 08 03:56:02.164271 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:02.164261 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" event={"ID":"45ae9afd-2dea-43b8-aa31-169b11e60fd2","Type":"ContainerStarted","Data":"85a045e726022d31ee976051a42e945e2a8b6c632ffbd8f00eaae570a30103fd"} Sep 08 03:56:02.164310 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:02.164299 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" event={"ID":"45ae9afd-2dea-43b8-aa31-169b11e60fd2","Type":"ContainerStarted","Data":"298c4891aa638fad2b3fb09ae9192d7edd44dff62cbd4fbeec0e7d37add76c3c"} Sep 08 03:56:02.164348 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:02.164332 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" event={"ID":"45ae9afd-2dea-43b8-aa31-169b11e60fd2","Type":"ContainerStarted","Data":"c1e5c1e15fd16e1b9ea04283c38f214cd636b398c1aba2035192bdd822039068"} Sep 08 03:56:02.164387 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:02.164363 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" event={"ID":"45ae9afd-2dea-43b8-aa31-169b11e60fd2","Type":"ContainerDied","Data":"e8f3a3bea7c1522d96b43504b71d5b73ac2d421ee171c8310e2df4edb71caaaa"} Sep 08 03:56:02.165644 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:02.165625 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-l47nd" event={"ID":"a60305a7-deaa-4247-b229-9ff28b1ad55b","Type":"ContainerStarted","Data":"466b022ddc566799aa2abdcc424dedf74ea2cba3add919c907cec7b36815929e"} Sep 08 03:56:02.167073 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:02.167052 2579 generic.go:356] "Generic (PLEG): container finished" podID="9a603e06-e5e6-4bc5-b352-42bb6dcdbb85" containerID="3d52ea70837582007efeceb6124e1938c5b45dd86b1550cb3800ec920f1b7a3a" exitCode=0 Sep 08 03:56:02.167149 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:02.167127 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nft7t" event={"ID":"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85","Type":"ContainerDied","Data":"3d52ea70837582007efeceb6124e1938c5b45dd86b1550cb3800ec920f1b7a3a"} Sep 08 03:56:02.168773 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:02.168748 2579 generic.go:356] "Generic (PLEG): container finished" podID="ed2033b059035428790cef4fd0e866ec" containerID="86efb55d5476c99078f69d9515516c760579cd0a84a9cf0b40f17bcae7d58fc4" exitCode=0 Sep 08 03:56:02.168858 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:02.168820 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal" event={"ID":"ed2033b059035428790cef4fd0e866ec","Type":"ContainerDied","Data":"86efb55d5476c99078f69d9515516c760579cd0a84a9cf0b40f17bcae7d58fc4"} Sep 08 03:56:02.170406 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:02.170379 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-djmk2" event={"ID":"ec702c02-1c7e-471b-9783-5593eefc92a3","Type":"ContainerStarted","Data":"0a6d9c3ef9bf3afb10812c24700dc3c1ef9ecb982c07ebafa781170f655e5dcc"} Sep 08 03:56:02.171849 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:02.171825 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wmrvz" event={"ID":"308e3330-d662-4672-8e7f-9892c04a5491","Type":"ContainerStarted","Data":"16daa34141d38d1e04be71e5627797cde2584bb25ebcf45ca475f16560a691c6"} Sep 08 03:56:02.173247 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:02.173214 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2rrvn" event={"ID":"cc5a49e7-1980-4418-acf0-0be7ad18e2f5","Type":"ContainerStarted","Data":"b53b7ffa513cf3e9af96048b16ed8c127f26cd1deb538d98e4770da6a53f447a"} Sep 08 03:56:02.199401 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:02.199350 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-141.ec2.internal" podStartSLOduration=21.199330937 podStartE2EDuration="21.199330937s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 03:55:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 03:56:01.214765614 +0000 UTC m=+20.835801583" watchObservedRunningTime="2026-09-08 03:56:02.199330937 +0000 UTC m=+21.820366904" Sep 08 03:56:02.230377 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:02.230335 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-l47nd" podStartSLOduration=2.716840883 podStartE2EDuration="22.230319502s" totalImagesPullingTime="19.513478619s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 03:55:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 03:56:02.200618506 +0000 UTC m=+21.821654473" watchObservedRunningTime="2026-09-08 03:56:02.230319502 +0000 UTC m=+21.851355532" Sep 08 03:56:02.252168 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:02.252105 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wmrvz" podStartSLOduration=2.702035713 podStartE2EDuration="21.252091851s" totalImagesPullingTime="18.550056138s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 03:55:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 03:56:02.25199228 +0000 UTC m=+21.873028249" watchObservedRunningTime="2026-09-08 03:56:02.252091851 +0000 UTC m=+21.873127820" Sep 08 03:56:02.357707 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:02.357636 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2rrvn" podStartSLOduration=2.7901288539999998 podStartE2EDuration="21.357615197s" totalImagesPullingTime="18.567486343s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 03:55:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 03:56:02.357433914 +0000 UTC m=+21.978469880" watchObservedRunningTime="2026-09-08 03:56:02.357615197 +0000 UTC m=+21.978651167" Sep 08 03:56:02.870079 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:02.870039 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:56:02.870268 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:02.870196 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xfl" podUID="b7239679-8052-415f-ae9a-1b20d61d1223" Sep 08 03:56:03.177894 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:03.177858 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal" event={"ID":"ed2033b059035428790cef4fd0e866ec","Type":"ContainerStarted","Data":"b91f40139670e80a5fa4ae1cd87a0fc92775b791bd20023a157f1cc7ccea1d63"} Sep 08 03:56:03.179840 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:03.179812 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-492b5" event={"ID":"e7e94ee9-2c58-4da9-9fa9-0364a55475a1","Type":"ContainerStarted","Data":"d396bfaaa0a83251acc609a815ace864de871d42b444c81dfc0dc056990ad9ca"} Sep 08 03:56:03.194523 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:03.194491 2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 08 03:56:03.209998 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:03.209942 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-djmk2" podStartSLOduration=3.6957586449999997 podStartE2EDuration="22.209919507s" totalImagesPullingTime="18.514160862s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 03:55:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 03:56:02.419768798 +0000 UTC m=+22.040804773" watchObservedRunningTime="2026-09-08 03:56:03.209919507 +0000 UTC m=+22.830955484" Sep 08 03:56:03.247973 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:03.247923 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal" podStartSLOduration=22.247902678 podStartE2EDuration="22.247902678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 03:55:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 03:56:03.210323589 +0000 UTC m=+22.831359556" watchObservedRunningTime="2026-09-08 03:56:03.247902678 +0000 UTC m=+22.868938650" Sep 08 03:56:03.248425 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:03.248386 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-492b5" podStartSLOduration=3.665500057 podStartE2EDuration="22.248376344s" totalImagesPullingTime="18.582876287s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 03:55:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 03:56:03.24808602 +0000 UTC m=+22.869121989" watchObservedRunningTime="2026-09-08 03:56:03.248376344 +0000 UTC m=+22.869412314" Sep 08 03:56:03.807855 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:03.807733 2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-08T03:56:03.194514791Z","UUID":"4a697c6a-01e6-4645-ac6f-bc7451e47fd0","Handler":null,"Name":"","Endpoint":""} Sep 08 03:56:03.809533 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:03.809508 2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 08 03:56:03.809673 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:03.809553 2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 08 03:56:03.870234 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:03.870200 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:56:03.870421 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:03.870346 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrmrn" podUID="26877828-beec-4ff6-a322-79cdf54e0039" Sep 08 03:56:03.870421 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:03.870388 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:56:03.870567 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:03.870527 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj4pw" podUID="c9c565ac-fd53-499d-b60a-e0af77499433" Sep 08 03:56:04.184845 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:04.184811 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" event={"ID":"32fbd6c2-64d8-4c05-9ae1-ad267d8f8150","Type":"ContainerStarted","Data":"136f1b6ddefbe849fedc2416f3e57b6fe3c9294aa6be61d6da96900a7d97eded"} Sep 08 03:56:04.188402 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:04.188377 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tg7hb_45ae9afd-2dea-43b8-aa31-169b11e60fd2/ovn-acl-logging/0.log" Sep 08 03:56:04.188949 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:04.188920 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" event={"ID":"45ae9afd-2dea-43b8-aa31-169b11e60fd2","Type":"ContainerStarted","Data":"bb481c5f5af65d928c8a13f647f2342c8cf31b1b14e72e60c5bbc6eeeb8a5803"} Sep 08 03:56:04.870070 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:04.870039 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:56:04.870251 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:04.870166 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xfl" podUID="b7239679-8052-415f-ae9a-1b20d61d1223" Sep 08 03:56:05.194307 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:05.194212 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" event={"ID":"32fbd6c2-64d8-4c05-9ae1-ad267d8f8150","Type":"ContainerStarted","Data":"526fb830126460d88bb60debcdd95a79ee3aa95099da6b0052b8ff98b5832691"} Sep 08 03:56:05.869736 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:05.869700 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:56:05.869902 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:05.869699 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:56:05.869902 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:05.869839 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj4pw" podUID="c9c565ac-fd53-499d-b60a-e0af77499433" Sep 08 03:56:05.870021 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:05.869931 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrmrn" podUID="26877828-beec-4ff6-a322-79cdf54e0039" Sep 08 03:56:06.533089 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:06.532902 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2rrvn" Sep 08 03:56:06.533559 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:06.533404 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2rrvn" Sep 08 03:56:06.553895 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:06.553854 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47grd" podStartSLOduration=4.649215644 podStartE2EDuration="25.553838161s" totalImagesPullingTime="20.904622517s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 03:55:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 03:56:05.226603291 +0000 UTC m=+24.847639258" watchObservedRunningTime="2026-09-08 03:56:06.553838161 +0000 UTC m=+26.174874128" Sep 08 03:56:06.870057 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:06.870023 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:56:06.870240 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:06.870144 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xfl" podUID="b7239679-8052-415f-ae9a-1b20d61d1223" Sep 08 03:56:07.202275 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:07.202202 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tg7hb_45ae9afd-2dea-43b8-aa31-169b11e60fd2/ovn-acl-logging/0.log" Sep 08 03:56:07.202660 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:07.202637 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" event={"ID":"45ae9afd-2dea-43b8-aa31-169b11e60fd2","Type":"ContainerStarted","Data":"07d2419a3e9da3297b58a328f63d175fc1dcc95f8ac265f64455f016e51a3d23"} Sep 08 03:56:07.202894 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:07.202874 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:56:07.203077 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:07.203059 2579 scope.go:117] "RemoveContainer" containerID="e8f3a3bea7c1522d96b43504b71d5b73ac2d421ee171c8310e2df4edb71caaaa" Sep 08 03:56:07.204711 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:07.204686 2579 generic.go:356] "Generic (PLEG): container finished" podID="9a603e06-e5e6-4bc5-b352-42bb6dcdbb85" containerID="52ef00625323aaffc94e440ee808e8d66daf79010bb9b703f931cfabda0cecfe" exitCode=0 Sep 08 03:56:07.204798 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:07.204717 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nft7t" event={"ID":"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85","Type":"ContainerDied","Data":"52ef00625323aaffc94e440ee808e8d66daf79010bb9b703f931cfabda0cecfe"} Sep 08 03:56:07.205008 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:07.204992 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2rrvn" Sep 08 03:56:07.205475 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:07.205410 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2rrvn" Sep 08 03:56:07.220598 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:07.220570 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:56:07.869849 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:07.869812 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:56:07.869849 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:07.869839 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:56:07.870549 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:07.869941 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrmrn" podUID="26877828-beec-4ff6-a322-79cdf54e0039" Sep 08 03:56:07.870549 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:07.870078 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj4pw" podUID="c9c565ac-fd53-499d-b60a-e0af77499433" Sep 08 03:56:08.211274 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:08.211253 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tg7hb_45ae9afd-2dea-43b8-aa31-169b11e60fd2/ovn-acl-logging/0.log" Sep 08 03:56:08.211695 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:08.211670 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" event={"ID":"45ae9afd-2dea-43b8-aa31-169b11e60fd2","Type":"ContainerStarted","Data":"3c9b4f1d785adedb9609e66bad4b1e6d35c68f19bcd3c9b2dee1ce4bb53cd86f"} Sep 08 03:56:08.211808 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:08.211770 2579 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 08 03:56:08.211903 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:08.211886 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:56:08.214311 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:08.214287 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nft7t" event={"ID":"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85","Type":"ContainerStarted","Data":"7951706da3b94aa9482f5ecf8e6af9f28cfa9a9d8058540e220658102b5c5614"} Sep 08 03:56:08.229092 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:08.229068 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:56:08.294596 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:08.294541 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" podStartSLOduration=8.683920206 podStartE2EDuration="27.294523064s" totalImagesPullingTime="18.610602858s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 03:55:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 03:56:08.261922136 +0000 UTC m=+27.882958103" watchObservedRunningTime="2026-09-08 03:56:08.294523064 +0000 UTC m=+27.915559047" Sep 08 03:56:08.638659 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:08.638628 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xj4pw"] Sep 08 03:56:08.638808 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:08.638757 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:56:08.638857 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:08.638829 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj4pw" podUID="c9c565ac-fd53-499d-b60a-e0af77499433" Sep 08 03:56:08.643036 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:08.643006 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r4xfl"] Sep 08 03:56:08.643178 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:08.643151 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:56:08.643263 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:08.643241 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xfl" podUID="b7239679-8052-415f-ae9a-1b20d61d1223" Sep 08 03:56:08.643521 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:08.643500 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hrmrn"] Sep 08 03:56:08.643604 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:08.643592 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:56:08.643701 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:08.643681 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrmrn" podUID="26877828-beec-4ff6-a322-79cdf54e0039" Sep 08 03:56:09.218984 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:09.218949 2579 generic.go:356] "Generic (PLEG): container finished" podID="9a603e06-e5e6-4bc5-b352-42bb6dcdbb85" containerID="7951706da3b94aa9482f5ecf8e6af9f28cfa9a9d8058540e220658102b5c5614" exitCode=0 Sep 08 03:56:09.219567 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:09.219035 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nft7t" event={"ID":"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85","Type":"ContainerDied","Data":"7951706da3b94aa9482f5ecf8e6af9f28cfa9a9d8058540e220658102b5c5614"} Sep 08 03:56:09.219567 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:09.219156 2579 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 08 03:56:09.869646 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:09.869613 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:56:09.869811 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:09.869766 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrmrn" podUID="26877828-beec-4ff6-a322-79cdf54e0039" Sep 08 03:56:10.221825 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:10.221756 2579 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 08 03:56:10.873006 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:10.872974 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:56:10.873163 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:10.872974 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:56:10.873163 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:10.873092 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xfl" podUID="b7239679-8052-415f-ae9a-1b20d61d1223" Sep 08 03:56:10.873163 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:10.873129 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj4pw" podUID="c9c565ac-fd53-499d-b60a-e0af77499433" Sep 08 03:56:11.174758 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:11.174725 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:56:11.193143 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:11.193101 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" podUID="45ae9afd-2dea-43b8-aa31-169b11e60fd2" containerName="ovnkube-controller" probeResult="failure" output="" Sep 08 03:56:11.201846 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:11.201807 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" podUID="45ae9afd-2dea-43b8-aa31-169b11e60fd2" containerName="ovnkube-controller" probeResult="failure" output="" Sep 08 03:56:11.226733 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:11.226702 2579 generic.go:356] "Generic (PLEG): container finished" podID="9a603e06-e5e6-4bc5-b352-42bb6dcdbb85" containerID="40b06f6ef7d131ac1701129e7d72ff5a1abe22b718072f509487027a0ab153c3" exitCode=0 Sep 08 03:56:11.227262 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:11.226779 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nft7t" event={"ID":"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85","Type":"ContainerDied","Data":"40b06f6ef7d131ac1701129e7d72ff5a1abe22b718072f509487027a0ab153c3"} Sep 08 03:56:11.236399 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:11.236366 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" podUID="45ae9afd-2dea-43b8-aa31-169b11e60fd2" containerName="ovnkube-controller" probeResult="failure" output="" Sep 08 03:56:11.869713 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:11.869676 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:56:11.869906 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:11.869812 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrmrn" podUID="26877828-beec-4ff6-a322-79cdf54e0039" Sep 08 03:56:12.869619 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:12.869583 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:56:12.869619 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:12.869615 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:56:12.870179 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:12.869730 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xfl" podUID="b7239679-8052-415f-ae9a-1b20d61d1223" Sep 08 03:56:12.870179 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:12.869860 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj4pw" podUID="c9c565ac-fd53-499d-b60a-e0af77499433" Sep 08 03:56:13.607216 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:13.607169 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26877828-beec-4ff6-a322-79cdf54e0039-original-pull-secret\") pod \"global-pull-secret-syncer-hrmrn\" (UID: \"26877828-beec-4ff6-a322-79cdf54e0039\") " pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:56:13.607416 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:13.607224 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs\") pod \"network-metrics-daemon-r4xfl\" (UID: \"b7239679-8052-415f-ae9a-1b20d61d1223\") " pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:56:13.607416 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:13.607340 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 03:56:13.607416 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:13.607399 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs podName:b7239679-8052-415f-ae9a-1b20d61d1223 nodeName:}" failed. No retries permitted until 2026-09-08 03:56:45.607385088 +0000 UTC m=+65.228421048 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs") pod "network-metrics-daemon-r4xfl" (UID: "b7239679-8052-415f-ae9a-1b20d61d1223") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 03:56:13.607574 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:13.607337 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 03:56:13.607574 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:13.607496 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26877828-beec-4ff6-a322-79cdf54e0039-original-pull-secret podName:26877828-beec-4ff6-a322-79cdf54e0039 nodeName:}" failed. No retries permitted until 2026-09-08 03:56:45.607481504 +0000 UTC m=+65.228517453 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/26877828-beec-4ff6-a322-79cdf54e0039-original-pull-secret") pod "global-pull-secret-syncer-hrmrn" (UID: "26877828-beec-4ff6-a322-79cdf54e0039") : object "kube-system"/"original-pull-secret" not registered Sep 08 03:56:13.707864 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:13.707821 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjmgb\" (UniqueName: \"kubernetes.io/projected/c9c565ac-fd53-499d-b60a-e0af77499433-kube-api-access-wjmgb\") pod \"network-check-target-xj4pw\" (UID: \"c9c565ac-fd53-499d-b60a-e0af77499433\") " pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:56:13.708084 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:13.708020 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 03:56:13.708084 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:13.708045 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 03:56:13.708084 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:13.708055 2579 projected.go:196] Error preparing data for projected volume kube-api-access-wjmgb for pod openshift-network-diagnostics/network-check-target-xj4pw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 03:56:13.708184 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:13.708108 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9c565ac-fd53-499d-b60a-e0af77499433-kube-api-access-wjmgb podName:c9c565ac-fd53-499d-b60a-e0af77499433 nodeName:}" failed. No retries permitted until 2026-09-08 03:56:45.708093491 +0000 UTC m=+65.329129436 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wjmgb" (UniqueName: "kubernetes.io/projected/c9c565ac-fd53-499d-b60a-e0af77499433-kube-api-access-wjmgb") pod "network-check-target-xj4pw" (UID: "c9c565ac-fd53-499d-b60a-e0af77499433") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 03:56:13.870194 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:13.870150 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:56:13.870906 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:13.870268 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrmrn" podUID="26877828-beec-4ff6-a322-79cdf54e0039" Sep 08 03:56:14.123173 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.123092 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeReady" Sep 08 03:56:14.123339 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.123253 2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 08 03:56:14.221265 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.221232 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2jhc2"] Sep 08 03:56:14.254778 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.254572 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fdn8c"] Sep 08 03:56:14.254778 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.254725 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2jhc2" Sep 08 03:56:14.258409 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.258359 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 08 03:56:14.258743 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.258715 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 08 03:56:14.258861 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.258822 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-sl2fz\"" Sep 08 03:56:14.270100 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.270073 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2jhc2"] Sep 08 03:56:14.270233 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.270115 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fdn8c" Sep 08 03:56:14.270233 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.270146 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fdn8c"] Sep 08 03:56:14.274314 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.274292 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 08 03:56:14.275094 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.274565 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 08 03:56:14.275094 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.274565 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 08 03:56:14.276104 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.276085 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cprn7\"" Sep 08 03:56:14.413259 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.413165 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls\") pod \"dns-default-2jhc2\" (UID: \"77b984c3-52a4-442d-9356-f05a01aaa1bb\") " pod="openshift-dns/dns-default-2jhc2" Sep 08 03:56:14.413259 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.413234 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d27b634-b977-4b2b-b5ad-0766b2b0e1e2-cert\") pod \"ingress-canary-fdn8c\" (UID: \"9d27b634-b977-4b2b-b5ad-0766b2b0e1e2\") " pod="openshift-ingress-canary/ingress-canary-fdn8c" Sep 08 03:56:14.413476 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.413276 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77b984c3-52a4-442d-9356-f05a01aaa1bb-tmp-dir\") pod \"dns-default-2jhc2\" (UID: \"77b984c3-52a4-442d-9356-f05a01aaa1bb\") " pod="openshift-dns/dns-default-2jhc2" Sep 08 03:56:14.413476 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.413316 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvgjj\" (UniqueName: \"kubernetes.io/projected/77b984c3-52a4-442d-9356-f05a01aaa1bb-kube-api-access-jvgjj\") pod \"dns-default-2jhc2\" (UID: \"77b984c3-52a4-442d-9356-f05a01aaa1bb\") " pod="openshift-dns/dns-default-2jhc2" Sep 08 03:56:14.413476 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.413342 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/77b984c3-52a4-442d-9356-f05a01aaa1bb-config-volume\") pod \"dns-default-2jhc2\" (UID: \"77b984c3-52a4-442d-9356-f05a01aaa1bb\") " pod="openshift-dns/dns-default-2jhc2" Sep 08 03:56:14.413476 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.413404 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdkk4\" (UniqueName: \"kubernetes.io/projected/9d27b634-b977-4b2b-b5ad-0766b2b0e1e2-kube-api-access-pdkk4\") pod \"ingress-canary-fdn8c\" (UID: \"9d27b634-b977-4b2b-b5ad-0766b2b0e1e2\") " pod="openshift-ingress-canary/ingress-canary-fdn8c" Sep 08 03:56:14.514500 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.514436 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d27b634-b977-4b2b-b5ad-0766b2b0e1e2-cert\") pod \"ingress-canary-fdn8c\" (UID: \"9d27b634-b977-4b2b-b5ad-0766b2b0e1e2\") " pod="openshift-ingress-canary/ingress-canary-fdn8c" Sep 08 03:56:14.514674 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.514519 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77b984c3-52a4-442d-9356-f05a01aaa1bb-tmp-dir\") pod \"dns-default-2jhc2\" (UID: \"77b984c3-52a4-442d-9356-f05a01aaa1bb\") " pod="openshift-dns/dns-default-2jhc2" Sep 08 03:56:14.514674 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.514558 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvgjj\" (UniqueName: \"kubernetes.io/projected/77b984c3-52a4-442d-9356-f05a01aaa1bb-kube-api-access-jvgjj\") pod \"dns-default-2jhc2\" (UID: \"77b984c3-52a4-442d-9356-f05a01aaa1bb\") " pod="openshift-dns/dns-default-2jhc2" Sep 08 03:56:14.514674 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.514581 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/77b984c3-52a4-442d-9356-f05a01aaa1bb-config-volume\") pod \"dns-default-2jhc2\" (UID: \"77b984c3-52a4-442d-9356-f05a01aaa1bb\") " pod="openshift-dns/dns-default-2jhc2" Sep 08 03:56:14.514674 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.514613 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdkk4\" (UniqueName: \"kubernetes.io/projected/9d27b634-b977-4b2b-b5ad-0766b2b0e1e2-kube-api-access-pdkk4\") pod \"ingress-canary-fdn8c\" (UID: \"9d27b634-b977-4b2b-b5ad-0766b2b0e1e2\") " pod="openshift-ingress-canary/ingress-canary-fdn8c" Sep 08 03:56:14.514674 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.514652 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls\") pod \"dns-default-2jhc2\" (UID: \"77b984c3-52a4-442d-9356-f05a01aaa1bb\") " pod="openshift-dns/dns-default-2jhc2" Sep 08 03:56:14.514863 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:14.514764 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 03:56:14.514863 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:14.514827 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls podName:77b984c3-52a4-442d-9356-f05a01aaa1bb nodeName:}" failed. No retries permitted until 2026-09-08 03:56:15.014808588 +0000 UTC m=+34.635844546 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls") pod "dns-default-2jhc2" (UID: "77b984c3-52a4-442d-9356-f05a01aaa1bb") : secret "dns-default-metrics-tls" not found Sep 08 03:56:14.514948 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.514901 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77b984c3-52a4-442d-9356-f05a01aaa1bb-tmp-dir\") pod \"dns-default-2jhc2\" (UID: \"77b984c3-52a4-442d-9356-f05a01aaa1bb\") " pod="openshift-dns/dns-default-2jhc2" Sep 08 03:56:14.515129 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.515113 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/77b984c3-52a4-442d-9356-f05a01aaa1bb-config-volume\") pod \"dns-default-2jhc2\" (UID: \"77b984c3-52a4-442d-9356-f05a01aaa1bb\") " pod="openshift-dns/dns-default-2jhc2" Sep 08 03:56:14.518851 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.518723 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d27b634-b977-4b2b-b5ad-0766b2b0e1e2-cert\") pod \"ingress-canary-fdn8c\" (UID: \"9d27b634-b977-4b2b-b5ad-0766b2b0e1e2\") " pod="openshift-ingress-canary/ingress-canary-fdn8c" Sep 08 03:56:14.525292 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.525257 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvgjj\" (UniqueName: \"kubernetes.io/projected/77b984c3-52a4-442d-9356-f05a01aaa1bb-kube-api-access-jvgjj\") pod \"dns-default-2jhc2\" (UID: \"77b984c3-52a4-442d-9356-f05a01aaa1bb\") " pod="openshift-dns/dns-default-2jhc2" Sep 08 03:56:14.525406 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.525387 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdkk4\" (UniqueName: \"kubernetes.io/projected/9d27b634-b977-4b2b-b5ad-0766b2b0e1e2-kube-api-access-pdkk4\") pod \"ingress-canary-fdn8c\" (UID: \"9d27b634-b977-4b2b-b5ad-0766b2b0e1e2\") " pod="openshift-ingress-canary/ingress-canary-fdn8c" Sep 08 03:56:14.580576 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.580538 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fdn8c" Sep 08 03:56:14.870342 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.870303 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:56:14.870804 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.870501 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:56:14.877283 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.877255 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 08 03:56:14.877919 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.877902 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tzmtk\"" Sep 08 03:56:14.878025 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.877930 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 08 03:56:14.878025 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.877966 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 08 03:56:14.878117 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:14.877904 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gr9j5\"" Sep 08 03:56:15.018237 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:15.018199 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls\") pod \"dns-default-2jhc2\" (UID: \"77b984c3-52a4-442d-9356-f05a01aaa1bb\") " pod="openshift-dns/dns-default-2jhc2" Sep 08 03:56:15.018424 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:15.018343 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 03:56:15.018424 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:15.018413 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls podName:77b984c3-52a4-442d-9356-f05a01aaa1bb nodeName:}" failed. No retries permitted until 2026-09-08 03:56:16.018393648 +0000 UTC m=+35.639429593 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls") pod "dns-default-2jhc2" (UID: "77b984c3-52a4-442d-9356-f05a01aaa1bb") : secret "dns-default-metrics-tls" not found Sep 08 03:56:15.869719 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:15.869679 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:56:15.878793 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:15.878769 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 08 03:56:16.026491 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:16.026439 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls\") pod \"dns-default-2jhc2\" (UID: \"77b984c3-52a4-442d-9356-f05a01aaa1bb\") " pod="openshift-dns/dns-default-2jhc2" Sep 08 03:56:16.026667 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:16.026561 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 03:56:16.026667 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:16.026624 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls podName:77b984c3-52a4-442d-9356-f05a01aaa1bb nodeName:}" failed. No retries permitted until 2026-09-08 03:56:18.026607642 +0000 UTC m=+37.647643591 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls") pod "dns-default-2jhc2" (UID: "77b984c3-52a4-442d-9356-f05a01aaa1bb") : secret "dns-default-metrics-tls" not found Sep 08 03:56:16.380472 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:16.380427 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fdn8c"] Sep 08 03:56:16.384016 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:56:16.383979 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d27b634_b977_4b2b_b5ad_0766b2b0e1e2.slice/crio-6d1603f2a1802e5a7537f56b1909aeb164cd68248ed5e486fa8d9282e1dbf54e WatchSource:0}: Error finding container 6d1603f2a1802e5a7537f56b1909aeb164cd68248ed5e486fa8d9282e1dbf54e: Status 404 returned error can't find the container with id 6d1603f2a1802e5a7537f56b1909aeb164cd68248ed5e486fa8d9282e1dbf54e Sep 08 03:56:17.245814 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:17.245773 2579 generic.go:356] "Generic (PLEG): container finished" podID="9a603e06-e5e6-4bc5-b352-42bb6dcdbb85" containerID="d5aeac335152c7ee4befb62a4c5ab55043ee598bae3102c15c83cc4a776bd98f" exitCode=0 Sep 08 03:56:17.246740 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:17.245854 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nft7t" event={"ID":"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85","Type":"ContainerDied","Data":"d5aeac335152c7ee4befb62a4c5ab55043ee598bae3102c15c83cc4a776bd98f"} Sep 08 03:56:17.247190 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:17.247145 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fdn8c" event={"ID":"9d27b634-b977-4b2b-b5ad-0766b2b0e1e2","Type":"ContainerStarted","Data":"6d1603f2a1802e5a7537f56b1909aeb164cd68248ed5e486fa8d9282e1dbf54e"} Sep 08 03:56:18.040868 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:18.040654 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls\") pod \"dns-default-2jhc2\" (UID: \"77b984c3-52a4-442d-9356-f05a01aaa1bb\") " pod="openshift-dns/dns-default-2jhc2" Sep 08 03:56:18.041043 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:18.040841 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 03:56:18.041043 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:18.040948 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls podName:77b984c3-52a4-442d-9356-f05a01aaa1bb nodeName:}" failed. No retries permitted until 2026-09-08 03:56:22.040930066 +0000 UTC m=+41.661966010 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls") pod "dns-default-2jhc2" (UID: "77b984c3-52a4-442d-9356-f05a01aaa1bb") : secret "dns-default-metrics-tls" not found Sep 08 03:56:18.254230 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:18.254111 2579 generic.go:356] "Generic (PLEG): container finished" podID="9a603e06-e5e6-4bc5-b352-42bb6dcdbb85" containerID="2a70821d0ebea63cf3872730caa156b09c5162d50d738ab1ec045482c0ee925f" exitCode=0 Sep 08 03:56:18.254230 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:18.254170 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nft7t" event={"ID":"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85","Type":"ContainerDied","Data":"2a70821d0ebea63cf3872730caa156b09c5162d50d738ab1ec045482c0ee925f"} Sep 08 03:56:19.257817 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:19.257772 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fdn8c" event={"ID":"9d27b634-b977-4b2b-b5ad-0766b2b0e1e2","Type":"ContainerStarted","Data":"da09f5673c9423b34a23f5ae18a10c43b0b4e79a4ee1b2aef99f7d418613aeaf"} Sep 08 03:56:19.261179 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:19.261153 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nft7t" event={"ID":"9a603e06-e5e6-4bc5-b352-42bb6dcdbb85","Type":"ContainerStarted","Data":"52dff6a87d60b4284bbe890aeaca1157aec013f98cb833c1b372496c247c4a93"} Sep 08 03:56:19.316840 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:19.316785 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nft7t" podStartSLOduration=9.242098058 podStartE2EDuration="39.316768719s" totalImagesPullingTime="30.074670661s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 03:55:40 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 03:56:19.31513255 +0000 UTC m=+38.936168516" watchObservedRunningTime="2026-09-08 03:56:19.316768719 +0000 UTC m=+38.937804687" Sep 08 03:56:19.317080 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:19.317057 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fdn8c" podStartSLOduration=3.196908082 podStartE2EDuration="5.317049911s" totalImagesPullingTime="2.120141829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 03:56:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 03:56:19.28800583 +0000 UTC m=+38.909041796" watchObservedRunningTime="2026-09-08 03:56:19.317049911 +0000 UTC m=+38.938085877" Sep 08 03:56:22.069309 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:22.069266 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls\") pod \"dns-default-2jhc2\" (UID: \"77b984c3-52a4-442d-9356-f05a01aaa1bb\") " pod="openshift-dns/dns-default-2jhc2" Sep 08 03:56:22.069716 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:22.069428 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 03:56:22.069716 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:22.069514 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls podName:77b984c3-52a4-442d-9356-f05a01aaa1bb nodeName:}" failed. No retries permitted until 2026-09-08 03:56:30.069495298 +0000 UTC m=+49.690531263 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls") pod "dns-default-2jhc2" (UID: "77b984c3-52a4-442d-9356-f05a01aaa1bb") : secret "dns-default-metrics-tls" not found Sep 08 03:56:30.127588 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:30.127544 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls\") pod \"dns-default-2jhc2\" (UID: \"77b984c3-52a4-442d-9356-f05a01aaa1bb\") " pod="openshift-dns/dns-default-2jhc2" Sep 08 03:56:30.128015 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:30.127703 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 03:56:30.128015 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:30.127767 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls podName:77b984c3-52a4-442d-9356-f05a01aaa1bb nodeName:}" failed. No retries permitted until 2026-09-08 03:56:46.127752688 +0000 UTC m=+65.748788633 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls") pod "dns-default-2jhc2" (UID: "77b984c3-52a4-442d-9356-f05a01aaa1bb") : secret "dns-default-metrics-tls" not found Sep 08 03:56:41.236570 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:41.236539 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tg7hb" Sep 08 03:56:45.630664 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:45.630614 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26877828-beec-4ff6-a322-79cdf54e0039-original-pull-secret\") pod \"global-pull-secret-syncer-hrmrn\" (UID: \"26877828-beec-4ff6-a322-79cdf54e0039\") " pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:56:45.630664 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:45.630669 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs\") pod \"network-metrics-daemon-r4xfl\" (UID: \"b7239679-8052-415f-ae9a-1b20d61d1223\") " pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:56:45.635732 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:45.635707 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 08 03:56:45.635966 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:45.635946 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 08 03:56:45.641101 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:45.641082 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 08 03:56:45.641196 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:45.641182 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs podName:b7239679-8052-415f-ae9a-1b20d61d1223 nodeName:}" failed. No retries permitted until 2026-09-08 03:57:49.64115798 +0000 UTC m=+129.262193928 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs") pod "network-metrics-daemon-r4xfl" (UID: "b7239679-8052-415f-ae9a-1b20d61d1223") : secret "metrics-daemon-secret" not found Sep 08 03:56:45.643670 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:45.643647 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/26877828-beec-4ff6-a322-79cdf54e0039-original-pull-secret\") pod \"global-pull-secret-syncer-hrmrn\" (UID: \"26877828-beec-4ff6-a322-79cdf54e0039\") " pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:56:45.731519 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:45.731478 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjmgb\" (UniqueName: \"kubernetes.io/projected/c9c565ac-fd53-499d-b60a-e0af77499433-kube-api-access-wjmgb\") pod \"network-check-target-xj4pw\" (UID: \"c9c565ac-fd53-499d-b60a-e0af77499433\") " pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:56:45.735076 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:45.735057 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 08 03:56:45.744913 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:45.744887 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 08 03:56:45.755235 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:45.755209 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjmgb\" (UniqueName: \"kubernetes.io/projected/c9c565ac-fd53-499d-b60a-e0af77499433-kube-api-access-wjmgb\") pod \"network-check-target-xj4pw\" (UID: \"c9c565ac-fd53-499d-b60a-e0af77499433\") " pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:56:45.792824 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:45.792792 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gr9j5\"" Sep 08 03:56:45.799201 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:45.799177 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:56:45.878552 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:45.878515 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrmrn" Sep 08 03:56:46.075038 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:46.074995 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xj4pw"] Sep 08 03:56:46.078528 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:56:46.078502 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9c565ac_fd53_499d_b60a_e0af77499433.slice/crio-cb5994d4a1dd4c3c363e543ab37212160801f7205671a8ca8f81c8a53f22b35a WatchSource:0}: Error finding container cb5994d4a1dd4c3c363e543ab37212160801f7205671a8ca8f81c8a53f22b35a: Status 404 returned error can't find the container with id cb5994d4a1dd4c3c363e543ab37212160801f7205671a8ca8f81c8a53f22b35a Sep 08 03:56:46.134950 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:46.134864 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls\") pod \"dns-default-2jhc2\" (UID: \"77b984c3-52a4-442d-9356-f05a01aaa1bb\") " pod="openshift-dns/dns-default-2jhc2" Sep 08 03:56:46.135072 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:46.135017 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 03:56:46.135110 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:56:46.135087 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls podName:77b984c3-52a4-442d-9356-f05a01aaa1bb nodeName:}" failed. No retries permitted until 2026-09-08 03:57:18.135068815 +0000 UTC m=+97.756104761 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls") pod "dns-default-2jhc2" (UID: "77b984c3-52a4-442d-9356-f05a01aaa1bb") : secret "dns-default-metrics-tls" not found Sep 08 03:56:46.148772 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:46.148741 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hrmrn"] Sep 08 03:56:46.151925 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:56:46.151899 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26877828_beec_4ff6_a322_79cdf54e0039.slice/crio-4b65a9036bb8749cbbb756021d984849762fb166a01032177785227e0d5b445a WatchSource:0}: Error finding container 4b65a9036bb8749cbbb756021d984849762fb166a01032177785227e0d5b445a: Status 404 returned error can't find the container with id 4b65a9036bb8749cbbb756021d984849762fb166a01032177785227e0d5b445a Sep 08 03:56:46.324144 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:46.324098 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hrmrn" event={"ID":"26877828-beec-4ff6-a322-79cdf54e0039","Type":"ContainerStarted","Data":"4b65a9036bb8749cbbb756021d984849762fb166a01032177785227e0d5b445a"} Sep 08 03:56:46.325171 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:46.325149 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xj4pw" event={"ID":"c9c565ac-fd53-499d-b60a-e0af77499433","Type":"ContainerStarted","Data":"cb5994d4a1dd4c3c363e543ab37212160801f7205671a8ca8f81c8a53f22b35a"} Sep 08 03:56:51.343663 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:51.343616 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xj4pw" event={"ID":"c9c565ac-fd53-499d-b60a-e0af77499433","Type":"ContainerStarted","Data":"65969a75ef68eefb94121cc992a6ed0b822110f5464dc4a08aafa800a90c388f"} Sep 08 03:56:51.344129 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:51.343764 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:56:51.344989 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:51.344969 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hrmrn" event={"ID":"26877828-beec-4ff6-a322-79cdf54e0039","Type":"ContainerStarted","Data":"81f977a2b4e9ea79a85542cda36d310c81490e7b0f1eecaab166ed222bd1ad32"} Sep 08 03:56:51.375931 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:51.375884 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xj4pw" podStartSLOduration=65.854393409 podStartE2EDuration="1m10.37587115s" totalImagesPullingTime="4.521477741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 03:55:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 03:56:51.374954763 +0000 UTC m=+70.995990729" watchObservedRunningTime="2026-09-08 03:56:51.37587115 +0000 UTC m=+70.996907111" Sep 08 03:56:51.413947 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:56:51.413896 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hrmrn" podStartSLOduration=65.961214726 podStartE2EDuration="1m10.4138779s" totalImagesPullingTime="4.452663174s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 03:55:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 03:56:51.412530275 +0000 UTC m=+71.033566241" watchObservedRunningTime="2026-09-08 03:56:51.4138779 +0000 UTC m=+71.034913867" Sep 08 03:57:18.156094 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:57:18.156056 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls\") pod \"dns-default-2jhc2\" (UID: \"77b984c3-52a4-442d-9356-f05a01aaa1bb\") " pod="openshift-dns/dns-default-2jhc2" Sep 08 03:57:18.156507 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:57:18.156206 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 03:57:18.156507 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:57:18.156283 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls podName:77b984c3-52a4-442d-9356-f05a01aaa1bb nodeName:}" failed. No retries permitted until 2026-09-08 03:58:22.156264813 +0000 UTC m=+161.777300774 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls") pod "dns-default-2jhc2" (UID: "77b984c3-52a4-442d-9356-f05a01aaa1bb") : secret "dns-default-metrics-tls" not found Sep 08 03:57:22.350012 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:57:22.349971 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xj4pw" Sep 08 03:57:49.665349 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:57:49.665298 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs\") pod \"network-metrics-daemon-r4xfl\" (UID: \"b7239679-8052-415f-ae9a-1b20d61d1223\") " pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:57:49.665870 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:57:49.665489 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 08 03:57:49.665870 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:57:49.665556 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs podName:b7239679-8052-415f-ae9a-1b20d61d1223 nodeName:}" failed. No retries permitted until 2026-09-08 03:59:51.665539635 +0000 UTC m=+251.286575585 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs") pod "network-metrics-daemon-r4xfl" (UID: "b7239679-8052-415f-ae9a-1b20d61d1223") : secret "metrics-daemon-secret" not found Sep 08 03:58:17.266618 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:58:17.266565 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-2jhc2" podUID="77b984c3-52a4-442d-9356-f05a01aaa1bb" Sep 08 03:58:17.550754 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:58:17.550676 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2jhc2" Sep 08 03:58:17.882499 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:58:17.882443 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-r4xfl" podUID="b7239679-8052-415f-ae9a-1b20d61d1223" Sep 08 03:58:22.181565 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:58:22.181508 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls\") pod \"dns-default-2jhc2\" (UID: \"77b984c3-52a4-442d-9356-f05a01aaa1bb\") " pod="openshift-dns/dns-default-2jhc2" Sep 08 03:58:22.181991 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:58:22.181669 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 03:58:22.181991 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:58:22.181752 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls podName:77b984c3-52a4-442d-9356-f05a01aaa1bb nodeName:}" failed. No retries permitted until 2026-09-08 04:00:24.181735511 +0000 UTC m=+283.802771456 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls") pod "dns-default-2jhc2" (UID: "77b984c3-52a4-442d-9356-f05a01aaa1bb") : secret "dns-default-metrics-tls" not found Sep 08 03:58:30.870868 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:58:30.870825 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:59:38.798173 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.798137 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-79hzk"] Sep 08 03:59:38.799802 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.799786 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-79hzk" Sep 08 03:59:38.807265 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.807246 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 08 03:59:38.807519 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.807503 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-kwz6b\"" Sep 08 03:59:38.808431 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.808418 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 08 03:59:38.808618 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.808602 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 08 03:59:38.808967 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.808951 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 08 03:59:38.834706 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.834681 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-79hzk"] Sep 08 03:59:38.890050 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.890024 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3cd2c98f-1800-40fa-a229-f835cae8915a-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-79hzk\" (UID: \"3cd2c98f-1800-40fa-a229-f835cae8915a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-79hzk" Sep 08 03:59:38.890166 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.890066 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3cd2c98f-1800-40fa-a229-f835cae8915a-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-79hzk\" (UID: \"3cd2c98f-1800-40fa-a229-f835cae8915a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-79hzk" Sep 08 03:59:38.890279 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.890201 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrjb4\" (UniqueName: \"kubernetes.io/projected/3cd2c98f-1800-40fa-a229-f835cae8915a-kube-api-access-mrjb4\") pod \"kube-storage-version-migrator-operator-8d7dd4675-79hzk\" (UID: \"3cd2c98f-1800-40fa-a229-f835cae8915a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-79hzk" Sep 08 03:59:38.894014 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.893994 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-7qzwn"] Sep 08 03:59:38.895689 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.895665 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-7qzwn" Sep 08 03:59:38.902376 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:38.902346 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"volume-data-source-validator-695c895bd6-7qzwn\" is forbidden: User \"system:node:ip-10-0-128-141.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-cluster-storage-operator\": no relationship found between node 'ip-10-0-128-141.ec2.internal' and this object" podUID="5ceb21d6-1277-4ff4-a22d-8b4f4b218e45" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-7qzwn" Sep 08 03:59:38.902491 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:38.902371 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"volume-data-source-validator-dockercfg-zptgg\" is forbidden: User \"system:node:ip-10-0-128-141.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-cluster-storage-operator\": no relationship found between node 'ip-10-0-128-141.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-zptgg\"" type="*v1.Secret" Sep 08 03:59:38.902491 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:38.902371 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-128-141.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-cluster-storage-operator\": no relationship found between node 'ip-10-0-128-141.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Sep 08 03:59:38.902491 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:38.902434 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-128-141.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-cluster-storage-operator\": no relationship found between node 'ip-10-0-128-141.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Sep 08 03:59:38.914698 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.914677 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-69bf99dbfb-78zrw"] Sep 08 03:59:38.916326 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.916313 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 03:59:38.920209 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:38.920187 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-128-141.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-128-141.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Sep 08 03:59:38.924303 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:38.924284 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-128-141.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-128-141.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Sep 08 03:59:38.924791 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:38.924767 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"router-metrics-certs-default\" is forbidden: User \"system:node:ip-10-0-128-141.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-128-141.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" type="*v1.Secret" Sep 08 03:59:38.924896 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:38.924869 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-ingress-cert\" is forbidden: User \"system:node:ip-10-0-128-141.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-128-141.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" type="*v1.Secret" Sep 08 03:59:38.924952 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:38.924907 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"service-ca-bundle\" is forbidden: User \"system:node:ip-10-0-128-141.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-128-141.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" type="*v1.ConfigMap" Sep 08 03:59:38.924952 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:38.924906 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"router-stats-default\" is forbidden: User \"system:node:ip-10-0-128-141.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-128-141.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" type="*v1.Secret" Sep 08 03:59:38.924952 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:38.924907 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"volume-data-source-validator-695c895bd6-7qzwn\" is forbidden: User \"system:node:ip-10-0-128-141.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-cluster-storage-operator\": no relationship found between node 'ip-10-0-128-141.ec2.internal' and this object" podUID="5ceb21d6-1277-4ff4-a22d-8b4f4b218e45" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-7qzwn" Sep 08 03:59:38.925055 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:38.925007 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"router-dockercfg-qg2m5\" is forbidden: User \"system:node:ip-10-0-128-141.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-128-141.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qg2m5\"" type="*v1.Secret" Sep 08 03:59:38.934688 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.934669 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-7qzwn"] Sep 08 03:59:38.951588 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.951567 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-69bf99dbfb-78zrw"] Sep 08 03:59:38.991163 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.991132 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3cd2c98f-1800-40fa-a229-f835cae8915a-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-79hzk\" (UID: \"3cd2c98f-1800-40fa-a229-f835cae8915a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-79hzk" Sep 08 03:59:38.991256 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.991172 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/179fd61c-eeab-4eef-b59c-c75528b5c6ee-service-ca-bundle\") pod \"router-default-69bf99dbfb-78zrw\" (UID: \"179fd61c-eeab-4eef-b59c-c75528b5c6ee\") " pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 03:59:38.991256 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.991195 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-786sx\" (UniqueName: \"kubernetes.io/projected/179fd61c-eeab-4eef-b59c-c75528b5c6ee-kube-api-access-786sx\") pod \"router-default-69bf99dbfb-78zrw\" (UID: \"179fd61c-eeab-4eef-b59c-c75528b5c6ee\") " pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 03:59:38.991346 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.991266 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/179fd61c-eeab-4eef-b59c-c75528b5c6ee-default-certificate\") pod \"router-default-69bf99dbfb-78zrw\" (UID: \"179fd61c-eeab-4eef-b59c-c75528b5c6ee\") " pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 03:59:38.991392 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.991362 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxsrm\" (UniqueName: \"kubernetes.io/projected/5ceb21d6-1277-4ff4-a22d-8b4f4b218e45-kube-api-access-fxsrm\") pod \"volume-data-source-validator-695c895bd6-7qzwn\" (UID: \"5ceb21d6-1277-4ff4-a22d-8b4f4b218e45\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-7qzwn" Sep 08 03:59:38.991445 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.991425 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrjb4\" (UniqueName: \"kubernetes.io/projected/3cd2c98f-1800-40fa-a229-f835cae8915a-kube-api-access-mrjb4\") pod \"kube-storage-version-migrator-operator-8d7dd4675-79hzk\" (UID: \"3cd2c98f-1800-40fa-a229-f835cae8915a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-79hzk" Sep 08 03:59:38.991524 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.991477 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/179fd61c-eeab-4eef-b59c-c75528b5c6ee-stats-auth\") pod \"router-default-69bf99dbfb-78zrw\" (UID: \"179fd61c-eeab-4eef-b59c-c75528b5c6ee\") " pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 03:59:38.991524 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.991505 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/179fd61c-eeab-4eef-b59c-c75528b5c6ee-metrics-certs\") pod \"router-default-69bf99dbfb-78zrw\" (UID: \"179fd61c-eeab-4eef-b59c-c75528b5c6ee\") " pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 03:59:38.991634 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.991617 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3cd2c98f-1800-40fa-a229-f835cae8915a-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-79hzk\" (UID: \"3cd2c98f-1800-40fa-a229-f835cae8915a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-79hzk" Sep 08 03:59:38.992041 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.992026 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3cd2c98f-1800-40fa-a229-f835cae8915a-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-79hzk\" (UID: \"3cd2c98f-1800-40fa-a229-f835cae8915a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-79hzk" Sep 08 03:59:38.993321 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:38.993300 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3cd2c98f-1800-40fa-a229-f835cae8915a-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-79hzk\" (UID: \"3cd2c98f-1800-40fa-a229-f835cae8915a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-79hzk" Sep 08 03:59:39.045760 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.045733 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-t55cp"] Sep 08 03:59:39.047602 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.047587 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-t55cp" Sep 08 03:59:39.061586 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:39.061559 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"openshift-insights-serving-cert\" is forbidden: User \"system:node:ip-10-0-128-141.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-128-141.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" type="*v1.Secret" Sep 08 03:59:39.061671 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:39.061583 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-128-141.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-128-141.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Sep 08 03:59:39.071195 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.071166 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dxhpn"] Sep 08 03:59:39.073048 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.073031 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dxhpn" Sep 08 03:59:39.075753 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:39.075729 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"service-ca-bundle\" is forbidden: User \"system:node:ip-10-0-128-141.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-128-141.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" type="*v1.ConfigMap" Sep 08 03:59:39.076130 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:39.076112 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"operator-dockercfg-kjds9\" is forbidden: User \"system:node:ip-10-0-128-141.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-128-141.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"operator-dockercfg-kjds9\"" type="*v1.Secret" Sep 08 03:59:39.078083 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:39.078058 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-128-141.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-128-141.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Sep 08 03:59:39.078178 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:39.078113 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"trusted-ca-bundle\" is forbidden: User \"system:node:ip-10-0-128-141.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-128-141.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" type="*v1.ConfigMap" Sep 08 03:59:39.087814 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.087795 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 08 03:59:39.092060 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.092044 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/179fd61c-eeab-4eef-b59c-c75528b5c6ee-service-ca-bundle\") pod \"router-default-69bf99dbfb-78zrw\" (UID: \"179fd61c-eeab-4eef-b59c-c75528b5c6ee\") " pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 03:59:39.092116 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.092070 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ab64adad-879e-4ec6-8425-86a70f48615f-snapshots\") pod \"insights-operator-76bcd6ccf5-t55cp\" (UID: \"ab64adad-879e-4ec6-8425-86a70f48615f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-t55cp" Sep 08 03:59:39.092116 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.092087 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab64adad-879e-4ec6-8425-86a70f48615f-tmp\") pod \"insights-operator-76bcd6ccf5-t55cp\" (UID: \"ab64adad-879e-4ec6-8425-86a70f48615f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-t55cp" Sep 08 03:59:39.092116 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.092106 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-786sx\" (UniqueName: \"kubernetes.io/projected/179fd61c-eeab-4eef-b59c-c75528b5c6ee-kube-api-access-786sx\") pod \"router-default-69bf99dbfb-78zrw\" (UID: \"179fd61c-eeab-4eef-b59c-c75528b5c6ee\") " pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 03:59:39.092248 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.092164 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ab64adad-879e-4ec6-8425-86a70f48615f-serving-cert\") pod \"insights-operator-76bcd6ccf5-t55cp\" (UID: \"ab64adad-879e-4ec6-8425-86a70f48615f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-t55cp" Sep 08 03:59:39.092248 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.092191 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2dwk\" (UniqueName: \"kubernetes.io/projected/ab64adad-879e-4ec6-8425-86a70f48615f-kube-api-access-j2dwk\") pod \"insights-operator-76bcd6ccf5-t55cp\" (UID: \"ab64adad-879e-4ec6-8425-86a70f48615f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-t55cp" Sep 08 03:59:39.092316 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.092265 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab64adad-879e-4ec6-8425-86a70f48615f-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-t55cp\" (UID: \"ab64adad-879e-4ec6-8425-86a70f48615f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-t55cp" Sep 08 03:59:39.092316 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.092293 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/179fd61c-eeab-4eef-b59c-c75528b5c6ee-default-certificate\") pod \"router-default-69bf99dbfb-78zrw\" (UID: \"179fd61c-eeab-4eef-b59c-c75528b5c6ee\") " pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 03:59:39.092376 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.092319 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxsrm\" (UniqueName: \"kubernetes.io/projected/5ceb21d6-1277-4ff4-a22d-8b4f4b218e45-kube-api-access-fxsrm\") pod \"volume-data-source-validator-695c895bd6-7qzwn\" (UID: \"5ceb21d6-1277-4ff4-a22d-8b4f4b218e45\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-7qzwn" Sep 08 03:59:39.092376 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.092342 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/179fd61c-eeab-4eef-b59c-c75528b5c6ee-stats-auth\") pod \"router-default-69bf99dbfb-78zrw\" (UID: \"179fd61c-eeab-4eef-b59c-c75528b5c6ee\") " pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 03:59:39.092376 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.092356 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/179fd61c-eeab-4eef-b59c-c75528b5c6ee-metrics-certs\") pod \"router-default-69bf99dbfb-78zrw\" (UID: \"179fd61c-eeab-4eef-b59c-c75528b5c6ee\") " pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 03:59:39.092480 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.092383 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab64adad-879e-4ec6-8425-86a70f48615f-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-t55cp\" (UID: \"ab64adad-879e-4ec6-8425-86a70f48615f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-t55cp" Sep 08 03:59:39.106931 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.106908 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 08 03:59:39.108784 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.108765 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrjb4\" (UniqueName: \"kubernetes.io/projected/3cd2c98f-1800-40fa-a229-f835cae8915a-kube-api-access-mrjb4\") pod \"kube-storage-version-migrator-operator-8d7dd4675-79hzk\" (UID: \"3cd2c98f-1800-40fa-a229-f835cae8915a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-79hzk" Sep 08 03:59:39.131419 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.131390 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 08 03:59:39.131540 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.131437 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 08 03:59:39.131540 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.131478 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-rgjvv\"" Sep 08 03:59:39.188939 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.188918 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dxhpn"] Sep 08 03:59:39.191565 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.191536 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-t55cp"] Sep 08 03:59:39.192914 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.192894 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ab64adad-879e-4ec6-8425-86a70f48615f-serving-cert\") pod \"insights-operator-76bcd6ccf5-t55cp\" (UID: \"ab64adad-879e-4ec6-8425-86a70f48615f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-t55cp" Sep 08 03:59:39.193004 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.192919 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2dwk\" (UniqueName: \"kubernetes.io/projected/ab64adad-879e-4ec6-8425-86a70f48615f-kube-api-access-j2dwk\") pod \"insights-operator-76bcd6ccf5-t55cp\" (UID: \"ab64adad-879e-4ec6-8425-86a70f48615f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-t55cp" Sep 08 03:59:39.193004 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.192941 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3027dc6f-51e4-45de-adfc-3f426e71e9b9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-dxhpn\" (UID: \"3027dc6f-51e4-45de-adfc-3f426e71e9b9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dxhpn" Sep 08 03:59:39.193004 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.192964 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3027dc6f-51e4-45de-adfc-3f426e71e9b9-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-dxhpn\" (UID: \"3027dc6f-51e4-45de-adfc-3f426e71e9b9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dxhpn" Sep 08 03:59:39.193004 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.192989 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab64adad-879e-4ec6-8425-86a70f48615f-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-t55cp\" (UID: \"ab64adad-879e-4ec6-8425-86a70f48615f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-t55cp" Sep 08 03:59:39.193213 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.193015 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab64adad-879e-4ec6-8425-86a70f48615f-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-t55cp\" (UID: \"ab64adad-879e-4ec6-8425-86a70f48615f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-t55cp" Sep 08 03:59:39.193213 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.193060 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ab64adad-879e-4ec6-8425-86a70f48615f-snapshots\") pod \"insights-operator-76bcd6ccf5-t55cp\" (UID: \"ab64adad-879e-4ec6-8425-86a70f48615f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-t55cp" Sep 08 03:59:39.193213 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.193077 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mctnt\" (UniqueName: \"kubernetes.io/projected/3027dc6f-51e4-45de-adfc-3f426e71e9b9-kube-api-access-mctnt\") pod \"cluster-monitoring-operator-6cdcc96965-dxhpn\" (UID: \"3027dc6f-51e4-45de-adfc-3f426e71e9b9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dxhpn" Sep 08 03:59:39.193213 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.193130 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab64adad-879e-4ec6-8425-86a70f48615f-tmp\") pod \"insights-operator-76bcd6ccf5-t55cp\" (UID: \"ab64adad-879e-4ec6-8425-86a70f48615f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-t55cp" Sep 08 03:59:39.193526 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.193510 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab64adad-879e-4ec6-8425-86a70f48615f-tmp\") pod \"insights-operator-76bcd6ccf5-t55cp\" (UID: \"ab64adad-879e-4ec6-8425-86a70f48615f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-t55cp" Sep 08 03:59:39.193571 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.193560 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ab64adad-879e-4ec6-8425-86a70f48615f-snapshots\") pod \"insights-operator-76bcd6ccf5-t55cp\" (UID: \"ab64adad-879e-4ec6-8425-86a70f48615f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-t55cp" Sep 08 03:59:39.294201 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.294166 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3027dc6f-51e4-45de-adfc-3f426e71e9b9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-dxhpn\" (UID: \"3027dc6f-51e4-45de-adfc-3f426e71e9b9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dxhpn" Sep 08 03:59:39.294201 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.294206 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3027dc6f-51e4-45de-adfc-3f426e71e9b9-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-dxhpn\" (UID: \"3027dc6f-51e4-45de-adfc-3f426e71e9b9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dxhpn" Sep 08 03:59:39.294420 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.294258 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mctnt\" (UniqueName: \"kubernetes.io/projected/3027dc6f-51e4-45de-adfc-3f426e71e9b9-kube-api-access-mctnt\") pod \"cluster-monitoring-operator-6cdcc96965-dxhpn\" (UID: \"3027dc6f-51e4-45de-adfc-3f426e71e9b9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dxhpn" Sep 08 03:59:39.294420 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:39.294317 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 08 03:59:39.294420 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:39.294405 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3027dc6f-51e4-45de-adfc-3f426e71e9b9-cluster-monitoring-operator-tls podName:3027dc6f-51e4-45de-adfc-3f426e71e9b9 nodeName:}" failed. No retries permitted until 2026-09-08 03:59:39.794383451 +0000 UTC m=+239.415419427 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3027dc6f-51e4-45de-adfc-3f426e71e9b9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-dxhpn" (UID: "3027dc6f-51e4-45de-adfc-3f426e71e9b9") : secret "cluster-monitoring-operator-tls" not found Sep 08 03:59:39.294881 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.294862 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3027dc6f-51e4-45de-adfc-3f426e71e9b9-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-dxhpn\" (UID: \"3027dc6f-51e4-45de-adfc-3f426e71e9b9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dxhpn" Sep 08 03:59:39.314379 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.314325 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mctnt\" (UniqueName: \"kubernetes.io/projected/3027dc6f-51e4-45de-adfc-3f426e71e9b9-kube-api-access-mctnt\") pod \"cluster-monitoring-operator-6cdcc96965-dxhpn\" (UID: \"3027dc6f-51e4-45de-adfc-3f426e71e9b9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dxhpn" Sep 08 03:59:39.408548 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.408519 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-79hzk" Sep 08 03:59:39.689947 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.689921 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-79hzk"] Sep 08 03:59:39.692044 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:59:39.692018 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cd2c98f_1800_40fa_a229_f835cae8915a.slice/crio-b24dcfd238b2a0b574c1a5aa9f51c22d4e45a10a94c49f26812f000edb2d3ebc WatchSource:0}: Error finding container b24dcfd238b2a0b574c1a5aa9f51c22d4e45a10a94c49f26812f000edb2d3ebc: Status 404 returned error can't find the container with id b24dcfd238b2a0b574c1a5aa9f51c22d4e45a10a94c49f26812f000edb2d3ebc Sep 08 03:59:39.746899 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.746872 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-79hzk" event={"ID":"3cd2c98f-1800-40fa-a229-f835cae8915a","Type":"ContainerStarted","Data":"b24dcfd238b2a0b574c1a5aa9f51c22d4e45a10a94c49f26812f000edb2d3ebc"} Sep 08 03:59:39.798826 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.798794 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3027dc6f-51e4-45de-adfc-3f426e71e9b9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-dxhpn\" (UID: \"3027dc6f-51e4-45de-adfc-3f426e71e9b9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dxhpn" Sep 08 03:59:39.799122 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:39.798923 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 08 03:59:39.799122 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:39.798971 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3027dc6f-51e4-45de-adfc-3f426e71e9b9-cluster-monitoring-operator-tls podName:3027dc6f-51e4-45de-adfc-3f426e71e9b9 nodeName:}" failed. No retries permitted until 2026-09-08 03:59:40.798957599 +0000 UTC m=+240.419993544 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3027dc6f-51e4-45de-adfc-3f426e71e9b9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-dxhpn" (UID: "3027dc6f-51e4-45de-adfc-3f426e71e9b9") : secret "cluster-monitoring-operator-tls" not found Sep 08 03:59:39.925321 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.925293 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 08 03:59:39.931301 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.931280 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 08 03:59:39.935643 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.935626 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/179fd61c-eeab-4eef-b59c-c75528b5c6ee-stats-auth\") pod \"router-default-69bf99dbfb-78zrw\" (UID: \"179fd61c-eeab-4eef-b59c-c75528b5c6ee\") " pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 03:59:39.966437 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:39.966384 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 08 03:59:39.972552 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:39.972531 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/179fd61c-eeab-4eef-b59c-c75528b5c6ee-service-ca-bundle podName:179fd61c-eeab-4eef-b59c-c75528b5c6ee nodeName:}" failed. No retries permitted until 2026-09-08 03:59:40.472514499 +0000 UTC m=+240.093550444 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/179fd61c-eeab-4eef-b59c-c75528b5c6ee-service-ca-bundle") pod "router-default-69bf99dbfb-78zrw" (UID: "179fd61c-eeab-4eef-b59c-c75528b5c6ee") : configmap references non-existent config key: service-ca.crt Sep 08 03:59:40.011077 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.011057 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 08 03:59:40.015159 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.015140 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ab64adad-879e-4ec6-8425-86a70f48615f-serving-cert\") pod \"insights-operator-76bcd6ccf5-t55cp\" (UID: \"ab64adad-879e-4ec6-8425-86a70f48615f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-t55cp" Sep 08 03:59:40.018161 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.018143 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-zptgg\"" Sep 08 03:59:40.091167 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.091142 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 08 03:59:40.093272 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:40.093257 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: failed to sync secret cache: timed out waiting for the condition Sep 08 03:59:40.093359 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:40.093328 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/179fd61c-eeab-4eef-b59c-c75528b5c6ee-metrics-certs podName:179fd61c-eeab-4eef-b59c-c75528b5c6ee nodeName:}" failed. No retries permitted until 2026-09-08 03:59:40.593313563 +0000 UTC m=+240.214349508 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/179fd61c-eeab-4eef-b59c-c75528b5c6ee-metrics-certs") pod "router-default-69bf99dbfb-78zrw" (UID: "179fd61c-eeab-4eef-b59c-c75528b5c6ee") : failed to sync secret cache: timed out waiting for the condition Sep 08 03:59:40.095283 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.095260 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/179fd61c-eeab-4eef-b59c-c75528b5c6ee-default-certificate\") pod \"router-default-69bf99dbfb-78zrw\" (UID: \"179fd61c-eeab-4eef-b59c-c75528b5c6ee\") " pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 03:59:40.132513 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:40.132490 2579 projected.go:291] Couldn't get configMap openshift-ingress/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition Sep 08 03:59:40.141230 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.141211 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-kjds9\"" Sep 08 03:59:40.149623 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.149603 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qg2m5\"" Sep 08 03:59:40.178154 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.178134 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 08 03:59:40.193570 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:40.193549 2579 configmap.go:193] Couldn't get configMap openshift-insights/trusted-ca-bundle: failed to sync configmap cache: timed out waiting for the condition Sep 08 03:59:40.193570 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:40.193564 2579 configmap.go:193] Couldn't get configMap openshift-insights/service-ca-bundle: failed to sync configmap cache: timed out waiting for the condition Sep 08 03:59:40.193685 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:40.193612 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ab64adad-879e-4ec6-8425-86a70f48615f-service-ca-bundle podName:ab64adad-879e-4ec6-8425-86a70f48615f nodeName:}" failed. No retries permitted until 2026-09-08 03:59:40.693599006 +0000 UTC m=+240.314634956 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ab64adad-879e-4ec6-8425-86a70f48615f-service-ca-bundle") pod "insights-operator-76bcd6ccf5-t55cp" (UID: "ab64adad-879e-4ec6-8425-86a70f48615f") : failed to sync configmap cache: timed out waiting for the condition Sep 08 03:59:40.193685 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:40.193627 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ab64adad-879e-4ec6-8425-86a70f48615f-trusted-ca-bundle podName:ab64adad-879e-4ec6-8425-86a70f48615f nodeName:}" failed. No retries permitted until 2026-09-08 03:59:40.693619827 +0000 UTC m=+240.314655777 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/ab64adad-879e-4ec6-8425-86a70f48615f-trusted-ca-bundle") pod "insights-operator-76bcd6ccf5-t55cp" (UID: "ab64adad-879e-4ec6-8425-86a70f48615f") : failed to sync configmap cache: timed out waiting for the condition Sep 08 03:59:40.240917 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.240861 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 08 03:59:40.242971 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:40.242956 2579 projected.go:196] Error preparing data for projected volume kube-api-access-786sx for pod openshift-ingress/router-default-69bf99dbfb-78zrw: failed to sync configmap cache: timed out waiting for the condition Sep 08 03:59:40.243042 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:40.243002 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/179fd61c-eeab-4eef-b59c-c75528b5c6ee-kube-api-access-786sx podName:179fd61c-eeab-4eef-b59c-c75528b5c6ee nodeName:}" failed. No retries permitted until 2026-09-08 03:59:40.742989919 +0000 UTC m=+240.364025864 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-786sx" (UniqueName: "kubernetes.io/projected/179fd61c-eeab-4eef-b59c-c75528b5c6ee-kube-api-access-786sx") pod "router-default-69bf99dbfb-78zrw" (UID: "179fd61c-eeab-4eef-b59c-c75528b5c6ee") : failed to sync configmap cache: timed out waiting for the condition Sep 08 03:59:40.325703 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.325679 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 08 03:59:40.381724 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.381690 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 08 03:59:40.384422 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.384403 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 08 03:59:40.394324 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.394303 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxsrm\" (UniqueName: \"kubernetes.io/projected/5ceb21d6-1277-4ff4-a22d-8b4f4b218e45-kube-api-access-fxsrm\") pod \"volume-data-source-validator-695c895bd6-7qzwn\" (UID: \"5ceb21d6-1277-4ff4-a22d-8b4f4b218e45\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-7qzwn" Sep 08 03:59:40.403454 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.403434 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-7qzwn" Sep 08 03:59:40.483873 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.483850 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 08 03:59:40.494686 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.494636 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 08 03:59:40.504140 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.504119 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2dwk\" (UniqueName: \"kubernetes.io/projected/ab64adad-879e-4ec6-8425-86a70f48615f-kube-api-access-j2dwk\") pod \"insights-operator-76bcd6ccf5-t55cp\" (UID: \"ab64adad-879e-4ec6-8425-86a70f48615f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-t55cp" Sep 08 03:59:40.505065 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.505049 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/179fd61c-eeab-4eef-b59c-c75528b5c6ee-service-ca-bundle\") pod \"router-default-69bf99dbfb-78zrw\" (UID: \"179fd61c-eeab-4eef-b59c-c75528b5c6ee\") " pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 03:59:40.505184 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:40.505172 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/179fd61c-eeab-4eef-b59c-c75528b5c6ee-service-ca-bundle podName:179fd61c-eeab-4eef-b59c-c75528b5c6ee nodeName:}" failed. No retries permitted until 2026-09-08 03:59:41.505159605 +0000 UTC m=+241.126195550 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/179fd61c-eeab-4eef-b59c-c75528b5c6ee-service-ca-bundle") pod "router-default-69bf99dbfb-78zrw" (UID: "179fd61c-eeab-4eef-b59c-c75528b5c6ee") : configmap references non-existent config key: service-ca.crt Sep 08 03:59:40.511600 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.511584 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 08 03:59:40.606005 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.605972 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/179fd61c-eeab-4eef-b59c-c75528b5c6ee-metrics-certs\") pod \"router-default-69bf99dbfb-78zrw\" (UID: \"179fd61c-eeab-4eef-b59c-c75528b5c6ee\") " pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 03:59:40.606174 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:40.606134 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 08 03:59:40.606237 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:40.606201 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/179fd61c-eeab-4eef-b59c-c75528b5c6ee-metrics-certs podName:179fd61c-eeab-4eef-b59c-c75528b5c6ee nodeName:}" failed. No retries permitted until 2026-09-08 03:59:41.606179558 +0000 UTC m=+241.227215503 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/179fd61c-eeab-4eef-b59c-c75528b5c6ee-metrics-certs") pod "router-default-69bf99dbfb-78zrw" (UID: "179fd61c-eeab-4eef-b59c-c75528b5c6ee") : secret "router-metrics-certs-default" not found Sep 08 03:59:40.673273 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.673240 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-7qzwn"] Sep 08 03:59:40.676882 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:59:40.676856 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ceb21d6_1277_4ff4_a22d_8b4f4b218e45.slice/crio-64ef69f768ce384886f2546cd1e1f9861469a3783ac8972cef6075d1d0a8a4c0 WatchSource:0}: Error finding container 64ef69f768ce384886f2546cd1e1f9861469a3783ac8972cef6075d1d0a8a4c0: Status 404 returned error can't find the container with id 64ef69f768ce384886f2546cd1e1f9861469a3783ac8972cef6075d1d0a8a4c0 Sep 08 03:59:40.706735 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.706706 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab64adad-879e-4ec6-8425-86a70f48615f-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-t55cp\" (UID: \"ab64adad-879e-4ec6-8425-86a70f48615f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-t55cp" Sep 08 03:59:40.706862 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.706837 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab64adad-879e-4ec6-8425-86a70f48615f-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-t55cp\" (UID: \"ab64adad-879e-4ec6-8425-86a70f48615f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-t55cp" Sep 08 03:59:40.707270 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.707238 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab64adad-879e-4ec6-8425-86a70f48615f-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-t55cp\" (UID: \"ab64adad-879e-4ec6-8425-86a70f48615f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-t55cp" Sep 08 03:59:40.707637 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.707615 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab64adad-879e-4ec6-8425-86a70f48615f-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-t55cp\" (UID: \"ab64adad-879e-4ec6-8425-86a70f48615f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-t55cp" Sep 08 03:59:40.750473 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.750386 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-7qzwn" event={"ID":"5ceb21d6-1277-4ff4-a22d-8b4f4b218e45","Type":"ContainerStarted","Data":"64ef69f768ce384886f2546cd1e1f9861469a3783ac8972cef6075d1d0a8a4c0"} Sep 08 03:59:40.808104 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.808076 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-786sx\" (UniqueName: \"kubernetes.io/projected/179fd61c-eeab-4eef-b59c-c75528b5c6ee-kube-api-access-786sx\") pod \"router-default-69bf99dbfb-78zrw\" (UID: \"179fd61c-eeab-4eef-b59c-c75528b5c6ee\") " pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 03:59:40.808503 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.808111 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3027dc6f-51e4-45de-adfc-3f426e71e9b9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-dxhpn\" (UID: \"3027dc6f-51e4-45de-adfc-3f426e71e9b9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dxhpn" Sep 08 03:59:40.808503 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:40.808236 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 08 03:59:40.808503 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:40.808300 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3027dc6f-51e4-45de-adfc-3f426e71e9b9-cluster-monitoring-operator-tls podName:3027dc6f-51e4-45de-adfc-3f426e71e9b9 nodeName:}" failed. No retries permitted until 2026-09-08 03:59:42.808282885 +0000 UTC m=+242.429318833 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3027dc6f-51e4-45de-adfc-3f426e71e9b9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-dxhpn" (UID: "3027dc6f-51e4-45de-adfc-3f426e71e9b9") : secret "cluster-monitoring-operator-tls" not found Sep 08 03:59:40.811044 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.811015 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-786sx\" (UniqueName: \"kubernetes.io/projected/179fd61c-eeab-4eef-b59c-c75528b5c6ee-kube-api-access-786sx\") pod \"router-default-69bf99dbfb-78zrw\" (UID: \"179fd61c-eeab-4eef-b59c-c75528b5c6ee\") " pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 03:59:40.859700 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.859673 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-kjds9\"" Sep 08 03:59:40.867152 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:40.867131 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-t55cp" Sep 08 03:59:41.144357 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:41.144263 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-t55cp"] Sep 08 03:59:41.146564 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:59:41.146536 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab64adad_879e_4ec6_8425_86a70f48615f.slice/crio-0015b0e9aa969de74d4cdb654c72509558cd5bbc5febb63fd8d99904030a7232 WatchSource:0}: Error finding container 0015b0e9aa969de74d4cdb654c72509558cd5bbc5febb63fd8d99904030a7232: Status 404 returned error can't find the container with id 0015b0e9aa969de74d4cdb654c72509558cd5bbc5febb63fd8d99904030a7232 Sep 08 03:59:41.515313 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:41.515221 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/179fd61c-eeab-4eef-b59c-c75528b5c6ee-service-ca-bundle\") pod \"router-default-69bf99dbfb-78zrw\" (UID: \"179fd61c-eeab-4eef-b59c-c75528b5c6ee\") " pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 03:59:41.515493 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:41.515432 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/179fd61c-eeab-4eef-b59c-c75528b5c6ee-service-ca-bundle podName:179fd61c-eeab-4eef-b59c-c75528b5c6ee nodeName:}" failed. No retries permitted until 2026-09-08 03:59:43.515413679 +0000 UTC m=+243.136449624 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/179fd61c-eeab-4eef-b59c-c75528b5c6ee-service-ca-bundle") pod "router-default-69bf99dbfb-78zrw" (UID: "179fd61c-eeab-4eef-b59c-c75528b5c6ee") : configmap references non-existent config key: service-ca.crt Sep 08 03:59:41.615920 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:41.615878 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/179fd61c-eeab-4eef-b59c-c75528b5c6ee-metrics-certs\") pod \"router-default-69bf99dbfb-78zrw\" (UID: \"179fd61c-eeab-4eef-b59c-c75528b5c6ee\") " pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 03:59:41.616090 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:41.616063 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 08 03:59:41.616165 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:41.616152 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/179fd61c-eeab-4eef-b59c-c75528b5c6ee-metrics-certs podName:179fd61c-eeab-4eef-b59c-c75528b5c6ee nodeName:}" failed. No retries permitted until 2026-09-08 03:59:43.616131897 +0000 UTC m=+243.237167865 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/179fd61c-eeab-4eef-b59c-c75528b5c6ee-metrics-certs") pod "router-default-69bf99dbfb-78zrw" (UID: "179fd61c-eeab-4eef-b59c-c75528b5c6ee") : secret "router-metrics-certs-default" not found Sep 08 03:59:41.754686 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:41.754644 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-t55cp" event={"ID":"ab64adad-879e-4ec6-8425-86a70f48615f","Type":"ContainerStarted","Data":"0015b0e9aa969de74d4cdb654c72509558cd5bbc5febb63fd8d99904030a7232"} Sep 08 03:59:42.759145 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:42.759106 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-79hzk" event={"ID":"3cd2c98f-1800-40fa-a229-f835cae8915a","Type":"ContainerStarted","Data":"8aab9d1748ea91e2cc168cb1b17b6becb0c286a4763acdc48543001733cf3d26"} Sep 08 03:59:42.760573 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:42.760550 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-7qzwn" event={"ID":"5ceb21d6-1277-4ff4-a22d-8b4f4b218e45","Type":"ContainerStarted","Data":"1084799f6f35c0255d766024fa988440a5217586b6db95623f6540df239a77dc"} Sep 08 03:59:42.777929 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:42.777883 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-79hzk" podStartSLOduration=2.439677758 podStartE2EDuration="4.77786883s" totalImagesPullingTime="2.338191072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 03:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 03:59:42.777176529 +0000 UTC m=+242.398212497" watchObservedRunningTime="2026-09-08 03:59:42.77786883 +0000 UTC m=+242.398904797" Sep 08 03:59:42.794652 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:42.794607 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-7qzwn" podStartSLOduration=3.444567719 podStartE2EDuration="4.794591708s" totalImagesPullingTime="1.350023989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 03:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 03:59:42.793716026 +0000 UTC m=+242.414751994" watchObservedRunningTime="2026-09-08 03:59:42.794591708 +0000 UTC m=+242.415627674" Sep 08 03:59:42.825062 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:42.824994 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3027dc6f-51e4-45de-adfc-3f426e71e9b9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-dxhpn\" (UID: \"3027dc6f-51e4-45de-adfc-3f426e71e9b9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dxhpn" Sep 08 03:59:42.825201 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:42.825109 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 08 03:59:42.825201 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:42.825178 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3027dc6f-51e4-45de-adfc-3f426e71e9b9-cluster-monitoring-operator-tls podName:3027dc6f-51e4-45de-adfc-3f426e71e9b9 nodeName:}" failed. No retries permitted until 2026-09-08 03:59:46.82516024 +0000 UTC m=+246.446196200 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3027dc6f-51e4-45de-adfc-3f426e71e9b9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-dxhpn" (UID: "3027dc6f-51e4-45de-adfc-3f426e71e9b9") : secret "cluster-monitoring-operator-tls" not found Sep 08 03:59:43.530438 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:43.530412 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/179fd61c-eeab-4eef-b59c-c75528b5c6ee-service-ca-bundle\") pod \"router-default-69bf99dbfb-78zrw\" (UID: \"179fd61c-eeab-4eef-b59c-c75528b5c6ee\") " pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 03:59:43.530618 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:43.530603 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/179fd61c-eeab-4eef-b59c-c75528b5c6ee-service-ca-bundle podName:179fd61c-eeab-4eef-b59c-c75528b5c6ee nodeName:}" failed. No retries permitted until 2026-09-08 03:59:47.530584177 +0000 UTC m=+247.151620140 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/179fd61c-eeab-4eef-b59c-c75528b5c6ee-service-ca-bundle") pod "router-default-69bf99dbfb-78zrw" (UID: "179fd61c-eeab-4eef-b59c-c75528b5c6ee") : configmap references non-existent config key: service-ca.crt Sep 08 03:59:43.631781 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:43.631740 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/179fd61c-eeab-4eef-b59c-c75528b5c6ee-metrics-certs\") pod \"router-default-69bf99dbfb-78zrw\" (UID: \"179fd61c-eeab-4eef-b59c-c75528b5c6ee\") " pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 03:59:43.631974 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:43.631904 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 08 03:59:43.631974 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:43.631974 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/179fd61c-eeab-4eef-b59c-c75528b5c6ee-metrics-certs podName:179fd61c-eeab-4eef-b59c-c75528b5c6ee nodeName:}" failed. No retries permitted until 2026-09-08 03:59:47.631957337 +0000 UTC m=+247.252993283 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/179fd61c-eeab-4eef-b59c-c75528b5c6ee-metrics-certs") pod "router-default-69bf99dbfb-78zrw" (UID: "179fd61c-eeab-4eef-b59c-c75528b5c6ee") : secret "router-metrics-certs-default" not found Sep 08 03:59:43.765823 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:43.765730 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-t55cp" event={"ID":"ab64adad-879e-4ec6-8425-86a70f48615f","Type":"ContainerStarted","Data":"6e85049a5ea2f69a7b79ebba57f290dab43d3df4165038c4b8dee52b77bcb432"} Sep 08 03:59:43.785886 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:43.785838 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-t55cp" podStartSLOduration=3.451457863 podStartE2EDuration="5.785823086s" totalImagesPullingTime="2.334365223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 03:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 03:59:43.785500264 +0000 UTC m=+243.406536232" watchObservedRunningTime="2026-09-08 03:59:43.785823086 +0000 UTC m=+243.406859079" Sep 08 03:59:44.796743 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:44.796701 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qkkfr"] Sep 08 03:59:44.798795 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:44.798777 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qkkfr" Sep 08 03:59:44.806647 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:44.806628 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 08 03:59:44.806732 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:44.806629 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 08 03:59:44.807805 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:44.807785 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-h24jh\"" Sep 08 03:59:44.822240 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:44.822210 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qkkfr"] Sep 08 03:59:44.841431 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:44.841395 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9llcg\" (UniqueName: \"kubernetes.io/projected/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-kube-api-access-9llcg\") pod \"insights-runtime-extractor-qkkfr\" (UID: \"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62\") " pod="openshift-insights/insights-runtime-extractor-qkkfr" Sep 08 03:59:44.841599 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:44.841500 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qkkfr\" (UID: \"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62\") " pod="openshift-insights/insights-runtime-extractor-qkkfr" Sep 08 03:59:44.841599 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:44.841523 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-data-volume\") pod \"insights-runtime-extractor-qkkfr\" (UID: \"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62\") " pod="openshift-insights/insights-runtime-extractor-qkkfr" Sep 08 03:59:44.841686 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:44.841628 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-crio-socket\") pod \"insights-runtime-extractor-qkkfr\" (UID: \"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62\") " pod="openshift-insights/insights-runtime-extractor-qkkfr" Sep 08 03:59:44.841737 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:44.841720 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qkkfr\" (UID: \"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62\") " pod="openshift-insights/insights-runtime-extractor-qkkfr" Sep 08 03:59:44.942679 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:44.942640 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-data-volume\") pod \"insights-runtime-extractor-qkkfr\" (UID: \"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62\") " pod="openshift-insights/insights-runtime-extractor-qkkfr" Sep 08 03:59:44.942861 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:44.942716 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-crio-socket\") pod \"insights-runtime-extractor-qkkfr\" (UID: \"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62\") " pod="openshift-insights/insights-runtime-extractor-qkkfr" Sep 08 03:59:44.942861 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:44.942744 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qkkfr\" (UID: \"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62\") " pod="openshift-insights/insights-runtime-extractor-qkkfr" Sep 08 03:59:44.942861 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:44.942765 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9llcg\" (UniqueName: \"kubernetes.io/projected/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-kube-api-access-9llcg\") pod \"insights-runtime-extractor-qkkfr\" (UID: \"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62\") " pod="openshift-insights/insights-runtime-extractor-qkkfr" Sep 08 03:59:44.943030 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:44.942857 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-crio-socket\") pod \"insights-runtime-extractor-qkkfr\" (UID: \"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62\") " pod="openshift-insights/insights-runtime-extractor-qkkfr" Sep 08 03:59:44.943030 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:44.942866 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qkkfr\" (UID: \"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62\") " pod="openshift-insights/insights-runtime-extractor-qkkfr" Sep 08 03:59:44.943030 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:44.942877 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 03:59:44.943030 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:44.942956 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-kube-rbac-proxy-cm podName:dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62 nodeName:}" failed. No retries permitted until 2026-09-08 03:59:45.442937795 +0000 UTC m=+245.063973747 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qkkfr" (UID: "dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62") : configmap "kube-rbac-proxy" not found Sep 08 03:59:44.943030 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:44.942967 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 03:59:44.943030 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:44.943002 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-insights-runtime-extractor-tls podName:dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62 nodeName:}" failed. No retries permitted until 2026-09-08 03:59:45.442991288 +0000 UTC m=+245.064027246 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qkkfr" (UID: "dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62") : secret "insights-runtime-extractor-tls" not found Sep 08 03:59:44.943330 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:44.943058 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-data-volume\") pod \"insights-runtime-extractor-qkkfr\" (UID: \"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62\") " pod="openshift-insights/insights-runtime-extractor-qkkfr" Sep 08 03:59:44.944722 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:44.944694 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-cc86d"] Sep 08 03:59:44.948946 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:44.948926 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-cc86d" Sep 08 03:59:44.953090 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:44.953067 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Sep 08 03:59:44.953090 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:44.953084 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Sep 08 03:59:44.953259 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:44.953217 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-9mb85\"" Sep 08 03:59:44.956498 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:44.956478 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9llcg\" (UniqueName: \"kubernetes.io/projected/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-kube-api-access-9llcg\") pod \"insights-runtime-extractor-qkkfr\" (UID: \"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62\") " pod="openshift-insights/insights-runtime-extractor-qkkfr" Sep 08 03:59:44.960323 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:44.960302 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-cc86d"] Sep 08 03:59:45.043420 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:45.043389 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2jfn\" (UniqueName: \"kubernetes.io/projected/166ea36c-7359-47fc-ace9-ad5d46c0f52a-kube-api-access-g2jfn\") pod \"migrator-fdbb8fdff-cc86d\" (UID: \"166ea36c-7359-47fc-ace9-ad5d46c0f52a\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-cc86d" Sep 08 03:59:45.144255 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:45.144217 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2jfn\" (UniqueName: \"kubernetes.io/projected/166ea36c-7359-47fc-ace9-ad5d46c0f52a-kube-api-access-g2jfn\") pod \"migrator-fdbb8fdff-cc86d\" (UID: \"166ea36c-7359-47fc-ace9-ad5d46c0f52a\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-cc86d" Sep 08 03:59:45.156748 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:45.156723 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2jfn\" (UniqueName: \"kubernetes.io/projected/166ea36c-7359-47fc-ace9-ad5d46c0f52a-kube-api-access-g2jfn\") pod \"migrator-fdbb8fdff-cc86d\" (UID: \"166ea36c-7359-47fc-ace9-ad5d46c0f52a\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-cc86d" Sep 08 03:59:45.258038 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:45.257992 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-cc86d" Sep 08 03:59:45.447060 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:45.446965 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qkkfr\" (UID: \"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62\") " pod="openshift-insights/insights-runtime-extractor-qkkfr" Sep 08 03:59:45.447227 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:45.447098 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 03:59:45.447227 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:45.447132 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qkkfr\" (UID: \"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62\") " pod="openshift-insights/insights-runtime-extractor-qkkfr" Sep 08 03:59:45.447227 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:45.447167 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-kube-rbac-proxy-cm podName:dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62 nodeName:}" failed. No retries permitted until 2026-09-08 03:59:46.447146539 +0000 UTC m=+246.068182487 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qkkfr" (UID: "dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62") : configmap "kube-rbac-proxy" not found Sep 08 03:59:45.447227 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:45.447217 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 03:59:45.447400 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:45.447263 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-insights-runtime-extractor-tls podName:dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62 nodeName:}" failed. No retries permitted until 2026-09-08 03:59:46.447250738 +0000 UTC m=+246.068286683 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qkkfr" (UID: "dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62") : secret "insights-runtime-extractor-tls" not found Sep 08 03:59:45.528434 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:45.528403 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-cc86d"] Sep 08 03:59:45.531260 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:59:45.531228 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod166ea36c_7359_47fc_ace9_ad5d46c0f52a.slice/crio-8bb049f05dbe1cd5119cb28297e5ab3fd9defd033bee1bf6e75ebc93f431ab65 WatchSource:0}: Error finding container 8bb049f05dbe1cd5119cb28297e5ab3fd9defd033bee1bf6e75ebc93f431ab65: Status 404 returned error can't find the container with id 8bb049f05dbe1cd5119cb28297e5ab3fd9defd033bee1bf6e75ebc93f431ab65 Sep 08 03:59:45.773026 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:45.772942 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-cc86d" event={"ID":"166ea36c-7359-47fc-ace9-ad5d46c0f52a","Type":"ContainerStarted","Data":"8bb049f05dbe1cd5119cb28297e5ab3fd9defd033bee1bf6e75ebc93f431ab65"} Sep 08 03:59:46.456269 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:46.456232 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qkkfr\" (UID: \"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62\") " pod="openshift-insights/insights-runtime-extractor-qkkfr" Sep 08 03:59:46.456749 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:46.456339 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qkkfr\" (UID: \"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62\") " pod="openshift-insights/insights-runtime-extractor-qkkfr" Sep 08 03:59:46.456749 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:46.456356 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 03:59:46.456749 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:46.456424 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-kube-rbac-proxy-cm podName:dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62 nodeName:}" failed. No retries permitted until 2026-09-08 03:59:48.456405777 +0000 UTC m=+248.077441741 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qkkfr" (UID: "dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62") : configmap "kube-rbac-proxy" not found Sep 08 03:59:46.456749 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:46.456447 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 03:59:46.456749 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:46.456498 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-insights-runtime-extractor-tls podName:dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62 nodeName:}" failed. No retries permitted until 2026-09-08 03:59:48.456487169 +0000 UTC m=+248.077523116 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qkkfr" (UID: "dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62") : secret "insights-runtime-extractor-tls" not found Sep 08 03:59:46.537045 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:46.537018 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-djmk2_ec702c02-1c7e-471b-9783-5593eefc92a3/dns-node-resolver/0.log" Sep 08 03:59:46.727299 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:46.727216 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-k49w5"] Sep 08 03:59:46.729634 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:46.729612 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-k49w5" Sep 08 03:59:46.733256 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:46.733228 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 08 03:59:46.733880 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:46.733860 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 08 03:59:46.733986 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:46.733864 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 08 03:59:46.733986 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:46.733900 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-cqds4\"" Sep 08 03:59:46.734131 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:46.734114 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 08 03:59:46.740066 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:46.740042 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-k49w5"] Sep 08 03:59:46.858648 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:46.858607 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3027dc6f-51e4-45de-adfc-3f426e71e9b9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-dxhpn\" (UID: \"3027dc6f-51e4-45de-adfc-3f426e71e9b9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dxhpn" Sep 08 03:59:46.858757 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:46.858657 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxgrw\" (UniqueName: \"kubernetes.io/projected/cd2ba244-d3cf-4f03-8dec-cdc307039e43-kube-api-access-pxgrw\") pod \"service-ca-75d99c484-k49w5\" (UID: \"cd2ba244-d3cf-4f03-8dec-cdc307039e43\") " pod="openshift-service-ca/service-ca-75d99c484-k49w5" Sep 08 03:59:46.858757 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:46.858726 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/cd2ba244-d3cf-4f03-8dec-cdc307039e43-signing-cabundle\") pod \"service-ca-75d99c484-k49w5\" (UID: \"cd2ba244-d3cf-4f03-8dec-cdc307039e43\") " pod="openshift-service-ca/service-ca-75d99c484-k49w5" Sep 08 03:59:46.858757 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:46.858750 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/cd2ba244-d3cf-4f03-8dec-cdc307039e43-signing-key\") pod \"service-ca-75d99c484-k49w5\" (UID: \"cd2ba244-d3cf-4f03-8dec-cdc307039e43\") " pod="openshift-service-ca/service-ca-75d99c484-k49w5" Sep 08 03:59:46.858901 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:46.858755 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 08 03:59:46.858901 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:46.858840 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3027dc6f-51e4-45de-adfc-3f426e71e9b9-cluster-monitoring-operator-tls podName:3027dc6f-51e4-45de-adfc-3f426e71e9b9 nodeName:}" failed. No retries permitted until 2026-09-08 03:59:54.858821616 +0000 UTC m=+254.479857576 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3027dc6f-51e4-45de-adfc-3f426e71e9b9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-dxhpn" (UID: "3027dc6f-51e4-45de-adfc-3f426e71e9b9") : secret "cluster-monitoring-operator-tls" not found Sep 08 03:59:46.959523 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:46.959485 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxgrw\" (UniqueName: \"kubernetes.io/projected/cd2ba244-d3cf-4f03-8dec-cdc307039e43-kube-api-access-pxgrw\") pod \"service-ca-75d99c484-k49w5\" (UID: \"cd2ba244-d3cf-4f03-8dec-cdc307039e43\") " pod="openshift-service-ca/service-ca-75d99c484-k49w5" Sep 08 03:59:46.959702 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:46.959543 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/cd2ba244-d3cf-4f03-8dec-cdc307039e43-signing-cabundle\") pod \"service-ca-75d99c484-k49w5\" (UID: \"cd2ba244-d3cf-4f03-8dec-cdc307039e43\") " pod="openshift-service-ca/service-ca-75d99c484-k49w5" Sep 08 03:59:46.959702 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:46.959560 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/cd2ba244-d3cf-4f03-8dec-cdc307039e43-signing-key\") pod \"service-ca-75d99c484-k49w5\" (UID: \"cd2ba244-d3cf-4f03-8dec-cdc307039e43\") " pod="openshift-service-ca/service-ca-75d99c484-k49w5" Sep 08 03:59:46.960245 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:46.960223 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/cd2ba244-d3cf-4f03-8dec-cdc307039e43-signing-cabundle\") pod \"service-ca-75d99c484-k49w5\" (UID: \"cd2ba244-d3cf-4f03-8dec-cdc307039e43\") " pod="openshift-service-ca/service-ca-75d99c484-k49w5" Sep 08 03:59:46.961836 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:46.961813 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/cd2ba244-d3cf-4f03-8dec-cdc307039e43-signing-key\") pod \"service-ca-75d99c484-k49w5\" (UID: \"cd2ba244-d3cf-4f03-8dec-cdc307039e43\") " pod="openshift-service-ca/service-ca-75d99c484-k49w5" Sep 08 03:59:46.969759 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:46.969736 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxgrw\" (UniqueName: \"kubernetes.io/projected/cd2ba244-d3cf-4f03-8dec-cdc307039e43-kube-api-access-pxgrw\") pod \"service-ca-75d99c484-k49w5\" (UID: \"cd2ba244-d3cf-4f03-8dec-cdc307039e43\") " pod="openshift-service-ca/service-ca-75d99c484-k49w5" Sep 08 03:59:47.040907 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:47.040821 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-k49w5" Sep 08 03:59:47.315374 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:47.315286 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-k49w5"] Sep 08 03:59:47.318176 ip-10-0-128-141 kubenswrapper[2579]: W0908 03:59:47.318150 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd2ba244_d3cf_4f03_8dec_cdc307039e43.slice/crio-382a516a878fc8e601c49ca130b58b1c0aedecc5c5fc58e370bfe28a047a0090 WatchSource:0}: Error finding container 382a516a878fc8e601c49ca130b58b1c0aedecc5c5fc58e370bfe28a047a0090: Status 404 returned error can't find the container with id 382a516a878fc8e601c49ca130b58b1c0aedecc5c5fc58e370bfe28a047a0090 Sep 08 03:59:47.565531 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:47.565405 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/179fd61c-eeab-4eef-b59c-c75528b5c6ee-service-ca-bundle\") pod \"router-default-69bf99dbfb-78zrw\" (UID: \"179fd61c-eeab-4eef-b59c-c75528b5c6ee\") " pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 03:59:47.565909 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:47.565602 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/179fd61c-eeab-4eef-b59c-c75528b5c6ee-service-ca-bundle podName:179fd61c-eeab-4eef-b59c-c75528b5c6ee nodeName:}" failed. No retries permitted until 2026-09-08 03:59:55.565584173 +0000 UTC m=+255.186620117 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/179fd61c-eeab-4eef-b59c-c75528b5c6ee-service-ca-bundle") pod "router-default-69bf99dbfb-78zrw" (UID: "179fd61c-eeab-4eef-b59c-c75528b5c6ee") : configmap references non-existent config key: service-ca.crt Sep 08 03:59:47.666165 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:47.666133 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/179fd61c-eeab-4eef-b59c-c75528b5c6ee-metrics-certs\") pod \"router-default-69bf99dbfb-78zrw\" (UID: \"179fd61c-eeab-4eef-b59c-c75528b5c6ee\") " pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 03:59:47.666343 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:47.666283 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 08 03:59:47.666396 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:47.666345 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/179fd61c-eeab-4eef-b59c-c75528b5c6ee-metrics-certs podName:179fd61c-eeab-4eef-b59c-c75528b5c6ee nodeName:}" failed. No retries permitted until 2026-09-08 03:59:55.666330141 +0000 UTC m=+255.287366086 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/179fd61c-eeab-4eef-b59c-c75528b5c6ee-metrics-certs") pod "router-default-69bf99dbfb-78zrw" (UID: "179fd61c-eeab-4eef-b59c-c75528b5c6ee") : secret "router-metrics-certs-default" not found Sep 08 03:59:47.783229 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:47.783190 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-cc86d" event={"ID":"166ea36c-7359-47fc-ace9-ad5d46c0f52a","Type":"ContainerStarted","Data":"470d5288bd9c759965f9bc4b71ecaf0cc305fe8a0c57ff6fa6993fc9ffb3fc64"} Sep 08 03:59:47.783402 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:47.783258 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-cc86d" event={"ID":"166ea36c-7359-47fc-ace9-ad5d46c0f52a","Type":"ContainerStarted","Data":"76bb61f2dba384aa16e67a8783035592ddd48ae50d91c0134f49507664fa3faa"} Sep 08 03:59:47.784721 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:47.784687 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-k49w5" event={"ID":"cd2ba244-d3cf-4f03-8dec-cdc307039e43","Type":"ContainerStarted","Data":"382a516a878fc8e601c49ca130b58b1c0aedecc5c5fc58e370bfe28a047a0090"} Sep 08 03:59:47.937688 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:47.937660 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wmrvz_308e3330-d662-4672-8e7f-9892c04a5491/node-ca/0.log" Sep 08 03:59:48.339734 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:48.339649 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fdn8c_9d27b634-b977-4b2b-b5ad-0766b2b0e1e2/serve-healthcheck-canary/0.log" Sep 08 03:59:48.473740 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:48.473696 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qkkfr\" (UID: \"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62\") " pod="openshift-insights/insights-runtime-extractor-qkkfr" Sep 08 03:59:48.473937 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:48.473782 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qkkfr\" (UID: \"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62\") " pod="openshift-insights/insights-runtime-extractor-qkkfr" Sep 08 03:59:48.473937 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:48.473881 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 03:59:48.473937 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:48.473918 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 03:59:48.474102 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:48.473961 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-insights-runtime-extractor-tls podName:dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62 nodeName:}" failed. No retries permitted until 2026-09-08 03:59:52.473940669 +0000 UTC m=+252.094976618 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qkkfr" (UID: "dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62") : secret "insights-runtime-extractor-tls" not found Sep 08 03:59:48.474102 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:48.473981 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-kube-rbac-proxy-cm podName:dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62 nodeName:}" failed. No retries permitted until 2026-09-08 03:59:52.473971491 +0000 UTC m=+252.095007442 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qkkfr" (UID: "dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62") : configmap "kube-rbac-proxy" not found Sep 08 03:59:49.793660 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:49.793625 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-k49w5" event={"ID":"cd2ba244-d3cf-4f03-8dec-cdc307039e43","Type":"ContainerStarted","Data":"c5777ca23185a9874fff1b9fbc2af6bce32e2ce3cc29aefec452d8750a775f64"} Sep 08 03:59:49.823735 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:49.823682 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-cc86d" podStartSLOduration=4.533769121 podStartE2EDuration="5.823665397s" totalImagesPullingTime="1.289896276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 03:59:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 03:59:47.81308903 +0000 UTC m=+247.434124997" watchObservedRunningTime="2026-09-08 03:59:49.823665397 +0000 UTC m=+249.444701363" Sep 08 03:59:49.823934 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:49.823762 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-k49w5" podStartSLOduration=2.035435608 podStartE2EDuration="3.823757656s" totalImagesPullingTime="1.788322048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 03:59:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 03:59:49.823178358 +0000 UTC m=+249.444214327" watchObservedRunningTime="2026-09-08 03:59:49.823757656 +0000 UTC m=+249.444793622" Sep 08 03:59:51.704906 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:51.704870 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs\") pod \"network-metrics-daemon-r4xfl\" (UID: \"b7239679-8052-415f-ae9a-1b20d61d1223\") " pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 03:59:51.705392 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:51.705067 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 08 03:59:51.705392 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:51.705148 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs podName:b7239679-8052-415f-ae9a-1b20d61d1223 nodeName:}" failed. No retries permitted until 2026-09-08 04:01:53.705127803 +0000 UTC m=+373.326163764 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs") pod "network-metrics-daemon-r4xfl" (UID: "b7239679-8052-415f-ae9a-1b20d61d1223") : secret "metrics-daemon-secret" not found Sep 08 03:59:52.512566 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:52.512526 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qkkfr\" (UID: \"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62\") " pod="openshift-insights/insights-runtime-extractor-qkkfr" Sep 08 03:59:52.512761 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:52.512628 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qkkfr\" (UID: \"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62\") " pod="openshift-insights/insights-runtime-extractor-qkkfr" Sep 08 03:59:52.512761 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:52.512676 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 03:59:52.512761 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:52.512746 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-insights-runtime-extractor-tls podName:dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62 nodeName:}" failed. No retries permitted until 2026-09-08 04:00:00.512730803 +0000 UTC m=+260.133766748 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qkkfr" (UID: "dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62") : secret "insights-runtime-extractor-tls" not found Sep 08 03:59:52.512893 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:52.512760 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 03:59:52.512893 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:52.512820 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-kube-rbac-proxy-cm podName:dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62 nodeName:}" failed. No retries permitted until 2026-09-08 04:00:00.512803343 +0000 UTC m=+260.133839344 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qkkfr" (UID: "dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62") : configmap "kube-rbac-proxy" not found Sep 08 03:59:54.933032 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:54.932987 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3027dc6f-51e4-45de-adfc-3f426e71e9b9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-dxhpn\" (UID: \"3027dc6f-51e4-45de-adfc-3f426e71e9b9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dxhpn" Sep 08 03:59:54.933557 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:54.933530 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 08 03:59:54.933641 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:54.933627 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3027dc6f-51e4-45de-adfc-3f426e71e9b9-cluster-monitoring-operator-tls podName:3027dc6f-51e4-45de-adfc-3f426e71e9b9 nodeName:}" failed. No retries permitted until 2026-09-08 04:00:10.933606538 +0000 UTC m=+270.554642502 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3027dc6f-51e4-45de-adfc-3f426e71e9b9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-dxhpn" (UID: "3027dc6f-51e4-45de-adfc-3f426e71e9b9") : secret "cluster-monitoring-operator-tls" not found Sep 08 03:59:55.639809 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:55.639773 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/179fd61c-eeab-4eef-b59c-c75528b5c6ee-service-ca-bundle\") pod \"router-default-69bf99dbfb-78zrw\" (UID: \"179fd61c-eeab-4eef-b59c-c75528b5c6ee\") " pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 03:59:55.639971 ip-10-0-128-141 kubenswrapper[2579]: E0908 03:59:55.639897 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/179fd61c-eeab-4eef-b59c-c75528b5c6ee-service-ca-bundle podName:179fd61c-eeab-4eef-b59c-c75528b5c6ee nodeName:}" failed. No retries permitted until 2026-09-08 04:00:11.639882859 +0000 UTC m=+271.260918804 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/179fd61c-eeab-4eef-b59c-c75528b5c6ee-service-ca-bundle") pod "router-default-69bf99dbfb-78zrw" (UID: "179fd61c-eeab-4eef-b59c-c75528b5c6ee") : configmap references non-existent config key: service-ca.crt Sep 08 03:59:55.741117 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:55.741074 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/179fd61c-eeab-4eef-b59c-c75528b5c6ee-metrics-certs\") pod \"router-default-69bf99dbfb-78zrw\" (UID: \"179fd61c-eeab-4eef-b59c-c75528b5c6ee\") " pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 03:59:55.743572 ip-10-0-128-141 kubenswrapper[2579]: I0908 03:59:55.743541 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/179fd61c-eeab-4eef-b59c-c75528b5c6ee-metrics-certs\") pod \"router-default-69bf99dbfb-78zrw\" (UID: \"179fd61c-eeab-4eef-b59c-c75528b5c6ee\") " pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 04:00:00.577996 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:00.577953 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qkkfr\" (UID: \"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62\") " pod="openshift-insights/insights-runtime-extractor-qkkfr" Sep 08 04:00:00.578510 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:00.578113 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qkkfr\" (UID: \"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62\") " pod="openshift-insights/insights-runtime-extractor-qkkfr" Sep 08 04:00:00.578711 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:00.578689 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qkkfr\" (UID: \"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62\") " pod="openshift-insights/insights-runtime-extractor-qkkfr" Sep 08 04:00:00.580297 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:00.580277 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qkkfr\" (UID: \"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62\") " pod="openshift-insights/insights-runtime-extractor-qkkfr" Sep 08 04:00:00.707306 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:00.707269 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qkkfr" Sep 08 04:00:01.007854 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:01.007832 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qkkfr"] Sep 08 04:00:01.010232 ip-10-0-128-141 kubenswrapper[2579]: W0908 04:00:01.010202 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd2c3b6c_f530_4e7c_b2a0_57877c3f5b62.slice/crio-8768ce18efc4df1c8d65f50383297eb7102b80a4126e8bbf2cb8b675147d2b8a WatchSource:0}: Error finding container 8768ce18efc4df1c8d65f50383297eb7102b80a4126e8bbf2cb8b675147d2b8a: Status 404 returned error can't find the container with id 8768ce18efc4df1c8d65f50383297eb7102b80a4126e8bbf2cb8b675147d2b8a Sep 08 04:00:01.835473 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:01.835425 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qkkfr" event={"ID":"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62","Type":"ContainerStarted","Data":"0a5673e27327a49712834adee063198ce2e53040c3b00924a7ce9a4442cf8567"} Sep 08 04:00:01.835834 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:01.835523 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qkkfr" event={"ID":"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62","Type":"ContainerStarted","Data":"8768ce18efc4df1c8d65f50383297eb7102b80a4126e8bbf2cb8b675147d2b8a"} Sep 08 04:00:02.840428 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:02.840387 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qkkfr" event={"ID":"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62","Type":"ContainerStarted","Data":"f5d98453e9a7d58a4af24693d6abce08151e717b62a0bc118bf164dfb2425b7f"} Sep 08 04:00:04.847993 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:04.847956 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qkkfr" event={"ID":"dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62","Type":"ContainerStarted","Data":"c91ecc2e46400cd01aaa738be2f5468676e827446fcd5cbbf93bdc2028c66787"} Sep 08 04:00:04.878118 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:04.878067 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qkkfr" podStartSLOduration=17.235688573 podStartE2EDuration="20.878052512s" totalImagesPullingTime="3.642363939s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 03:59:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:00:04.877556815 +0000 UTC m=+264.498592784" watchObservedRunningTime="2026-09-08 04:00:04.878052512 +0000 UTC m=+264.499088478" Sep 08 04:00:10.961661 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:10.961608 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3027dc6f-51e4-45de-adfc-3f426e71e9b9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-dxhpn\" (UID: \"3027dc6f-51e4-45de-adfc-3f426e71e9b9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dxhpn" Sep 08 04:00:10.964141 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:10.964112 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3027dc6f-51e4-45de-adfc-3f426e71e9b9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-dxhpn\" (UID: \"3027dc6f-51e4-45de-adfc-3f426e71e9b9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dxhpn" Sep 08 04:00:11.184481 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:11.184431 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-rgjvv\"" Sep 08 04:00:11.191230 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:11.191202 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dxhpn" Sep 08 04:00:11.474556 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:11.474513 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dxhpn"] Sep 08 04:00:11.478578 ip-10-0-128-141 kubenswrapper[2579]: W0908 04:00:11.478556 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3027dc6f_51e4_45de_adfc_3f426e71e9b9.slice/crio-8dceebf4aaee8965de90bea7501468f91a9c4bdb2c4ba8e51313f5ffa3a4cb6e WatchSource:0}: Error finding container 8dceebf4aaee8965de90bea7501468f91a9c4bdb2c4ba8e51313f5ffa3a4cb6e: Status 404 returned error can't find the container with id 8dceebf4aaee8965de90bea7501468f91a9c4bdb2c4ba8e51313f5ffa3a4cb6e Sep 08 04:00:11.667171 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:11.667129 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/179fd61c-eeab-4eef-b59c-c75528b5c6ee-service-ca-bundle\") pod \"router-default-69bf99dbfb-78zrw\" (UID: \"179fd61c-eeab-4eef-b59c-c75528b5c6ee\") " pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 04:00:11.667841 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:11.667819 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/179fd61c-eeab-4eef-b59c-c75528b5c6ee-service-ca-bundle\") pod \"router-default-69bf99dbfb-78zrw\" (UID: \"179fd61c-eeab-4eef-b59c-c75528b5c6ee\") " pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 04:00:11.869802 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:11.869764 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dxhpn" event={"ID":"3027dc6f-51e4-45de-adfc-3f426e71e9b9","Type":"ContainerStarted","Data":"8dceebf4aaee8965de90bea7501468f91a9c4bdb2c4ba8e51313f5ffa3a4cb6e"} Sep 08 04:00:11.927102 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:11.927036 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qg2m5\"" Sep 08 04:00:11.934673 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:11.934653 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 04:00:12.212350 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:12.212276 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-69bf99dbfb-78zrw"] Sep 08 04:00:12.215180 ip-10-0-128-141 kubenswrapper[2579]: W0908 04:00:12.215153 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod179fd61c_eeab_4eef_b59c_c75528b5c6ee.slice/crio-aa3ba1fe41af94da6053bf2d8d348958ec9536c7f5bdda1e1bf7140251e2d6e1 WatchSource:0}: Error finding container aa3ba1fe41af94da6053bf2d8d348958ec9536c7f5bdda1e1bf7140251e2d6e1: Status 404 returned error can't find the container with id aa3ba1fe41af94da6053bf2d8d348958ec9536c7f5bdda1e1bf7140251e2d6e1 Sep 08 04:00:12.876991 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:12.876961 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-69bf99dbfb-78zrw" event={"ID":"179fd61c-eeab-4eef-b59c-c75528b5c6ee","Type":"ContainerStarted","Data":"6d81412f10680d6a658a1e69a6754e0bc71df1726221d26a77e2cf73764852a2"} Sep 08 04:00:12.877179 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:12.877011 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-69bf99dbfb-78zrw" event={"ID":"179fd61c-eeab-4eef-b59c-c75528b5c6ee","Type":"ContainerStarted","Data":"aa3ba1fe41af94da6053bf2d8d348958ec9536c7f5bdda1e1bf7140251e2d6e1"} Sep 08 04:00:12.910229 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:12.910173 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-69bf99dbfb-78zrw" podStartSLOduration=34.910155758 podStartE2EDuration="34.910155758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 03:59:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:00:12.910003214 +0000 UTC m=+272.531039181" watchObservedRunningTime="2026-09-08 04:00:12.910155758 +0000 UTC m=+272.531191752" Sep 08 04:00:12.935696 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:12.935664 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 04:00:12.938644 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:12.938618 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 04:00:13.882384 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:13.882348 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dxhpn" event={"ID":"3027dc6f-51e4-45de-adfc-3f426e71e9b9","Type":"ContainerStarted","Data":"336836f75e847881ae94e6fce62a4ee7f8def8e1a2c94d41119087453d18c80f"} Sep 08 04:00:13.882843 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:13.882605 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 04:00:13.883756 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:13.883733 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-69bf99dbfb-78zrw" Sep 08 04:00:13.995420 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:13.995367 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dxhpn" podStartSLOduration=33.904388835 podStartE2EDuration="35.995348866s" totalImagesPullingTime="2.090960031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 03:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:00:13.995133604 +0000 UTC m=+273.616169573" watchObservedRunningTime="2026-09-08 04:00:13.995348866 +0000 UTC m=+273.616384852" Sep 08 04:00:20.551770 ip-10-0-128-141 kubenswrapper[2579]: E0908 04:00:20.551707 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-2jhc2" podUID="77b984c3-52a4-442d-9356-f05a01aaa1bb" Sep 08 04:00:20.902976 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:20.902943 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2jhc2" Sep 08 04:00:24.261947 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.261908 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls\") pod \"dns-default-2jhc2\" (UID: \"77b984c3-52a4-442d-9356-f05a01aaa1bb\") " pod="openshift-dns/dns-default-2jhc2" Sep 08 04:00:24.264452 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.264425 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/77b984c3-52a4-442d-9356-f05a01aaa1bb-metrics-tls\") pod \"dns-default-2jhc2\" (UID: \"77b984c3-52a4-442d-9356-f05a01aaa1bb\") " pod="openshift-dns/dns-default-2jhc2" Sep 08 04:00:24.510052 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.510013 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-sl2fz\"" Sep 08 04:00:24.514386 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.514321 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2jhc2" Sep 08 04:00:24.813173 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.813037 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2jhc2"] Sep 08 04:00:24.816282 ip-10-0-128-141 kubenswrapper[2579]: W0908 04:00:24.816241 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77b984c3_52a4_442d_9356_f05a01aaa1bb.slice/crio-79dd7ec8b62af3230e4c326c1a0e21a33397d04416502c334a657e356e89bc5b WatchSource:0}: Error finding container 79dd7ec8b62af3230e4c326c1a0e21a33397d04416502c334a657e356e89bc5b: Status 404 returned error can't find the container with id 79dd7ec8b62af3230e4c326c1a0e21a33397d04416502c334a657e356e89bc5b Sep 08 04:00:24.865688 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.865653 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf"] Sep 08 04:00:24.871534 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.871499 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf" Sep 08 04:00:24.878168 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.877862 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 08 04:00:24.880001 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.879410 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-m67qg"] Sep 08 04:00:24.881370 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.881347 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 08 04:00:24.882141 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.882115 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-t2hsq\"" Sep 08 04:00:24.882237 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.882147 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 08 04:00:24.883385 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.883365 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-m67qg" Sep 08 04:00:24.886752 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.886731 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-775m2\"" Sep 08 04:00:24.887017 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.886996 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 08 04:00:24.887254 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.887234 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 08 04:00:24.888094 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.888075 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 08 04:00:24.917542 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.917503 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2jhc2" event={"ID":"77b984c3-52a4-442d-9356-f05a01aaa1bb","Type":"ContainerStarted","Data":"79dd7ec8b62af3230e4c326c1a0e21a33397d04416502c334a657e356e89bc5b"} Sep 08 04:00:24.917736 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.917574 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-m67qg"] Sep 08 04:00:24.918640 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.918603 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf"] Sep 08 04:00:24.919783 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.919762 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wm4h4"] Sep 08 04:00:24.923757 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.923733 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:24.926780 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.926754 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 08 04:00:24.926904 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.926811 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 08 04:00:24.927144 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.927100 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-7k9nw\"" Sep 08 04:00:24.927346 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.927325 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 08 04:00:24.966577 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.966540 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9c080e9-aeba-4554-a0a2-66cf8881dce3-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-m67qg\" (UID: \"f9c080e9-aeba-4554-a0a2-66cf8881dce3\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-m67qg" Sep 08 04:00:24.966794 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.966598 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f9c080e9-aeba-4554-a0a2-66cf8881dce3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-m67qg\" (UID: \"f9c080e9-aeba-4554-a0a2-66cf8881dce3\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-m67qg" Sep 08 04:00:24.966794 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.966632 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b100a94e-b7da-4c28-9fd4-0842380dc1f1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wm4h4\" (UID: \"b100a94e-b7da-4c28-9fd4-0842380dc1f1\") " pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:24.966794 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.966711 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d6b54952-ee54-48f0-a1ce-c0255b19fe37-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-kfqkf\" (UID: \"d6b54952-ee54-48f0-a1ce-c0255b19fe37\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf" Sep 08 04:00:24.966794 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.966738 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b100a94e-b7da-4c28-9fd4-0842380dc1f1-node-exporter-accelerators-collector-config\") pod \"node-exporter-wm4h4\" (UID: \"b100a94e-b7da-4c28-9fd4-0842380dc1f1\") " pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:24.967001 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.966791 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4xzz\" (UniqueName: \"kubernetes.io/projected/d6b54952-ee54-48f0-a1ce-c0255b19fe37-kube-api-access-f4xzz\") pod \"kube-state-metrics-74cb57df7-kfqkf\" (UID: \"d6b54952-ee54-48f0-a1ce-c0255b19fe37\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf" Sep 08 04:00:24.967001 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.966819 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b100a94e-b7da-4c28-9fd4-0842380dc1f1-metrics-client-ca\") pod \"node-exporter-wm4h4\" (UID: \"b100a94e-b7da-4c28-9fd4-0842380dc1f1\") " pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:24.967001 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.966854 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54f4w\" (UniqueName: \"kubernetes.io/projected/f9c080e9-aeba-4554-a0a2-66cf8881dce3-kube-api-access-54f4w\") pod \"openshift-state-metrics-657fd6c54-m67qg\" (UID: \"f9c080e9-aeba-4554-a0a2-66cf8881dce3\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-m67qg" Sep 08 04:00:24.967001 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.966938 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d6b54952-ee54-48f0-a1ce-c0255b19fe37-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-kfqkf\" (UID: \"d6b54952-ee54-48f0-a1ce-c0255b19fe37\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf" Sep 08 04:00:24.967001 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.966979 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b100a94e-b7da-4c28-9fd4-0842380dc1f1-node-exporter-tls\") pod \"node-exporter-wm4h4\" (UID: \"b100a94e-b7da-4c28-9fd4-0842380dc1f1\") " pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:24.967238 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.967015 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9m9p\" (UniqueName: \"kubernetes.io/projected/b100a94e-b7da-4c28-9fd4-0842380dc1f1-kube-api-access-b9m9p\") pod \"node-exporter-wm4h4\" (UID: \"b100a94e-b7da-4c28-9fd4-0842380dc1f1\") " pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:24.967238 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.967057 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6b54952-ee54-48f0-a1ce-c0255b19fe37-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-kfqkf\" (UID: \"d6b54952-ee54-48f0-a1ce-c0255b19fe37\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf" Sep 08 04:00:24.967238 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.967088 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b100a94e-b7da-4c28-9fd4-0842380dc1f1-node-exporter-textfile\") pod \"node-exporter-wm4h4\" (UID: \"b100a94e-b7da-4c28-9fd4-0842380dc1f1\") " pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:24.967238 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.967130 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d6b54952-ee54-48f0-a1ce-c0255b19fe37-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-kfqkf\" (UID: \"d6b54952-ee54-48f0-a1ce-c0255b19fe37\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf" Sep 08 04:00:24.967238 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.967181 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b100a94e-b7da-4c28-9fd4-0842380dc1f1-node-exporter-wtmp\") pod \"node-exporter-wm4h4\" (UID: \"b100a94e-b7da-4c28-9fd4-0842380dc1f1\") " pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:24.967238 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.967217 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d6b54952-ee54-48f0-a1ce-c0255b19fe37-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-kfqkf\" (UID: \"d6b54952-ee54-48f0-a1ce-c0255b19fe37\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf" Sep 08 04:00:24.967552 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.967240 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9c080e9-aeba-4554-a0a2-66cf8881dce3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-m67qg\" (UID: \"f9c080e9-aeba-4554-a0a2-66cf8881dce3\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-m67qg" Sep 08 04:00:24.967552 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.967261 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b100a94e-b7da-4c28-9fd4-0842380dc1f1-sys\") pod \"node-exporter-wm4h4\" (UID: \"b100a94e-b7da-4c28-9fd4-0842380dc1f1\") " pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:24.967552 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:24.967285 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b100a94e-b7da-4c28-9fd4-0842380dc1f1-root\") pod \"node-exporter-wm4h4\" (UID: \"b100a94e-b7da-4c28-9fd4-0842380dc1f1\") " pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:25.068427 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.068351 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9c080e9-aeba-4554-a0a2-66cf8881dce3-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-m67qg\" (UID: \"f9c080e9-aeba-4554-a0a2-66cf8881dce3\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-m67qg" Sep 08 04:00:25.068427 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.068393 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f9c080e9-aeba-4554-a0a2-66cf8881dce3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-m67qg\" (UID: \"f9c080e9-aeba-4554-a0a2-66cf8881dce3\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-m67qg" Sep 08 04:00:25.069948 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.068424 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b100a94e-b7da-4c28-9fd4-0842380dc1f1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wm4h4\" (UID: \"b100a94e-b7da-4c28-9fd4-0842380dc1f1\") " pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:25.069948 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.068482 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d6b54952-ee54-48f0-a1ce-c0255b19fe37-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-kfqkf\" (UID: \"d6b54952-ee54-48f0-a1ce-c0255b19fe37\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf" Sep 08 04:00:25.069948 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.068509 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b100a94e-b7da-4c28-9fd4-0842380dc1f1-node-exporter-accelerators-collector-config\") pod \"node-exporter-wm4h4\" (UID: \"b100a94e-b7da-4c28-9fd4-0842380dc1f1\") " pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:25.069948 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.068536 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4xzz\" (UniqueName: \"kubernetes.io/projected/d6b54952-ee54-48f0-a1ce-c0255b19fe37-kube-api-access-f4xzz\") pod \"kube-state-metrics-74cb57df7-kfqkf\" (UID: \"d6b54952-ee54-48f0-a1ce-c0255b19fe37\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf" Sep 08 04:00:25.069948 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.068560 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b100a94e-b7da-4c28-9fd4-0842380dc1f1-metrics-client-ca\") pod \"node-exporter-wm4h4\" (UID: \"b100a94e-b7da-4c28-9fd4-0842380dc1f1\") " pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:25.069948 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.068593 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54f4w\" (UniqueName: \"kubernetes.io/projected/f9c080e9-aeba-4554-a0a2-66cf8881dce3-kube-api-access-54f4w\") pod \"openshift-state-metrics-657fd6c54-m67qg\" (UID: \"f9c080e9-aeba-4554-a0a2-66cf8881dce3\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-m67qg" Sep 08 04:00:25.069948 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.068642 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d6b54952-ee54-48f0-a1ce-c0255b19fe37-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-kfqkf\" (UID: \"d6b54952-ee54-48f0-a1ce-c0255b19fe37\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf" Sep 08 04:00:25.069948 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.068666 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b100a94e-b7da-4c28-9fd4-0842380dc1f1-node-exporter-tls\") pod \"node-exporter-wm4h4\" (UID: \"b100a94e-b7da-4c28-9fd4-0842380dc1f1\") " pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:25.069948 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.068689 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9m9p\" (UniqueName: \"kubernetes.io/projected/b100a94e-b7da-4c28-9fd4-0842380dc1f1-kube-api-access-b9m9p\") pod \"node-exporter-wm4h4\" (UID: \"b100a94e-b7da-4c28-9fd4-0842380dc1f1\") " pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:25.069948 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.068716 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6b54952-ee54-48f0-a1ce-c0255b19fe37-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-kfqkf\" (UID: \"d6b54952-ee54-48f0-a1ce-c0255b19fe37\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf" Sep 08 04:00:25.069948 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.068740 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b100a94e-b7da-4c28-9fd4-0842380dc1f1-node-exporter-textfile\") pod \"node-exporter-wm4h4\" (UID: \"b100a94e-b7da-4c28-9fd4-0842380dc1f1\") " pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:25.069948 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.068769 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d6b54952-ee54-48f0-a1ce-c0255b19fe37-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-kfqkf\" (UID: \"d6b54952-ee54-48f0-a1ce-c0255b19fe37\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf" Sep 08 04:00:25.069948 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.068797 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b100a94e-b7da-4c28-9fd4-0842380dc1f1-node-exporter-wtmp\") pod \"node-exporter-wm4h4\" (UID: \"b100a94e-b7da-4c28-9fd4-0842380dc1f1\") " pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:25.069948 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.068832 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d6b54952-ee54-48f0-a1ce-c0255b19fe37-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-kfqkf\" (UID: \"d6b54952-ee54-48f0-a1ce-c0255b19fe37\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf" Sep 08 04:00:25.069948 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.068857 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9c080e9-aeba-4554-a0a2-66cf8881dce3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-m67qg\" (UID: \"f9c080e9-aeba-4554-a0a2-66cf8881dce3\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-m67qg" Sep 08 04:00:25.069948 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.068880 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b100a94e-b7da-4c28-9fd4-0842380dc1f1-sys\") pod \"node-exporter-wm4h4\" (UID: \"b100a94e-b7da-4c28-9fd4-0842380dc1f1\") " pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:25.070858 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.068901 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b100a94e-b7da-4c28-9fd4-0842380dc1f1-root\") pod \"node-exporter-wm4h4\" (UID: \"b100a94e-b7da-4c28-9fd4-0842380dc1f1\") " pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:25.070858 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.068925 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d6b54952-ee54-48f0-a1ce-c0255b19fe37-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-kfqkf\" (UID: \"d6b54952-ee54-48f0-a1ce-c0255b19fe37\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf" Sep 08 04:00:25.070858 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.068988 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b100a94e-b7da-4c28-9fd4-0842380dc1f1-root\") pod \"node-exporter-wm4h4\" (UID: \"b100a94e-b7da-4c28-9fd4-0842380dc1f1\") " pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:25.070858 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.069201 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b100a94e-b7da-4c28-9fd4-0842380dc1f1-node-exporter-accelerators-collector-config\") pod \"node-exporter-wm4h4\" (UID: \"b100a94e-b7da-4c28-9fd4-0842380dc1f1\") " pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:25.070858 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.069208 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b100a94e-b7da-4c28-9fd4-0842380dc1f1-node-exporter-textfile\") pod \"node-exporter-wm4h4\" (UID: \"b100a94e-b7da-4c28-9fd4-0842380dc1f1\") " pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:25.070858 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.069250 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9c080e9-aeba-4554-a0a2-66cf8881dce3-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-m67qg\" (UID: \"f9c080e9-aeba-4554-a0a2-66cf8881dce3\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-m67qg" Sep 08 04:00:25.070858 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.069309 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b100a94e-b7da-4c28-9fd4-0842380dc1f1-metrics-client-ca\") pod \"node-exporter-wm4h4\" (UID: \"b100a94e-b7da-4c28-9fd4-0842380dc1f1\") " pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:25.070858 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.069386 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b100a94e-b7da-4c28-9fd4-0842380dc1f1-node-exporter-wtmp\") pod \"node-exporter-wm4h4\" (UID: \"b100a94e-b7da-4c28-9fd4-0842380dc1f1\") " pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:25.070858 ip-10-0-128-141 kubenswrapper[2579]: E0908 04:00:25.069648 2579 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Sep 08 04:00:25.070858 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.069674 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d6b54952-ee54-48f0-a1ce-c0255b19fe37-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-kfqkf\" (UID: \"d6b54952-ee54-48f0-a1ce-c0255b19fe37\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf" Sep 08 04:00:25.070858 ip-10-0-128-141 kubenswrapper[2579]: E0908 04:00:25.069724 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9c080e9-aeba-4554-a0a2-66cf8881dce3-openshift-state-metrics-tls podName:f9c080e9-aeba-4554-a0a2-66cf8881dce3 nodeName:}" failed. No retries permitted until 2026-09-08 04:00:25.569705605 +0000 UTC m=+285.190741553 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/f9c080e9-aeba-4554-a0a2-66cf8881dce3-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-m67qg" (UID: "f9c080e9-aeba-4554-a0a2-66cf8881dce3") : secret "openshift-state-metrics-tls" not found Sep 08 04:00:25.070858 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.069748 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b100a94e-b7da-4c28-9fd4-0842380dc1f1-sys\") pod \"node-exporter-wm4h4\" (UID: \"b100a94e-b7da-4c28-9fd4-0842380dc1f1\") " pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:25.070858 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.069911 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d6b54952-ee54-48f0-a1ce-c0255b19fe37-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-kfqkf\" (UID: \"d6b54952-ee54-48f0-a1ce-c0255b19fe37\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf" Sep 08 04:00:25.071740 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.071719 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b100a94e-b7da-4c28-9fd4-0842380dc1f1-node-exporter-tls\") pod \"node-exporter-wm4h4\" (UID: \"b100a94e-b7da-4c28-9fd4-0842380dc1f1\") " pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:25.071960 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.071937 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d6b54952-ee54-48f0-a1ce-c0255b19fe37-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-kfqkf\" (UID: \"d6b54952-ee54-48f0-a1ce-c0255b19fe37\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf" Sep 08 04:00:25.072032 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.071989 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6b54952-ee54-48f0-a1ce-c0255b19fe37-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-kfqkf\" (UID: \"d6b54952-ee54-48f0-a1ce-c0255b19fe37\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf" Sep 08 04:00:25.072439 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.072413 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b100a94e-b7da-4c28-9fd4-0842380dc1f1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wm4h4\" (UID: \"b100a94e-b7da-4c28-9fd4-0842380dc1f1\") " pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:25.072809 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.072785 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f9c080e9-aeba-4554-a0a2-66cf8881dce3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-m67qg\" (UID: \"f9c080e9-aeba-4554-a0a2-66cf8881dce3\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-m67qg" Sep 08 04:00:25.087012 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.086979 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54f4w\" (UniqueName: \"kubernetes.io/projected/f9c080e9-aeba-4554-a0a2-66cf8881dce3-kube-api-access-54f4w\") pod \"openshift-state-metrics-657fd6c54-m67qg\" (UID: \"f9c080e9-aeba-4554-a0a2-66cf8881dce3\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-m67qg" Sep 08 04:00:25.087627 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.087605 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4xzz\" (UniqueName: \"kubernetes.io/projected/d6b54952-ee54-48f0-a1ce-c0255b19fe37-kube-api-access-f4xzz\") pod \"kube-state-metrics-74cb57df7-kfqkf\" (UID: \"d6b54952-ee54-48f0-a1ce-c0255b19fe37\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf" Sep 08 04:00:25.088085 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.088063 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9m9p\" (UniqueName: \"kubernetes.io/projected/b100a94e-b7da-4c28-9fd4-0842380dc1f1-kube-api-access-b9m9p\") pod \"node-exporter-wm4h4\" (UID: \"b100a94e-b7da-4c28-9fd4-0842380dc1f1\") " pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:25.182788 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.182752 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf" Sep 08 04:00:25.234553 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.234503 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wm4h4" Sep 08 04:00:25.475548 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.475508 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf"] Sep 08 04:00:25.478779 ip-10-0-128-141 kubenswrapper[2579]: W0908 04:00:25.478750 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6b54952_ee54_48f0_a1ce_c0255b19fe37.slice/crio-d1c92eb552c36d878c9d7f385c0b2fafeeb0aab309e7d76b0e01ef9d1f8cd2e4 WatchSource:0}: Error finding container d1c92eb552c36d878c9d7f385c0b2fafeeb0aab309e7d76b0e01ef9d1f8cd2e4: Status 404 returned error can't find the container with id d1c92eb552c36d878c9d7f385c0b2fafeeb0aab309e7d76b0e01ef9d1f8cd2e4 Sep 08 04:00:25.572950 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.572905 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9c080e9-aeba-4554-a0a2-66cf8881dce3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-m67qg\" (UID: \"f9c080e9-aeba-4554-a0a2-66cf8881dce3\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-m67qg" Sep 08 04:00:25.573140 ip-10-0-128-141 kubenswrapper[2579]: E0908 04:00:25.573029 2579 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Sep 08 04:00:25.573140 ip-10-0-128-141 kubenswrapper[2579]: E0908 04:00:25.573095 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9c080e9-aeba-4554-a0a2-66cf8881dce3-openshift-state-metrics-tls podName:f9c080e9-aeba-4554-a0a2-66cf8881dce3 nodeName:}" failed. No retries permitted until 2026-09-08 04:00:26.573077925 +0000 UTC m=+286.194113873 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/f9c080e9-aeba-4554-a0a2-66cf8881dce3-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-m67qg" (UID: "f9c080e9-aeba-4554-a0a2-66cf8881dce3") : secret "openshift-state-metrics-tls" not found Sep 08 04:00:25.830216 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.830137 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 04:00:25.834810 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.834779 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.838655 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.838627 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 08 04:00:25.838923 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.838876 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 08 04:00:25.839017 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.838959 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 08 04:00:25.839263 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.839108 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 08 04:00:25.839263 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.839156 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 08 04:00:25.840952 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.840781 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 08 04:00:25.842288 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.842260 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-8dtdw\"" Sep 08 04:00:25.843320 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.843296 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 08 04:00:25.843762 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.843740 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 08 04:00:25.844000 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.843975 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 08 04:00:25.860996 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.860963 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 04:00:25.875327 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.875132 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.875327 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.875185 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.875327 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.875233 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.875327 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.875280 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b8b220a7-b8f6-4fab-9c88-233216c515a6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.875663 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.875362 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b8b220a7-b8f6-4fab-9c88-233216c515a6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.875663 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.875396 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.875663 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.875418 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh7b2\" (UniqueName: \"kubernetes.io/projected/b8b220a7-b8f6-4fab-9c88-233216c515a6-kube-api-access-xh7b2\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.875663 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.875471 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8b220a7-b8f6-4fab-9c88-233216c515a6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.875663 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.875514 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.875663 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.875537 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8b220a7-b8f6-4fab-9c88-233216c515a6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.875663 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.875568 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-config-volume\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.875663 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.875596 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-web-config\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.875663 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.875626 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b8b220a7-b8f6-4fab-9c88-233216c515a6-config-out\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.924596 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.924550 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wm4h4" event={"ID":"b100a94e-b7da-4c28-9fd4-0842380dc1f1","Type":"ContainerStarted","Data":"ecfdd65ea7b019b3fb4681d0430e28c320532cfa201434c5d32264a61a99081d"} Sep 08 04:00:25.926118 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.926079 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf" event={"ID":"d6b54952-ee54-48f0-a1ce-c0255b19fe37","Type":"ContainerStarted","Data":"d1c92eb552c36d878c9d7f385c0b2fafeeb0aab309e7d76b0e01ef9d1f8cd2e4"} Sep 08 04:00:25.977996 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.976270 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-web-config\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.977996 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.976325 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b8b220a7-b8f6-4fab-9c88-233216c515a6-config-out\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.977996 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.976354 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.977996 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.976387 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.977996 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.976428 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.977996 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.976496 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b8b220a7-b8f6-4fab-9c88-233216c515a6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.977996 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.976545 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b8b220a7-b8f6-4fab-9c88-233216c515a6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.977996 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.976575 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.977996 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.976597 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh7b2\" (UniqueName: \"kubernetes.io/projected/b8b220a7-b8f6-4fab-9c88-233216c515a6-kube-api-access-xh7b2\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.977996 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.976640 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8b220a7-b8f6-4fab-9c88-233216c515a6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.977996 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.976684 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.977996 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.976706 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8b220a7-b8f6-4fab-9c88-233216c515a6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.977996 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.976736 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-config-volume\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.977996 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.977636 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b8b220a7-b8f6-4fab-9c88-233216c515a6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.978856 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.978594 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8b220a7-b8f6-4fab-9c88-233216c515a6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.979337 ip-10-0-128-141 kubenswrapper[2579]: E0908 04:00:25.979048 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8b220a7-b8f6-4fab-9c88-233216c515a6-alertmanager-trusted-ca-bundle podName:b8b220a7-b8f6-4fab-9c88-233216c515a6 nodeName:}" failed. No retries permitted until 2026-09-08 04:00:26.479022793 +0000 UTC m=+286.100058741 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/b8b220a7-b8f6-4fab-9c88-233216c515a6-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "b8b220a7-b8f6-4fab-9c88-233216c515a6") : configmap references non-existent config key: ca-bundle.crt Sep 08 04:00:25.981845 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.981798 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b8b220a7-b8f6-4fab-9c88-233216c515a6-config-out\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.982987 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.982588 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-web-config\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.982987 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.982924 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b8b220a7-b8f6-4fab-9c88-233216c515a6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.983603 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.983580 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-config-volume\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.983801 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.983776 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.984300 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.984253 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.984515 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.984493 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.985532 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.985507 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.985627 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.985558 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:25.992609 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:25.992583 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh7b2\" (UniqueName: \"kubernetes.io/projected/b8b220a7-b8f6-4fab-9c88-233216c515a6-kube-api-access-xh7b2\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:26.481361 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:26.481331 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8b220a7-b8f6-4fab-9c88-233216c515a6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:26.482318 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:26.482284 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8b220a7-b8f6-4fab-9c88-233216c515a6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:26.582289 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:26.582248 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9c080e9-aeba-4554-a0a2-66cf8881dce3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-m67qg\" (UID: \"f9c080e9-aeba-4554-a0a2-66cf8881dce3\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-m67qg" Sep 08 04:00:26.585271 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:26.585218 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9c080e9-aeba-4554-a0a2-66cf8881dce3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-m67qg\" (UID: \"f9c080e9-aeba-4554-a0a2-66cf8881dce3\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-m67qg" Sep 08 04:00:26.693657 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:26.693611 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-m67qg" Sep 08 04:00:26.754387 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:26.754301 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:00:26.933233 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:26.932892 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wm4h4" event={"ID":"b100a94e-b7da-4c28-9fd4-0842380dc1f1","Type":"ContainerStarted","Data":"c0b99f4d6ebc8e5219afb412e560e8164e8172edd075d7543fd6eb0f989a953f"} Sep 08 04:00:27.705918 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:27.705831 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-m67qg"] Sep 08 04:00:27.721506 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:27.721388 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 04:00:27.728099 ip-10-0-128-141 kubenswrapper[2579]: W0908 04:00:27.728068 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8b220a7_b8f6_4fab_9c88_233216c515a6.slice/crio-631bef22c0a7c767459c25d7f6d876da57b8492c2a571772e6df2cac01a895ee WatchSource:0}: Error finding container 631bef22c0a7c767459c25d7f6d876da57b8492c2a571772e6df2cac01a895ee: Status 404 returned error can't find the container with id 631bef22c0a7c767459c25d7f6d876da57b8492c2a571772e6df2cac01a895ee Sep 08 04:00:27.941834 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:27.941733 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2jhc2" event={"ID":"77b984c3-52a4-442d-9356-f05a01aaa1bb","Type":"ContainerStarted","Data":"399c00e59a0a2087fa68b3592541bf69785c13bc56ea6352b43768307495a4b4"} Sep 08 04:00:27.941834 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:27.941801 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2jhc2" event={"ID":"77b984c3-52a4-442d-9356-f05a01aaa1bb","Type":"ContainerStarted","Data":"b24a4d0b0e3304bd04678a68cdb62c39408b32cdd3111195a676664a4b311b36"} Sep 08 04:00:27.941834 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:27.941832 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2jhc2" Sep 08 04:00:27.943330 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:27.943296 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b220a7-b8f6-4fab-9c88-233216c515a6","Type":"ContainerStarted","Data":"631bef22c0a7c767459c25d7f6d876da57b8492c2a571772e6df2cac01a895ee"} Sep 08 04:00:27.945019 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:27.944992 2579 generic.go:356] "Generic (PLEG): container finished" podID="b100a94e-b7da-4c28-9fd4-0842380dc1f1" containerID="c0b99f4d6ebc8e5219afb412e560e8164e8172edd075d7543fd6eb0f989a953f" exitCode=0 Sep 08 04:00:27.945134 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:27.945070 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wm4h4" event={"ID":"b100a94e-b7da-4c28-9fd4-0842380dc1f1","Type":"ContainerDied","Data":"c0b99f4d6ebc8e5219afb412e560e8164e8172edd075d7543fd6eb0f989a953f"} Sep 08 04:00:27.947662 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:27.947630 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf" event={"ID":"d6b54952-ee54-48f0-a1ce-c0255b19fe37","Type":"ContainerStarted","Data":"d73258652662f60f405dd72005088d422a21adea6be229921b9f04bd89c557f4"} Sep 08 04:00:27.947766 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:27.947686 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf" event={"ID":"d6b54952-ee54-48f0-a1ce-c0255b19fe37","Type":"ContainerStarted","Data":"a71ffd08eb477dcf339cd25b08b4b0e78bd5d5e880855134f76db6f0e166cb7f"} Sep 08 04:00:27.947766 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:27.947722 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf" event={"ID":"d6b54952-ee54-48f0-a1ce-c0255b19fe37","Type":"ContainerStarted","Data":"59e2018f4902a1a3db0a683c7c3f5190e7377810bb9da97941d0aeef79826ffb"} Sep 08 04:00:27.949740 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:27.949720 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-m67qg" event={"ID":"f9c080e9-aeba-4554-a0a2-66cf8881dce3","Type":"ContainerStarted","Data":"bc2cc47722457db674b28edfb7e95ab8d790dbdbc7e93a84ca17bf7744d1507f"} Sep 08 04:00:27.949819 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:27.949767 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-m67qg" event={"ID":"f9c080e9-aeba-4554-a0a2-66cf8881dce3","Type":"ContainerStarted","Data":"5d6c1c11c6c9fc7076f874eedf3cdb5eadd587a9cc272cdcb0719b928b501f9c"} Sep 08 04:00:27.949819 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:27.949786 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-m67qg" event={"ID":"f9c080e9-aeba-4554-a0a2-66cf8881dce3","Type":"ContainerStarted","Data":"46da40e579cff32dfec4bad167ed81ff95b937664083b8723d2233be972dbd52"} Sep 08 04:00:27.989539 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:27.989453 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2jhc2" podStartSLOduration=251.388387572 podStartE2EDuration="4m13.989430806s" totalImagesPullingTime="2.601043234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 03:56:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:00:27.962340647 +0000 UTC m=+287.583376613" watchObservedRunningTime="2026-09-08 04:00:27.989430806 +0000 UTC m=+287.610466773" Sep 08 04:00:27.989909 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:27.989876 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-kfqkf" podStartSLOduration=2.049210638 podStartE2EDuration="3.989868329s" totalImagesPullingTime="1.940657691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 04:00:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:00:27.988700735 +0000 UTC m=+287.609736701" watchObservedRunningTime="2026-09-08 04:00:27.989868329 +0000 UTC m=+287.610904334" Sep 08 04:00:28.956644 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:28.956600 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wm4h4" event={"ID":"b100a94e-b7da-4c28-9fd4-0842380dc1f1","Type":"ContainerStarted","Data":"4c401822067dec7f92a96c8edfdf34b67e49d0cd59ffc46d34e0a5df679a57ba"} Sep 08 04:00:28.957115 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:28.956660 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wm4h4" event={"ID":"b100a94e-b7da-4c28-9fd4-0842380dc1f1","Type":"ContainerStarted","Data":"3d1d8e8c89636aab0b1a5e09f2094793493ac9d9ca8d85973a835c589694d5d9"} Sep 08 04:00:28.992004 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:28.991951 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wm4h4" podStartSLOduration=3.700154058 podStartE2EDuration="4.99193473s" totalImagesPullingTime="1.219226258s" totalInitContainerRuntime="72.554414ms" isStatefulPod=true podCreationTimestamp="2026-09-08 04:00:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:00:28.991109116 +0000 UTC m=+288.612145083" watchObservedRunningTime="2026-09-08 04:00:28.99193473 +0000 UTC m=+288.612970736" Sep 08 04:00:29.328630 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:29.328546 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-76j8x"] Sep 08 04:00:29.331933 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:29.331917 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-76j8x" Sep 08 04:00:29.335895 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:29.335869 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-pcrhc\"" Sep 08 04:00:29.336039 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:29.335972 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 08 04:00:29.413144 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:29.413101 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/927871d3-f692-417d-8cd1-024e6bd0b603-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-76j8x\" (UID: \"927871d3-f692-417d-8cd1-024e6bd0b603\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-76j8x" Sep 08 04:00:29.413330 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:29.413189 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpgdx\" (UniqueName: \"kubernetes.io/projected/927871d3-f692-417d-8cd1-024e6bd0b603-kube-api-access-jpgdx\") pod \"monitoring-plugin-85bbbbbf4-76j8x\" (UID: \"927871d3-f692-417d-8cd1-024e6bd0b603\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-76j8x" Sep 08 04:00:29.416111 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:29.416054 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-76j8x"] Sep 08 04:00:29.513866 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:29.513829 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpgdx\" (UniqueName: \"kubernetes.io/projected/927871d3-f692-417d-8cd1-024e6bd0b603-kube-api-access-jpgdx\") pod \"monitoring-plugin-85bbbbbf4-76j8x\" (UID: \"927871d3-f692-417d-8cd1-024e6bd0b603\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-76j8x" Sep 08 04:00:29.514099 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:29.513931 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/927871d3-f692-417d-8cd1-024e6bd0b603-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-76j8x\" (UID: \"927871d3-f692-417d-8cd1-024e6bd0b603\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-76j8x" Sep 08 04:00:29.516732 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:29.516707 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/927871d3-f692-417d-8cd1-024e6bd0b603-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-76j8x\" (UID: \"927871d3-f692-417d-8cd1-024e6bd0b603\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-76j8x" Sep 08 04:00:29.527743 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:29.527708 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpgdx\" (UniqueName: \"kubernetes.io/projected/927871d3-f692-417d-8cd1-024e6bd0b603-kube-api-access-jpgdx\") pod \"monitoring-plugin-85bbbbbf4-76j8x\" (UID: \"927871d3-f692-417d-8cd1-024e6bd0b603\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-76j8x" Sep 08 04:00:29.640837 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:29.640800 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-76j8x" Sep 08 04:00:30.142843 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:30.142804 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-76j8x"] Sep 08 04:00:30.148354 ip-10-0-128-141 kubenswrapper[2579]: W0908 04:00:30.148328 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod927871d3_f692_417d_8cd1_024e6bd0b603.slice/crio-d4830b1a984abc9a9e6ca61c41381661fbc343b6e4f7aeed507daaa9ec75b72f WatchSource:0}: Error finding container d4830b1a984abc9a9e6ca61c41381661fbc343b6e4f7aeed507daaa9ec75b72f: Status 404 returned error can't find the container with id d4830b1a984abc9a9e6ca61c41381661fbc343b6e4f7aeed507daaa9ec75b72f Sep 08 04:00:30.966471 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:30.966421 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-76j8x" event={"ID":"927871d3-f692-417d-8cd1-024e6bd0b603","Type":"ContainerStarted","Data":"d4830b1a984abc9a9e6ca61c41381661fbc343b6e4f7aeed507daaa9ec75b72f"} Sep 08 04:00:30.968586 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:30.968559 2579 generic.go:356] "Generic (PLEG): container finished" podID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerID="e3155849cf7612ab42c3e98c4329db4eaa17a12a7aaeeecdd26a834f503291d3" exitCode=0 Sep 08 04:00:30.968693 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:30.968650 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b220a7-b8f6-4fab-9c88-233216c515a6","Type":"ContainerDied","Data":"e3155849cf7612ab42c3e98c4329db4eaa17a12a7aaeeecdd26a834f503291d3"} Sep 08 04:00:31.148303 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.148265 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 08 04:00:31.155157 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.155118 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.161972 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.161917 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 08 04:00:31.161972 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.161958 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 08 04:00:31.162247 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.162097 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 08 04:00:31.162389 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.162374 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 08 04:00:31.162481 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.162385 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2pf7jaspgjh6o\"" Sep 08 04:00:31.162898 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.162876 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 08 04:00:31.166575 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.166548 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 08 04:00:31.166904 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.166886 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 08 04:00:31.167010 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.166995 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 08 04:00:31.167284 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.167270 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 08 04:00:31.167357 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.167304 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 08 04:00:31.167663 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.167641 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 08 04:00:31.167774 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.167672 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 08 04:00:31.168058 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.168043 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 08 04:00:31.168259 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.168245 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-mb5qw\"" Sep 08 04:00:31.172999 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.172980 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 08 04:00:31.173114 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.172981 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 08 04:00:31.229810 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.229737 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/86706a88-1623-4e82-8360-387ba40739e2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.229810 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.229792 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.230002 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.229848 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/86706a88-1623-4e82-8360-387ba40739e2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.230002 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.229878 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.230002 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.229905 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd957\" (UniqueName: \"kubernetes.io/projected/86706a88-1623-4e82-8360-387ba40739e2-kube-api-access-zd957\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.230002 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.229924 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.230002 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.229941 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.230002 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.229967 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.230244 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.230042 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.230244 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.230091 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.230244 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.230123 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.230244 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.230140 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.230244 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.230158 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.230244 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.230235 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/86706a88-1623-4e82-8360-387ba40739e2-config-out\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.230562 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.230264 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.230562 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.230324 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.230562 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.230380 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-config\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.230562 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.230403 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.230562 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.230428 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.230562 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.230472 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-web-config\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.260023 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.259987 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 08 04:00:31.331897 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.331860 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.332070 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.331916 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/86706a88-1623-4e82-8360-387ba40739e2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.332070 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.331943 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.332070 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.331970 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd957\" (UniqueName: \"kubernetes.io/projected/86706a88-1623-4e82-8360-387ba40739e2-kube-api-access-zd957\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.332070 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.331996 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.332070 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.332022 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.332070 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.332048 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.332394 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.332087 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.332394 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.332141 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.332394 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.332165 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.332394 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.332190 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.332394 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.332232 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.332394 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.332272 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/86706a88-1623-4e82-8360-387ba40739e2-config-out\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.332394 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.332309 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.332394 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.332338 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.332394 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.332382 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-config\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.332834 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.332404 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.332834 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.332429 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.332834 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.332476 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-web-config\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.332834 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.332504 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/86706a88-1623-4e82-8360-387ba40739e2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.333044 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.332878 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.333145 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.333120 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.333447 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.333421 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.333582 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.333541 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.334851 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.334442 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.334851 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.334646 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.334851 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.334754 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/86706a88-1623-4e82-8360-387ba40739e2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.337906 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.337881 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-config\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.337906 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.337901 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.338089 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.337989 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.338277 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.338256 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/86706a88-1623-4e82-8360-387ba40739e2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.338424 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.338256 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.338662 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.338272 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.338662 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.338450 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.338662 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.338578 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.338835 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.338661 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.338835 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.338692 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.339531 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.339507 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-web-config\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.340109 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.340081 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/86706a88-1623-4e82-8360-387ba40739e2-config-out\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.346663 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.346624 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd957\" (UniqueName: \"kubernetes.io/projected/86706a88-1623-4e82-8360-387ba40739e2-kube-api-access-zd957\") pod \"prometheus-k8s-0\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.466874 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.466831 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:31.845379 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.845293 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 08 04:00:31.854759 ip-10-0-128-141 kubenswrapper[2579]: W0908 04:00:31.854733 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86706a88_1623_4e82_8360_387ba40739e2.slice/crio-7c7d937a7dd746207777183468d23f486d6bf52cd66179ff0d5c12b32f5447b8 WatchSource:0}: Error finding container 7c7d937a7dd746207777183468d23f486d6bf52cd66179ff0d5c12b32f5447b8: Status 404 returned error can't find the container with id 7c7d937a7dd746207777183468d23f486d6bf52cd66179ff0d5c12b32f5447b8 Sep 08 04:00:31.974420 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.974382 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-m67qg" event={"ID":"f9c080e9-aeba-4554-a0a2-66cf8881dce3","Type":"ContainerStarted","Data":"989ee3b66d09f78e145a49d42258bdc21c7fa67d3602c5ffd6af433b685eedcc"} Sep 08 04:00:31.975965 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.975938 2579 generic.go:356] "Generic (PLEG): container finished" podID="86706a88-1623-4e82-8360-387ba40739e2" containerID="4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1" exitCode=0 Sep 08 04:00:31.976095 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.975979 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"86706a88-1623-4e82-8360-387ba40739e2","Type":"ContainerDied","Data":"4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1"} Sep 08 04:00:31.976095 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.976036 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"86706a88-1623-4e82-8360-387ba40739e2","Type":"ContainerStarted","Data":"7c7d937a7dd746207777183468d23f486d6bf52cd66179ff0d5c12b32f5447b8"} Sep 08 04:00:31.977623 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.977554 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-76j8x" event={"ID":"927871d3-f692-417d-8cd1-024e6bd0b603","Type":"ContainerStarted","Data":"6e93c89636ff8111292d316d5ed9fc148ea6f8d421b4982a9bb5d55219c88967"} Sep 08 04:00:31.977766 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.977751 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-76j8x" Sep 08 04:00:31.982979 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:31.982942 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-76j8x" Sep 08 04:00:32.079949 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:32.079891 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-m67qg" podStartSLOduration=4.982008915 podStartE2EDuration="8.079872224s" totalImagesPullingTime="3.097863309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 04:00:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:00:32.009421368 +0000 UTC m=+291.630457334" watchObservedRunningTime="2026-09-08 04:00:32.079872224 +0000 UTC m=+291.700908192" Sep 08 04:00:32.124989 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:32.124928 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-76j8x" podStartSLOduration=1.694614456 podStartE2EDuration="3.124906634s" totalImagesPullingTime="1.430292178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 04:00:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:00:32.124598482 +0000 UTC m=+291.745634449" watchObservedRunningTime="2026-09-08 04:00:32.124906634 +0000 UTC m=+291.745942609" Sep 08 04:00:33.871692 ip-10-0-128-141 kubenswrapper[2579]: E0908 04:00:33.871660 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-r4xfl" podUID="b7239679-8052-415f-ae9a-1b20d61d1223" Sep 08 04:00:33.990816 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:33.990776 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b220a7-b8f6-4fab-9c88-233216c515a6","Type":"ContainerStarted","Data":"8d7653295b5d80f5b94a4d8b4db989709304a9b8721f6b55d76840bdf3373ae2"} Sep 08 04:00:33.990990 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:33.990866 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b220a7-b8f6-4fab-9c88-233216c515a6","Type":"ContainerStarted","Data":"da6fa564c2098ffc8c9ee2ce6296faea7ff4c51cba86688a5d512c921b8fdc44"} Sep 08 04:00:33.990990 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:33.990914 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b220a7-b8f6-4fab-9c88-233216c515a6","Type":"ContainerStarted","Data":"7f0764da8106eeaff3b22814a67c8b1e42f8541a384639eb0b73b260ba067d88"} Sep 08 04:00:33.990990 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:33.990954 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b220a7-b8f6-4fab-9c88-233216c515a6","Type":"ContainerStarted","Data":"32c66d83ba30958fd726242d5f772f0e7fb0d62c47fa41f61a95403fa7a95202"} Sep 08 04:00:33.991151 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:33.990994 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b220a7-b8f6-4fab-9c88-233216c515a6","Type":"ContainerStarted","Data":"e139cfe2ad47184d20b72b0cb4c3a85954ff5d5316200e97d2b302c7b2b80a57"} Sep 08 04:00:36.002782 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:36.002742 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b220a7-b8f6-4fab-9c88-233216c515a6","Type":"ContainerStarted","Data":"485eaf064038f2fb2bf4159a13233747cbc3a6f25a3a0d66e986ae783f7432f1"} Sep 08 04:00:36.072766 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:36.072693 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.760662211 podStartE2EDuration="11.072678244s" totalImagesPullingTime="6.296739561s" totalInitContainerRuntime="15.276472ms" isStatefulPod=false podCreationTimestamp="2026-09-08 04:00:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:00:36.072338652 +0000 UTC m=+295.693374621" watchObservedRunningTime="2026-09-08 04:00:36.072678244 +0000 UTC m=+295.693714210" Sep 08 04:00:37.008495 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:37.008438 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"86706a88-1623-4e82-8360-387ba40739e2","Type":"ContainerStarted","Data":"3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a"} Sep 08 04:00:37.008857 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:37.008534 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"86706a88-1623-4e82-8360-387ba40739e2","Type":"ContainerStarted","Data":"3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7"} Sep 08 04:00:37.959429 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:37.959395 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2jhc2" Sep 08 04:00:40.024095 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:40.024060 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"86706a88-1623-4e82-8360-387ba40739e2","Type":"ContainerStarted","Data":"e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3"} Sep 08 04:00:40.024452 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:40.024141 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"86706a88-1623-4e82-8360-387ba40739e2","Type":"ContainerStarted","Data":"0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94"} Sep 08 04:00:40.024452 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:40.024187 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"86706a88-1623-4e82-8360-387ba40739e2","Type":"ContainerStarted","Data":"6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844"} Sep 08 04:00:40.024452 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:40.024223 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"86706a88-1623-4e82-8360-387ba40739e2","Type":"ContainerStarted","Data":"6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24"} Sep 08 04:00:40.068840 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:40.068777 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.731957815 podStartE2EDuration="9.068754148s" totalImagesPullingTime="7.317394405s" totalInitContainerRuntime="19.401928ms" isStatefulPod=false podCreationTimestamp="2026-09-08 04:00:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:00:40.066444233 +0000 UTC m=+299.687480202" watchObservedRunningTime="2026-09-08 04:00:40.068754148 +0000 UTC m=+299.689790116" Sep 08 04:00:40.784206 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:40.784174 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tg7hb_45ae9afd-2dea-43b8-aa31-169b11e60fd2/ovn-acl-logging/0.log" Sep 08 04:00:40.784430 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:40.784412 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tg7hb_45ae9afd-2dea-43b8-aa31-169b11e60fd2/ovn-acl-logging/0.log" Sep 08 04:00:40.789676 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:40.789656 2579 kubelet.go:1648] "Image garbage collection succeeded" Sep 08 04:00:41.467302 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:41.467271 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:00:47.870206 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:47.870173 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 04:00:48.057820 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:48.057785 2579 generic.go:356] "Generic (PLEG): container finished" podID="3cd2c98f-1800-40fa-a229-f835cae8915a" containerID="8aab9d1748ea91e2cc168cb1b17b6becb0c286a4763acdc48543001733cf3d26" exitCode=0 Sep 08 04:00:48.057984 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:48.057838 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-79hzk" event={"ID":"3cd2c98f-1800-40fa-a229-f835cae8915a","Type":"ContainerDied","Data":"8aab9d1748ea91e2cc168cb1b17b6becb0c286a4763acdc48543001733cf3d26"} Sep 08 04:00:48.058148 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:48.058133 2579 scope.go:117] "RemoveContainer" containerID="8aab9d1748ea91e2cc168cb1b17b6becb0c286a4763acdc48543001733cf3d26" Sep 08 04:00:49.064190 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:49.064084 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-79hzk" event={"ID":"3cd2c98f-1800-40fa-a229-f835cae8915a","Type":"ContainerStarted","Data":"ba46d595b0bc47a9edcc4e494f4dc1ea5fc96c946dcce0d4ec10250a7c247426"} Sep 08 04:00:49.065711 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:49.065680 2579 generic.go:356] "Generic (PLEG): container finished" podID="ab64adad-879e-4ec6-8425-86a70f48615f" containerID="6e85049a5ea2f69a7b79ebba57f290dab43d3df4165038c4b8dee52b77bcb432" exitCode=0 Sep 08 04:00:49.065813 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:49.065726 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-t55cp" event={"ID":"ab64adad-879e-4ec6-8425-86a70f48615f","Type":"ContainerDied","Data":"6e85049a5ea2f69a7b79ebba57f290dab43d3df4165038c4b8dee52b77bcb432"} Sep 08 04:00:49.066026 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:49.066013 2579 scope.go:117] "RemoveContainer" containerID="6e85049a5ea2f69a7b79ebba57f290dab43d3df4165038c4b8dee52b77bcb432" Sep 08 04:00:50.071110 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:00:50.071071 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-t55cp" event={"ID":"ab64adad-879e-4ec6-8425-86a70f48615f","Type":"ContainerStarted","Data":"5622b9117758d16a75d356859271bb8164fe2d47adb319ae48cdc4a144d0f25d"} Sep 08 04:01:31.467094 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:31.467049 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:31.483066 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:31.483038 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:32.261566 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:32.261536 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:45.185767 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:45.185686 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 04:01:45.186231 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:45.185993 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerName="alertmanager" containerID="cri-o://e139cfe2ad47184d20b72b0cb4c3a85954ff5d5316200e97d2b302c7b2b80a57" gracePeriod=120 Sep 08 04:01:45.186231 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:45.186051 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerName="kube-rbac-proxy-metric" containerID="cri-o://8d7653295b5d80f5b94a4d8b4db989709304a9b8721f6b55d76840bdf3373ae2" gracePeriod=120 Sep 08 04:01:45.186231 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:45.186100 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerName="config-reloader" containerID="cri-o://32c66d83ba30958fd726242d5f772f0e7fb0d62c47fa41f61a95403fa7a95202" gracePeriod=120 Sep 08 04:01:45.186697 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:45.186650 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerName="prom-label-proxy" containerID="cri-o://485eaf064038f2fb2bf4159a13233747cbc3a6f25a3a0d66e986ae783f7432f1" gracePeriod=120 Sep 08 04:01:45.186819 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:45.186689 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerName="kube-rbac-proxy" containerID="cri-o://da6fa564c2098ffc8c9ee2ce6296faea7ff4c51cba86688a5d512c921b8fdc44" gracePeriod=120 Sep 08 04:01:45.187352 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:45.186963 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerName="kube-rbac-proxy-web" containerID="cri-o://7f0764da8106eeaff3b22814a67c8b1e42f8541a384639eb0b73b260ba067d88" gracePeriod=120 Sep 08 04:01:46.309717 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.309682 2579 generic.go:356] "Generic (PLEG): container finished" podID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerID="485eaf064038f2fb2bf4159a13233747cbc3a6f25a3a0d66e986ae783f7432f1" exitCode=0 Sep 08 04:01:46.309717 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.309710 2579 generic.go:356] "Generic (PLEG): container finished" podID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerID="8d7653295b5d80f5b94a4d8b4db989709304a9b8721f6b55d76840bdf3373ae2" exitCode=0 Sep 08 04:01:46.309717 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.309719 2579 generic.go:356] "Generic (PLEG): container finished" podID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerID="da6fa564c2098ffc8c9ee2ce6296faea7ff4c51cba86688a5d512c921b8fdc44" exitCode=0 Sep 08 04:01:46.309717 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.309727 2579 generic.go:356] "Generic (PLEG): container finished" podID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerID="7f0764da8106eeaff3b22814a67c8b1e42f8541a384639eb0b73b260ba067d88" exitCode=0 Sep 08 04:01:46.310289 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.309733 2579 generic.go:356] "Generic (PLEG): container finished" podID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerID="32c66d83ba30958fd726242d5f772f0e7fb0d62c47fa41f61a95403fa7a95202" exitCode=0 Sep 08 04:01:46.310289 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.309726 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b220a7-b8f6-4fab-9c88-233216c515a6","Type":"ContainerDied","Data":"485eaf064038f2fb2bf4159a13233747cbc3a6f25a3a0d66e986ae783f7432f1"} Sep 08 04:01:46.310289 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.309738 2579 generic.go:356] "Generic (PLEG): container finished" podID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerID="e139cfe2ad47184d20b72b0cb4c3a85954ff5d5316200e97d2b302c7b2b80a57" exitCode=0 Sep 08 04:01:46.310289 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.309828 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b220a7-b8f6-4fab-9c88-233216c515a6","Type":"ContainerDied","Data":"8d7653295b5d80f5b94a4d8b4db989709304a9b8721f6b55d76840bdf3373ae2"} Sep 08 04:01:46.310289 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.309863 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b220a7-b8f6-4fab-9c88-233216c515a6","Type":"ContainerDied","Data":"da6fa564c2098ffc8c9ee2ce6296faea7ff4c51cba86688a5d512c921b8fdc44"} Sep 08 04:01:46.310289 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.309890 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b220a7-b8f6-4fab-9c88-233216c515a6","Type":"ContainerDied","Data":"7f0764da8106eeaff3b22814a67c8b1e42f8541a384639eb0b73b260ba067d88"} Sep 08 04:01:46.310289 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.309913 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b220a7-b8f6-4fab-9c88-233216c515a6","Type":"ContainerDied","Data":"32c66d83ba30958fd726242d5f772f0e7fb0d62c47fa41f61a95403fa7a95202"} Sep 08 04:01:46.310289 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.309941 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b220a7-b8f6-4fab-9c88-233216c515a6","Type":"ContainerDied","Data":"e139cfe2ad47184d20b72b0cb4c3a85954ff5d5316200e97d2b302c7b2b80a57"} Sep 08 04:01:46.429646 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.429618 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:01:46.504973 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.504940 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-secret-alertmanager-kube-rbac-proxy-web\") pod \"b8b220a7-b8f6-4fab-9c88-233216c515a6\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " Sep 08 04:01:46.505183 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.504984 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b8b220a7-b8f6-4fab-9c88-233216c515a6-config-out\") pod \"b8b220a7-b8f6-4fab-9c88-233216c515a6\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " Sep 08 04:01:46.505183 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.505016 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b8b220a7-b8f6-4fab-9c88-233216c515a6-tls-assets\") pod \"b8b220a7-b8f6-4fab-9c88-233216c515a6\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " Sep 08 04:01:46.505183 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.505037 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8b220a7-b8f6-4fab-9c88-233216c515a6-metrics-client-ca\") pod \"b8b220a7-b8f6-4fab-9c88-233216c515a6\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " Sep 08 04:01:46.505183 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.505069 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-secret-alertmanager-main-tls\") pod \"b8b220a7-b8f6-4fab-9c88-233216c515a6\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " Sep 08 04:01:46.505183 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.505092 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-secret-alertmanager-kube-rbac-proxy\") pod \"b8b220a7-b8f6-4fab-9c88-233216c515a6\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " Sep 08 04:01:46.505183 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.505128 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8b220a7-b8f6-4fab-9c88-233216c515a6-alertmanager-trusted-ca-bundle\") pod \"b8b220a7-b8f6-4fab-9c88-233216c515a6\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " Sep 08 04:01:46.505183 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.505157 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-config-volume\") pod \"b8b220a7-b8f6-4fab-9c88-233216c515a6\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " Sep 08 04:01:46.505593 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.505192 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"b8b220a7-b8f6-4fab-9c88-233216c515a6\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " Sep 08 04:01:46.505593 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.505224 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-web-config\") pod \"b8b220a7-b8f6-4fab-9c88-233216c515a6\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " Sep 08 04:01:46.505593 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.505266 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xh7b2\" (UniqueName: \"kubernetes.io/projected/b8b220a7-b8f6-4fab-9c88-233216c515a6-kube-api-access-xh7b2\") pod \"b8b220a7-b8f6-4fab-9c88-233216c515a6\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " Sep 08 04:01:46.505593 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.505294 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-cluster-tls-config\") pod \"b8b220a7-b8f6-4fab-9c88-233216c515a6\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " Sep 08 04:01:46.505593 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.505318 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b8b220a7-b8f6-4fab-9c88-233216c515a6-alertmanager-main-db\") pod \"b8b220a7-b8f6-4fab-9c88-233216c515a6\" (UID: \"b8b220a7-b8f6-4fab-9c88-233216c515a6\") " Sep 08 04:01:46.505593 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.505579 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8b220a7-b8f6-4fab-9c88-233216c515a6-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "b8b220a7-b8f6-4fab-9c88-233216c515a6" (UID: "b8b220a7-b8f6-4fab-9c88-233216c515a6"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 04:01:46.505882 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.505731 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8b220a7-b8f6-4fab-9c88-233216c515a6-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "b8b220a7-b8f6-4fab-9c88-233216c515a6" (UID: "b8b220a7-b8f6-4fab-9c88-233216c515a6"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 04:01:46.505943 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.505874 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8b220a7-b8f6-4fab-9c88-233216c515a6-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "b8b220a7-b8f6-4fab-9c88-233216c515a6" (UID: "b8b220a7-b8f6-4fab-9c88-233216c515a6"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 04:01:46.508395 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.508361 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "b8b220a7-b8f6-4fab-9c88-233216c515a6" (UID: "b8b220a7-b8f6-4fab-9c88-233216c515a6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 04:01:46.508978 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.508949 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "b8b220a7-b8f6-4fab-9c88-233216c515a6" (UID: "b8b220a7-b8f6-4fab-9c88-233216c515a6"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 04:01:46.509080 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.508980 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8b220a7-b8f6-4fab-9c88-233216c515a6-kube-api-access-xh7b2" (OuterVolumeSpecName: "kube-api-access-xh7b2") pod "b8b220a7-b8f6-4fab-9c88-233216c515a6" (UID: "b8b220a7-b8f6-4fab-9c88-233216c515a6"). InnerVolumeSpecName "kube-api-access-xh7b2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 04:01:46.509233 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.509206 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8b220a7-b8f6-4fab-9c88-233216c515a6-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "b8b220a7-b8f6-4fab-9c88-233216c515a6" (UID: "b8b220a7-b8f6-4fab-9c88-233216c515a6"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 04:01:46.509610 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.509575 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "b8b220a7-b8f6-4fab-9c88-233216c515a6" (UID: "b8b220a7-b8f6-4fab-9c88-233216c515a6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 04:01:46.509801 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.509754 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "b8b220a7-b8f6-4fab-9c88-233216c515a6" (UID: "b8b220a7-b8f6-4fab-9c88-233216c515a6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 04:01:46.509897 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.509837 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8b220a7-b8f6-4fab-9c88-233216c515a6-config-out" (OuterVolumeSpecName: "config-out") pod "b8b220a7-b8f6-4fab-9c88-233216c515a6" (UID: "b8b220a7-b8f6-4fab-9c88-233216c515a6"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 04:01:46.510280 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.510255 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-config-volume" (OuterVolumeSpecName: "config-volume") pod "b8b220a7-b8f6-4fab-9c88-233216c515a6" (UID: "b8b220a7-b8f6-4fab-9c88-233216c515a6"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 04:01:46.514148 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.514123 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "b8b220a7-b8f6-4fab-9c88-233216c515a6" (UID: "b8b220a7-b8f6-4fab-9c88-233216c515a6"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 04:01:46.520930 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.520898 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-web-config" (OuterVolumeSpecName: "web-config") pod "b8b220a7-b8f6-4fab-9c88-233216c515a6" (UID: "b8b220a7-b8f6-4fab-9c88-233216c515a6"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 04:01:46.606721 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.606680 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xh7b2\" (UniqueName: \"kubernetes.io/projected/b8b220a7-b8f6-4fab-9c88-233216c515a6-kube-api-access-xh7b2\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:46.606721 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.606713 2579 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-cluster-tls-config\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:46.606721 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.606723 2579 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b8b220a7-b8f6-4fab-9c88-233216c515a6-alertmanager-main-db\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:46.606721 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.606732 2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:46.606988 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.606744 2579 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b8b220a7-b8f6-4fab-9c88-233216c515a6-config-out\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:46.606988 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.606752 2579 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b8b220a7-b8f6-4fab-9c88-233216c515a6-tls-assets\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:46.606988 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.606760 2579 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8b220a7-b8f6-4fab-9c88-233216c515a6-metrics-client-ca\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:46.606988 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.606768 2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-secret-alertmanager-main-tls\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:46.606988 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.606776 2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:46.606988 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.606784 2579 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8b220a7-b8f6-4fab-9c88-233216c515a6-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:46.606988 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.606792 2579 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-config-volume\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:46.606988 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.606800 2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:46.606988 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:46.606809 2579 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b8b220a7-b8f6-4fab-9c88-233216c515a6-web-config\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:47.320549 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:47.320513 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b220a7-b8f6-4fab-9c88-233216c515a6","Type":"ContainerDied","Data":"631bef22c0a7c767459c25d7f6d876da57b8492c2a571772e6df2cac01a895ee"} Sep 08 04:01:47.320549 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:47.320544 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 04:01:47.321014 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:47.320610 2579 scope.go:117] "RemoveContainer" containerID="485eaf064038f2fb2bf4159a13233747cbc3a6f25a3a0d66e986ae783f7432f1" Sep 08 04:01:47.329202 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:47.329184 2579 scope.go:117] "RemoveContainer" containerID="8d7653295b5d80f5b94a4d8b4db989709304a9b8721f6b55d76840bdf3373ae2" Sep 08 04:01:47.337722 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:47.337696 2579 scope.go:117] "RemoveContainer" containerID="da6fa564c2098ffc8c9ee2ce6296faea7ff4c51cba86688a5d512c921b8fdc44" Sep 08 04:01:47.345752 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:47.345722 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 04:01:47.346114 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:47.346094 2579 scope.go:117] "RemoveContainer" containerID="7f0764da8106eeaff3b22814a67c8b1e42f8541a384639eb0b73b260ba067d88" Sep 08 04:01:47.351686 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:47.351659 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 04:01:47.354305 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:47.354284 2579 scope.go:117] "RemoveContainer" containerID="32c66d83ba30958fd726242d5f772f0e7fb0d62c47fa41f61a95403fa7a95202" Sep 08 04:01:47.362217 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:47.362198 2579 scope.go:117] "RemoveContainer" containerID="e139cfe2ad47184d20b72b0cb4c3a85954ff5d5316200e97d2b302c7b2b80a57" Sep 08 04:01:47.370185 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:47.370159 2579 scope.go:117] "RemoveContainer" containerID="e3155849cf7612ab42c3e98c4329db4eaa17a12a7aaeeecdd26a834f503291d3" Sep 08 04:01:48.875151 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:48.875121 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8b220a7-b8f6-4fab-9c88-233216c515a6" path="/var/lib/kubelet/pods/b8b220a7-b8f6-4fab-9c88-233216c515a6/volumes" Sep 08 04:01:49.231026 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.230990 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7496599d74-sjlm8"] Sep 08 04:01:49.231296 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.231285 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerName="config-reloader" Sep 08 04:01:49.231342 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.231299 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerName="config-reloader" Sep 08 04:01:49.231342 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.231312 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerName="kube-rbac-proxy" Sep 08 04:01:49.231342 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.231317 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerName="kube-rbac-proxy" Sep 08 04:01:49.231342 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.231330 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerName="kube-rbac-proxy-metric" Sep 08 04:01:49.231342 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.231336 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerName="kube-rbac-proxy-metric" Sep 08 04:01:49.231508 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.231345 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerName="kube-rbac-proxy-web" Sep 08 04:01:49.231508 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.231351 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerName="kube-rbac-proxy-web" Sep 08 04:01:49.231508 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.231359 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerName="init-config-reloader" Sep 08 04:01:49.231508 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.231363 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerName="init-config-reloader" Sep 08 04:01:49.231508 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.231369 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerName="prom-label-proxy" Sep 08 04:01:49.231508 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.231374 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerName="prom-label-proxy" Sep 08 04:01:49.231508 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.231380 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerName="alertmanager" Sep 08 04:01:49.231508 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.231385 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerName="alertmanager" Sep 08 04:01:49.231508 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.231429 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerName="prom-label-proxy" Sep 08 04:01:49.231508 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.231436 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerName="kube-rbac-proxy-metric" Sep 08 04:01:49.231508 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.231441 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerName="kube-rbac-proxy-web" Sep 08 04:01:49.231508 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.231446 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerName="config-reloader" Sep 08 04:01:49.231508 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.231454 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerName="alertmanager" Sep 08 04:01:49.231508 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.231473 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8b220a7-b8f6-4fab-9c88-233216c515a6" containerName="kube-rbac-proxy" Sep 08 04:01:49.235178 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.235156 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" Sep 08 04:01:49.239262 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.239233 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 08 04:01:49.239434 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.239283 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 08 04:01:49.240344 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.240321 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 08 04:01:49.240481 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.240328 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-zmb4x\"" Sep 08 04:01:49.240676 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.240656 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 08 04:01:49.240783 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.240689 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 08 04:01:49.244221 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.244198 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 08 04:01:49.256223 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.256165 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7496599d74-sjlm8"] Sep 08 04:01:49.329868 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.329831 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75c29\" (UniqueName: \"kubernetes.io/projected/64946685-65fe-4715-9125-26985b039d15-kube-api-access-75c29\") pod \"telemeter-client-7496599d74-sjlm8\" (UID: \"64946685-65fe-4715-9125-26985b039d15\") " pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" Sep 08 04:01:49.330058 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.329885 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64946685-65fe-4715-9125-26985b039d15-metrics-client-ca\") pod \"telemeter-client-7496599d74-sjlm8\" (UID: \"64946685-65fe-4715-9125-26985b039d15\") " pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" Sep 08 04:01:49.330058 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.329908 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/64946685-65fe-4715-9125-26985b039d15-telemeter-client-tls\") pod \"telemeter-client-7496599d74-sjlm8\" (UID: \"64946685-65fe-4715-9125-26985b039d15\") " pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" Sep 08 04:01:49.330058 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.329962 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64946685-65fe-4715-9125-26985b039d15-serving-certs-ca-bundle\") pod \"telemeter-client-7496599d74-sjlm8\" (UID: \"64946685-65fe-4715-9125-26985b039d15\") " pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" Sep 08 04:01:49.330058 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.329992 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/64946685-65fe-4715-9125-26985b039d15-federate-client-tls\") pod \"telemeter-client-7496599d74-sjlm8\" (UID: \"64946685-65fe-4715-9125-26985b039d15\") " pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" Sep 08 04:01:49.330058 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.330012 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/64946685-65fe-4715-9125-26985b039d15-secret-telemeter-client\") pod \"telemeter-client-7496599d74-sjlm8\" (UID: \"64946685-65fe-4715-9125-26985b039d15\") " pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" Sep 08 04:01:49.330058 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.330038 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64946685-65fe-4715-9125-26985b039d15-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7496599d74-sjlm8\" (UID: \"64946685-65fe-4715-9125-26985b039d15\") " pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" Sep 08 04:01:49.330253 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.330106 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64946685-65fe-4715-9125-26985b039d15-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7496599d74-sjlm8\" (UID: \"64946685-65fe-4715-9125-26985b039d15\") " pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" Sep 08 04:01:49.431226 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.431183 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64946685-65fe-4715-9125-26985b039d15-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7496599d74-sjlm8\" (UID: \"64946685-65fe-4715-9125-26985b039d15\") " pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" Sep 08 04:01:49.431341 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.431235 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75c29\" (UniqueName: \"kubernetes.io/projected/64946685-65fe-4715-9125-26985b039d15-kube-api-access-75c29\") pod \"telemeter-client-7496599d74-sjlm8\" (UID: \"64946685-65fe-4715-9125-26985b039d15\") " pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" Sep 08 04:01:49.431425 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.431401 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64946685-65fe-4715-9125-26985b039d15-metrics-client-ca\") pod \"telemeter-client-7496599d74-sjlm8\" (UID: \"64946685-65fe-4715-9125-26985b039d15\") " pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" Sep 08 04:01:49.431503 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.431454 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/64946685-65fe-4715-9125-26985b039d15-telemeter-client-tls\") pod \"telemeter-client-7496599d74-sjlm8\" (UID: \"64946685-65fe-4715-9125-26985b039d15\") " pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" Sep 08 04:01:49.431550 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.431525 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64946685-65fe-4715-9125-26985b039d15-serving-certs-ca-bundle\") pod \"telemeter-client-7496599d74-sjlm8\" (UID: \"64946685-65fe-4715-9125-26985b039d15\") " pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" Sep 08 04:01:49.431611 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.431581 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/64946685-65fe-4715-9125-26985b039d15-federate-client-tls\") pod \"telemeter-client-7496599d74-sjlm8\" (UID: \"64946685-65fe-4715-9125-26985b039d15\") " pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" Sep 08 04:01:49.431666 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.431609 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/64946685-65fe-4715-9125-26985b039d15-secret-telemeter-client\") pod \"telemeter-client-7496599d74-sjlm8\" (UID: \"64946685-65fe-4715-9125-26985b039d15\") " pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" Sep 08 04:01:49.431666 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.431636 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64946685-65fe-4715-9125-26985b039d15-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7496599d74-sjlm8\" (UID: \"64946685-65fe-4715-9125-26985b039d15\") " pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" Sep 08 04:01:49.432205 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.432181 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64946685-65fe-4715-9125-26985b039d15-metrics-client-ca\") pod \"telemeter-client-7496599d74-sjlm8\" (UID: \"64946685-65fe-4715-9125-26985b039d15\") " pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" Sep 08 04:01:49.432367 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.432339 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64946685-65fe-4715-9125-26985b039d15-serving-certs-ca-bundle\") pod \"telemeter-client-7496599d74-sjlm8\" (UID: \"64946685-65fe-4715-9125-26985b039d15\") " pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" Sep 08 04:01:49.432447 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.432365 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64946685-65fe-4715-9125-26985b039d15-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7496599d74-sjlm8\" (UID: \"64946685-65fe-4715-9125-26985b039d15\") " pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" Sep 08 04:01:49.434187 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.434157 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64946685-65fe-4715-9125-26985b039d15-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7496599d74-sjlm8\" (UID: \"64946685-65fe-4715-9125-26985b039d15\") " pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" Sep 08 04:01:49.434288 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.434269 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/64946685-65fe-4715-9125-26985b039d15-telemeter-client-tls\") pod \"telemeter-client-7496599d74-sjlm8\" (UID: \"64946685-65fe-4715-9125-26985b039d15\") " pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" Sep 08 04:01:49.434332 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.434315 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/64946685-65fe-4715-9125-26985b039d15-federate-client-tls\") pod \"telemeter-client-7496599d74-sjlm8\" (UID: \"64946685-65fe-4715-9125-26985b039d15\") " pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" Sep 08 04:01:49.434396 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.434371 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/64946685-65fe-4715-9125-26985b039d15-secret-telemeter-client\") pod \"telemeter-client-7496599d74-sjlm8\" (UID: \"64946685-65fe-4715-9125-26985b039d15\") " pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" Sep 08 04:01:49.442111 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.442092 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75c29\" (UniqueName: \"kubernetes.io/projected/64946685-65fe-4715-9125-26985b039d15-kube-api-access-75c29\") pod \"telemeter-client-7496599d74-sjlm8\" (UID: \"64946685-65fe-4715-9125-26985b039d15\") " pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" Sep 08 04:01:49.544816 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.544723 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" Sep 08 04:01:49.556119 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.556090 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 08 04:01:49.556470 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.556424 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="86706a88-1623-4e82-8360-387ba40739e2" containerName="prometheus" containerID="cri-o://3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7" gracePeriod=600 Sep 08 04:01:49.556470 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.556441 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="86706a88-1623-4e82-8360-387ba40739e2" containerName="kube-rbac-proxy" containerID="cri-o://0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94" gracePeriod=600 Sep 08 04:01:49.556633 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.556490 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="86706a88-1623-4e82-8360-387ba40739e2" containerName="thanos-sidecar" containerID="cri-o://6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24" gracePeriod=600 Sep 08 04:01:49.556633 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.556513 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="86706a88-1623-4e82-8360-387ba40739e2" containerName="kube-rbac-proxy-thanos" containerID="cri-o://e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3" gracePeriod=600 Sep 08 04:01:49.556633 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.556592 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="86706a88-1623-4e82-8360-387ba40739e2" containerName="kube-rbac-proxy-web" containerID="cri-o://6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844" gracePeriod=600 Sep 08 04:01:49.556633 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.556598 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="86706a88-1623-4e82-8360-387ba40739e2" containerName="config-reloader" containerID="cri-o://3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a" gracePeriod=600 Sep 08 04:01:49.808326 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.808300 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:49.835447 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.834786 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-configmap-serving-certs-ca-bundle\") pod \"86706a88-1623-4e82-8360-387ba40739e2\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " Sep 08 04:01:49.835447 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.834848 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-metrics-client-certs\") pod \"86706a88-1623-4e82-8360-387ba40739e2\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " Sep 08 04:01:49.835447 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.834874 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-configmap-metrics-client-ca\") pod \"86706a88-1623-4e82-8360-387ba40739e2\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " Sep 08 04:01:49.835447 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.834906 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-configmap-kubelet-serving-ca-bundle\") pod \"86706a88-1623-4e82-8360-387ba40739e2\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " Sep 08 04:01:49.835447 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.834935 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-prometheus-k8s-rulefiles-1\") pod \"86706a88-1623-4e82-8360-387ba40739e2\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " Sep 08 04:01:49.835447 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.834984 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-prometheus-trusted-ca-bundle\") pod \"86706a88-1623-4e82-8360-387ba40739e2\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " Sep 08 04:01:49.835447 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.835012 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-config\") pod \"86706a88-1623-4e82-8360-387ba40739e2\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " Sep 08 04:01:49.835447 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.835042 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-thanos-prometheus-http-client-file\") pod \"86706a88-1623-4e82-8360-387ba40739e2\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " Sep 08 04:01:49.835447 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.835072 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/86706a88-1623-4e82-8360-387ba40739e2-prometheus-k8s-db\") pod \"86706a88-1623-4e82-8360-387ba40739e2\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " Sep 08 04:01:49.835447 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.835099 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"86706a88-1623-4e82-8360-387ba40739e2\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " Sep 08 04:01:49.835447 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.835123 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-prometheus-k8s-rulefiles-2\") pod \"86706a88-1623-4e82-8360-387ba40739e2\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " Sep 08 04:01:49.835447 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.835150 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-prometheus-k8s-rulefiles-0\") pod \"86706a88-1623-4e82-8360-387ba40739e2\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " Sep 08 04:01:49.835447 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.835175 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-web-config\") pod \"86706a88-1623-4e82-8360-387ba40739e2\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " Sep 08 04:01:49.835447 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.835197 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-grpc-tls\") pod \"86706a88-1623-4e82-8360-387ba40739e2\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " Sep 08 04:01:49.835447 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.835223 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/86706a88-1623-4e82-8360-387ba40739e2-config-out\") pod \"86706a88-1623-4e82-8360-387ba40739e2\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " Sep 08 04:01:49.835447 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.835255 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/86706a88-1623-4e82-8360-387ba40739e2-tls-assets\") pod \"86706a88-1623-4e82-8360-387ba40739e2\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " Sep 08 04:01:49.835447 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.835284 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zd957\" (UniqueName: \"kubernetes.io/projected/86706a88-1623-4e82-8360-387ba40739e2-kube-api-access-zd957\") pod \"86706a88-1623-4e82-8360-387ba40739e2\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " Sep 08 04:01:49.835447 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.835305 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-kube-rbac-proxy\") pod \"86706a88-1623-4e82-8360-387ba40739e2\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " Sep 08 04:01:49.836444 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.835339 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-prometheus-k8s-tls\") pod \"86706a88-1623-4e82-8360-387ba40739e2\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " Sep 08 04:01:49.836444 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.835361 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"86706a88-1623-4e82-8360-387ba40739e2\" (UID: \"86706a88-1623-4e82-8360-387ba40739e2\") " Sep 08 04:01:49.837621 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.837583 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "86706a88-1623-4e82-8360-387ba40739e2" (UID: "86706a88-1623-4e82-8360-387ba40739e2"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 04:01:49.838053 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.838021 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "86706a88-1623-4e82-8360-387ba40739e2" (UID: "86706a88-1623-4e82-8360-387ba40739e2"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 04:01:49.838741 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.838699 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7496599d74-sjlm8"] Sep 08 04:01:49.839166 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.839133 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "86706a88-1623-4e82-8360-387ba40739e2" (UID: "86706a88-1623-4e82-8360-387ba40739e2"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 04:01:49.839250 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.839161 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "86706a88-1623-4e82-8360-387ba40739e2" (UID: "86706a88-1623-4e82-8360-387ba40739e2"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 04:01:49.839698 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.839608 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "86706a88-1623-4e82-8360-387ba40739e2" (UID: "86706a88-1623-4e82-8360-387ba40739e2"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 04:01:49.839698 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.839617 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "86706a88-1623-4e82-8360-387ba40739e2" (UID: "86706a88-1623-4e82-8360-387ba40739e2"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 04:01:49.840313 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.839762 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "86706a88-1623-4e82-8360-387ba40739e2" (UID: "86706a88-1623-4e82-8360-387ba40739e2"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 04:01:49.840313 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.840059 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "86706a88-1623-4e82-8360-387ba40739e2" (UID: "86706a88-1623-4e82-8360-387ba40739e2"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 04:01:49.840313 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.840200 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "86706a88-1623-4e82-8360-387ba40739e2" (UID: "86706a88-1623-4e82-8360-387ba40739e2"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 04:01:49.840723 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.840688 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86706a88-1623-4e82-8360-387ba40739e2-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "86706a88-1623-4e82-8360-387ba40739e2" (UID: "86706a88-1623-4e82-8360-387ba40739e2"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 04:01:49.840989 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.840963 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86706a88-1623-4e82-8360-387ba40739e2-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "86706a88-1623-4e82-8360-387ba40739e2" (UID: "86706a88-1623-4e82-8360-387ba40739e2"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 04:01:49.841201 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.841162 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86706a88-1623-4e82-8360-387ba40739e2-config-out" (OuterVolumeSpecName: "config-out") pod "86706a88-1623-4e82-8360-387ba40739e2" (UID: "86706a88-1623-4e82-8360-387ba40739e2"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 04:01:49.841201 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.841177 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "86706a88-1623-4e82-8360-387ba40739e2" (UID: "86706a88-1623-4e82-8360-387ba40739e2"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 04:01:49.841315 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.841203 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "86706a88-1623-4e82-8360-387ba40739e2" (UID: "86706a88-1623-4e82-8360-387ba40739e2"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 04:01:49.841374 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.841324 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "86706a88-1623-4e82-8360-387ba40739e2" (UID: "86706a88-1623-4e82-8360-387ba40739e2"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 04:01:49.841619 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.841592 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86706a88-1623-4e82-8360-387ba40739e2-kube-api-access-zd957" (OuterVolumeSpecName: "kube-api-access-zd957") pod "86706a88-1623-4e82-8360-387ba40739e2" (UID: "86706a88-1623-4e82-8360-387ba40739e2"). InnerVolumeSpecName "kube-api-access-zd957". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 04:01:49.842049 ip-10-0-128-141 kubenswrapper[2579]: W0908 04:01:49.842022 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64946685_65fe_4715_9125_26985b039d15.slice/crio-0fa628609c1ebf5396811a64e93c6dc08759896dcbd902a88f469295251b0259 WatchSource:0}: Error finding container 0fa628609c1ebf5396811a64e93c6dc08759896dcbd902a88f469295251b0259: Status 404 returned error can't find the container with id 0fa628609c1ebf5396811a64e93c6dc08759896dcbd902a88f469295251b0259 Sep 08 04:01:49.842049 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.842030 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "86706a88-1623-4e82-8360-387ba40739e2" (UID: "86706a88-1623-4e82-8360-387ba40739e2"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 04:01:49.842189 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.842020 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "86706a88-1623-4e82-8360-387ba40739e2" (UID: "86706a88-1623-4e82-8360-387ba40739e2"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 04:01:49.842727 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.842692 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-config" (OuterVolumeSpecName: "config") pod "86706a88-1623-4e82-8360-387ba40739e2" (UID: "86706a88-1623-4e82-8360-387ba40739e2"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 04:01:49.843967 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.843949 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 04:01:49.853422 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.853390 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-web-config" (OuterVolumeSpecName: "web-config") pod "86706a88-1623-4e82-8360-387ba40739e2" (UID: "86706a88-1623-4e82-8360-387ba40739e2"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 04:01:49.936829 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.936783 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:49.936829 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.936820 2579 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-config\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:49.936829 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.936830 2579 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:49.936829 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.936839 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/86706a88-1623-4e82-8360-387ba40739e2-prometheus-k8s-db\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:49.937331 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.936849 2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:49.937331 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.936858 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:49.937331 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.936871 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:49.937331 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.936883 2579 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-web-config\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:49.937331 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.936891 2579 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-grpc-tls\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:49.937331 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.936898 2579 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/86706a88-1623-4e82-8360-387ba40739e2-config-out\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:49.937331 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.936905 2579 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/86706a88-1623-4e82-8360-387ba40739e2-tls-assets\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:49.937331 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.936912 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zd957\" (UniqueName: \"kubernetes.io/projected/86706a88-1623-4e82-8360-387ba40739e2-kube-api-access-zd957\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:49.937331 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.936920 2579 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-kube-rbac-proxy\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:49.937331 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.936928 2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:49.937331 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.936938 2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:49.937331 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.936951 2579 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:49.937331 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.936964 2579 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/86706a88-1623-4e82-8360-387ba40739e2-secret-metrics-client-certs\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:49.937331 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.936972 2579 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-configmap-metrics-client-ca\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:49.937331 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.936980 2579 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:49.937331 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:49.936988 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/86706a88-1623-4e82-8360-387ba40739e2-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:01:50.336417 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.336380 2579 generic.go:356] "Generic (PLEG): container finished" podID="86706a88-1623-4e82-8360-387ba40739e2" containerID="e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3" exitCode=0 Sep 08 04:01:50.336417 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.336406 2579 generic.go:356] "Generic (PLEG): container finished" podID="86706a88-1623-4e82-8360-387ba40739e2" containerID="0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94" exitCode=0 Sep 08 04:01:50.336417 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.336415 2579 generic.go:356] "Generic (PLEG): container finished" podID="86706a88-1623-4e82-8360-387ba40739e2" containerID="6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844" exitCode=0 Sep 08 04:01:50.336417 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.336422 2579 generic.go:356] "Generic (PLEG): container finished" podID="86706a88-1623-4e82-8360-387ba40739e2" containerID="6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24" exitCode=0 Sep 08 04:01:50.336417 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.336428 2579 generic.go:356] "Generic (PLEG): container finished" podID="86706a88-1623-4e82-8360-387ba40739e2" containerID="3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a" exitCode=0 Sep 08 04:01:50.336798 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.336436 2579 generic.go:356] "Generic (PLEG): container finished" podID="86706a88-1623-4e82-8360-387ba40739e2" containerID="3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7" exitCode=0 Sep 08 04:01:50.336798 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.336475 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.336798 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.336470 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"86706a88-1623-4e82-8360-387ba40739e2","Type":"ContainerDied","Data":"e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3"} Sep 08 04:01:50.336798 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.336624 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"86706a88-1623-4e82-8360-387ba40739e2","Type":"ContainerDied","Data":"0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94"} Sep 08 04:01:50.336798 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.336660 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"86706a88-1623-4e82-8360-387ba40739e2","Type":"ContainerDied","Data":"6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844"} Sep 08 04:01:50.336798 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.336687 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"86706a88-1623-4e82-8360-387ba40739e2","Type":"ContainerDied","Data":"6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24"} Sep 08 04:01:50.336798 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.336688 2579 scope.go:117] "RemoveContainer" containerID="e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3" Sep 08 04:01:50.336798 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.336718 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"86706a88-1623-4e82-8360-387ba40739e2","Type":"ContainerDied","Data":"3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a"} Sep 08 04:01:50.336798 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.336747 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"86706a88-1623-4e82-8360-387ba40739e2","Type":"ContainerDied","Data":"3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7"} Sep 08 04:01:50.336798 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.336772 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"86706a88-1623-4e82-8360-387ba40739e2","Type":"ContainerDied","Data":"7c7d937a7dd746207777183468d23f486d6bf52cd66179ff0d5c12b32f5447b8"} Sep 08 04:01:50.337944 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.337900 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" event={"ID":"64946685-65fe-4715-9125-26985b039d15","Type":"ContainerStarted","Data":"0fa628609c1ebf5396811a64e93c6dc08759896dcbd902a88f469295251b0259"} Sep 08 04:01:50.345502 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.345484 2579 scope.go:117] "RemoveContainer" containerID="0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94" Sep 08 04:01:50.354358 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.354312 2579 scope.go:117] "RemoveContainer" containerID="6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844" Sep 08 04:01:50.362960 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.362930 2579 scope.go:117] "RemoveContainer" containerID="6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24" Sep 08 04:01:50.364965 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.364850 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 08 04:01:50.372165 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.372137 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 08 04:01:50.373285 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.373268 2579 scope.go:117] "RemoveContainer" containerID="3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a" Sep 08 04:01:50.382512 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.382491 2579 scope.go:117] "RemoveContainer" containerID="3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7" Sep 08 04:01:50.392018 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.391995 2579 scope.go:117] "RemoveContainer" containerID="4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1" Sep 08 04:01:50.400569 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.400536 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 08 04:01:50.400871 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.400854 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86706a88-1623-4e82-8360-387ba40739e2" containerName="prometheus" Sep 08 04:01:50.400944 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.400875 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="86706a88-1623-4e82-8360-387ba40739e2" containerName="prometheus" Sep 08 04:01:50.400944 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.400896 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86706a88-1623-4e82-8360-387ba40739e2" containerName="config-reloader" Sep 08 04:01:50.400944 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.400902 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="86706a88-1623-4e82-8360-387ba40739e2" containerName="config-reloader" Sep 08 04:01:50.400944 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.400913 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86706a88-1623-4e82-8360-387ba40739e2" containerName="kube-rbac-proxy-thanos" Sep 08 04:01:50.400944 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.400919 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="86706a88-1623-4e82-8360-387ba40739e2" containerName="kube-rbac-proxy-thanos" Sep 08 04:01:50.400944 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.400928 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86706a88-1623-4e82-8360-387ba40739e2" containerName="init-config-reloader" Sep 08 04:01:50.400944 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.400935 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="86706a88-1623-4e82-8360-387ba40739e2" containerName="init-config-reloader" Sep 08 04:01:50.400944 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.400941 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86706a88-1623-4e82-8360-387ba40739e2" containerName="thanos-sidecar" Sep 08 04:01:50.400944 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.400947 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="86706a88-1623-4e82-8360-387ba40739e2" containerName="thanos-sidecar" Sep 08 04:01:50.401434 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.400958 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86706a88-1623-4e82-8360-387ba40739e2" containerName="kube-rbac-proxy-web" Sep 08 04:01:50.401434 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.400963 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="86706a88-1623-4e82-8360-387ba40739e2" containerName="kube-rbac-proxy-web" Sep 08 04:01:50.401434 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.400968 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86706a88-1623-4e82-8360-387ba40739e2" containerName="kube-rbac-proxy" Sep 08 04:01:50.401434 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.400973 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="86706a88-1623-4e82-8360-387ba40739e2" containerName="kube-rbac-proxy" Sep 08 04:01:50.401434 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.401027 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="86706a88-1623-4e82-8360-387ba40739e2" containerName="kube-rbac-proxy-web" Sep 08 04:01:50.401434 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.401036 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="86706a88-1623-4e82-8360-387ba40739e2" containerName="prometheus" Sep 08 04:01:50.401434 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.401044 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="86706a88-1623-4e82-8360-387ba40739e2" containerName="thanos-sidecar" Sep 08 04:01:50.401434 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.401053 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="86706a88-1623-4e82-8360-387ba40739e2" containerName="kube-rbac-proxy" Sep 08 04:01:50.401434 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.401063 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="86706a88-1623-4e82-8360-387ba40739e2" containerName="config-reloader" Sep 08 04:01:50.401434 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.401072 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="86706a88-1623-4e82-8360-387ba40739e2" containerName="kube-rbac-proxy-thanos" Sep 08 04:01:50.401434 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.401304 2579 scope.go:117] "RemoveContainer" containerID="e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3" Sep 08 04:01:50.401987 ip-10-0-128-141 kubenswrapper[2579]: E0908 04:01:50.401827 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3\": container with ID starting with e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3 not found: ID does not exist" containerID="e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3" Sep 08 04:01:50.401987 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.401862 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3"} err="failed to get container status \"e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3\": rpc error: code = NotFound desc = could not find container \"e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3\": container with ID starting with e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3 not found: ID does not exist" Sep 08 04:01:50.401987 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.401919 2579 scope.go:117] "RemoveContainer" containerID="0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94" Sep 08 04:01:50.402348 ip-10-0-128-141 kubenswrapper[2579]: E0908 04:01:50.402323 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94\": container with ID starting with 0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94 not found: ID does not exist" containerID="0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94" Sep 08 04:01:50.402437 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.402355 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94"} err="failed to get container status \"0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94\": rpc error: code = NotFound desc = could not find container \"0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94\": container with ID starting with 0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94 not found: ID does not exist" Sep 08 04:01:50.402437 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.402376 2579 scope.go:117] "RemoveContainer" containerID="6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844" Sep 08 04:01:50.402785 ip-10-0-128-141 kubenswrapper[2579]: E0908 04:01:50.402762 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844\": container with ID starting with 6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844 not found: ID does not exist" containerID="6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844" Sep 08 04:01:50.402843 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.402811 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844"} err="failed to get container status \"6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844\": rpc error: code = NotFound desc = could not find container \"6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844\": container with ID starting with 6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844 not found: ID does not exist" Sep 08 04:01:50.402843 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.402828 2579 scope.go:117] "RemoveContainer" containerID="6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24" Sep 08 04:01:50.403159 ip-10-0-128-141 kubenswrapper[2579]: E0908 04:01:50.403097 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24\": container with ID starting with 6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24 not found: ID does not exist" containerID="6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24" Sep 08 04:01:50.403159 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.403134 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24"} err="failed to get container status \"6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24\": rpc error: code = NotFound desc = could not find container \"6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24\": container with ID starting with 6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24 not found: ID does not exist" Sep 08 04:01:50.403159 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.403155 2579 scope.go:117] "RemoveContainer" containerID="3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a" Sep 08 04:01:50.403420 ip-10-0-128-141 kubenswrapper[2579]: E0908 04:01:50.403399 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a\": container with ID starting with 3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a not found: ID does not exist" containerID="3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a" Sep 08 04:01:50.403532 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.403427 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a"} err="failed to get container status \"3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a\": rpc error: code = NotFound desc = could not find container \"3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a\": container with ID starting with 3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a not found: ID does not exist" Sep 08 04:01:50.403532 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.403447 2579 scope.go:117] "RemoveContainer" containerID="3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7" Sep 08 04:01:50.403743 ip-10-0-128-141 kubenswrapper[2579]: E0908 04:01:50.403722 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7\": container with ID starting with 3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7 not found: ID does not exist" containerID="3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7" Sep 08 04:01:50.403789 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.403751 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7"} err="failed to get container status \"3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7\": rpc error: code = NotFound desc = could not find container \"3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7\": container with ID starting with 3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7 not found: ID does not exist" Sep 08 04:01:50.403789 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.403770 2579 scope.go:117] "RemoveContainer" containerID="4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1" Sep 08 04:01:50.404039 ip-10-0-128-141 kubenswrapper[2579]: E0908 04:01:50.404021 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1\": container with ID starting with 4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1 not found: ID does not exist" containerID="4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1" Sep 08 04:01:50.404097 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.404045 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1"} err="failed to get container status \"4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1\": rpc error: code = NotFound desc = could not find container \"4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1\": container with ID starting with 4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1 not found: ID does not exist" Sep 08 04:01:50.404097 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.404065 2579 scope.go:117] "RemoveContainer" containerID="e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3" Sep 08 04:01:50.404412 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.404366 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3"} err="failed to get container status \"e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3\": rpc error: code = NotFound desc = could not find container \"e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3\": container with ID starting with e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3 not found: ID does not exist" Sep 08 04:01:50.404412 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.404389 2579 scope.go:117] "RemoveContainer" containerID="0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94" Sep 08 04:01:50.404692 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.404666 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94"} err="failed to get container status \"0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94\": rpc error: code = NotFound desc = could not find container \"0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94\": container with ID starting with 0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94 not found: ID does not exist" Sep 08 04:01:50.404692 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.404692 2579 scope.go:117] "RemoveContainer" containerID="6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844" Sep 08 04:01:50.404839 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.404705 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.405257 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.404974 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844"} err="failed to get container status \"6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844\": rpc error: code = NotFound desc = could not find container \"6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844\": container with ID starting with 6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844 not found: ID does not exist" Sep 08 04:01:50.405257 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.405007 2579 scope.go:117] "RemoveContainer" containerID="6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24" Sep 08 04:01:50.405611 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.405578 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24"} err="failed to get container status \"6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24\": rpc error: code = NotFound desc = could not find container \"6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24\": container with ID starting with 6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24 not found: ID does not exist" Sep 08 04:01:50.405718 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.405611 2579 scope.go:117] "RemoveContainer" containerID="3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a" Sep 08 04:01:50.405947 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.405918 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a"} err="failed to get container status \"3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a\": rpc error: code = NotFound desc = could not find container \"3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a\": container with ID starting with 3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a not found: ID does not exist" Sep 08 04:01:50.406023 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.405950 2579 scope.go:117] "RemoveContainer" containerID="3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7" Sep 08 04:01:50.406211 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.406185 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7"} err="failed to get container status \"3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7\": rpc error: code = NotFound desc = could not find container \"3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7\": container with ID starting with 3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7 not found: ID does not exist" Sep 08 04:01:50.406308 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.406212 2579 scope.go:117] "RemoveContainer" containerID="4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1" Sep 08 04:01:50.406535 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.406512 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1"} err="failed to get container status \"4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1\": rpc error: code = NotFound desc = could not find container \"4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1\": container with ID starting with 4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1 not found: ID does not exist" Sep 08 04:01:50.406640 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.406537 2579 scope.go:117] "RemoveContainer" containerID="e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3" Sep 08 04:01:50.406870 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.406848 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3"} err="failed to get container status \"e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3\": rpc error: code = NotFound desc = could not find container \"e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3\": container with ID starting with e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3 not found: ID does not exist" Sep 08 04:01:50.406944 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.406871 2579 scope.go:117] "RemoveContainer" containerID="0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94" Sep 08 04:01:50.407825 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.407736 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94"} err="failed to get container status \"0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94\": rpc error: code = NotFound desc = could not find container \"0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94\": container with ID starting with 0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94 not found: ID does not exist" Sep 08 04:01:50.407825 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.407760 2579 scope.go:117] "RemoveContainer" containerID="6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844" Sep 08 04:01:50.409765 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.409734 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844"} err="failed to get container status \"6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844\": rpc error: code = NotFound desc = could not find container \"6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844\": container with ID starting with 6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844 not found: ID does not exist" Sep 08 04:01:50.409846 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.409766 2579 scope.go:117] "RemoveContainer" containerID="6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24" Sep 08 04:01:50.410435 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.409937 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 08 04:01:50.410435 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.410088 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24"} err="failed to get container status \"6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24\": rpc error: code = NotFound desc = could not find container \"6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24\": container with ID starting with 6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24 not found: ID does not exist" Sep 08 04:01:50.410435 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.410112 2579 scope.go:117] "RemoveContainer" containerID="3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a" Sep 08 04:01:50.410435 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.410128 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 08 04:01:50.410435 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.410142 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 08 04:01:50.410435 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.410255 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 08 04:01:50.410435 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.410263 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 08 04:01:50.410435 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.410340 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 08 04:01:50.410435 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.410383 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 08 04:01:50.410986 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.410480 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a"} err="failed to get container status \"3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a\": rpc error: code = NotFound desc = could not find container \"3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a\": container with ID starting with 3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a not found: ID does not exist" Sep 08 04:01:50.410986 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.410503 2579 scope.go:117] "RemoveContainer" containerID="3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7" Sep 08 04:01:50.410986 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.410924 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 08 04:01:50.410986 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.410939 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2pf7jaspgjh6o\"" Sep 08 04:01:50.411223 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.411170 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-mb5qw\"" Sep 08 04:01:50.411223 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.411175 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 08 04:01:50.411223 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.411191 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 08 04:01:50.411223 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.411206 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 08 04:01:50.411422 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.411194 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 08 04:01:50.411518 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.411492 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7"} err="failed to get container status \"3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7\": rpc error: code = NotFound desc = could not find container \"3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7\": container with ID starting with 3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7 not found: ID does not exist" Sep 08 04:01:50.411586 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.411519 2579 scope.go:117] "RemoveContainer" containerID="4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1" Sep 08 04:01:50.412654 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.411829 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 08 04:01:50.412654 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.412021 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1"} err="failed to get container status \"4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1\": rpc error: code = NotFound desc = could not find container \"4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1\": container with ID starting with 4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1 not found: ID does not exist" Sep 08 04:01:50.412654 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.412042 2579 scope.go:117] "RemoveContainer" containerID="e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3" Sep 08 04:01:50.412654 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.412260 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3"} err="failed to get container status \"e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3\": rpc error: code = NotFound desc = could not find container \"e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3\": container with ID starting with e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3 not found: ID does not exist" Sep 08 04:01:50.412654 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.412282 2579 scope.go:117] "RemoveContainer" containerID="0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94" Sep 08 04:01:50.412654 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.412590 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94"} err="failed to get container status \"0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94\": rpc error: code = NotFound desc = could not find container \"0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94\": container with ID starting with 0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94 not found: ID does not exist" Sep 08 04:01:50.412654 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.412609 2579 scope.go:117] "RemoveContainer" containerID="6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844" Sep 08 04:01:50.413023 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.412865 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844"} err="failed to get container status \"6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844\": rpc error: code = NotFound desc = could not find container \"6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844\": container with ID starting with 6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844 not found: ID does not exist" Sep 08 04:01:50.413023 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.412888 2579 scope.go:117] "RemoveContainer" containerID="6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24" Sep 08 04:01:50.413518 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.413495 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 08 04:01:50.413662 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.413443 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24"} err="failed to get container status \"6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24\": rpc error: code = NotFound desc = could not find container \"6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24\": container with ID starting with 6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24 not found: ID does not exist" Sep 08 04:01:50.413662 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.413534 2579 scope.go:117] "RemoveContainer" containerID="3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a" Sep 08 04:01:50.413915 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.413820 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a"} err="failed to get container status \"3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a\": rpc error: code = NotFound desc = could not find container \"3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a\": container with ID starting with 3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a not found: ID does not exist" Sep 08 04:01:50.413915 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.413847 2579 scope.go:117] "RemoveContainer" containerID="3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7" Sep 08 04:01:50.414632 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.414192 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7"} err="failed to get container status \"3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7\": rpc error: code = NotFound desc = could not find container \"3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7\": container with ID starting with 3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7 not found: ID does not exist" Sep 08 04:01:50.414632 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.414237 2579 scope.go:117] "RemoveContainer" containerID="4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1" Sep 08 04:01:50.415035 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.415010 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1"} err="failed to get container status \"4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1\": rpc error: code = NotFound desc = could not find container \"4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1\": container with ID starting with 4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1 not found: ID does not exist" Sep 08 04:01:50.415119 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.415039 2579 scope.go:117] "RemoveContainer" containerID="e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3" Sep 08 04:01:50.415393 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.415367 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3"} err="failed to get container status \"e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3\": rpc error: code = NotFound desc = could not find container \"e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3\": container with ID starting with e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3 not found: ID does not exist" Sep 08 04:01:50.415500 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.415396 2579 scope.go:117] "RemoveContainer" containerID="0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94" Sep 08 04:01:50.415759 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.415685 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94"} err="failed to get container status \"0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94\": rpc error: code = NotFound desc = could not find container \"0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94\": container with ID starting with 0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94 not found: ID does not exist" Sep 08 04:01:50.415759 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.415710 2579 scope.go:117] "RemoveContainer" containerID="6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844" Sep 08 04:01:50.415993 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.415965 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844"} err="failed to get container status \"6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844\": rpc error: code = NotFound desc = could not find container \"6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844\": container with ID starting with 6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844 not found: ID does not exist" Sep 08 04:01:50.416081 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.415994 2579 scope.go:117] "RemoveContainer" containerID="6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24" Sep 08 04:01:50.416141 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.416123 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 08 04:01:50.416336 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.416308 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24"} err="failed to get container status \"6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24\": rpc error: code = NotFound desc = could not find container \"6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24\": container with ID starting with 6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24 not found: ID does not exist" Sep 08 04:01:50.416336 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.416334 2579 scope.go:117] "RemoveContainer" containerID="3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a" Sep 08 04:01:50.416627 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.416600 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a"} err="failed to get container status \"3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a\": rpc error: code = NotFound desc = could not find container \"3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a\": container with ID starting with 3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a not found: ID does not exist" Sep 08 04:01:50.416719 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.416629 2579 scope.go:117] "RemoveContainer" containerID="3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7" Sep 08 04:01:50.416915 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.416892 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7"} err="failed to get container status \"3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7\": rpc error: code = NotFound desc = could not find container \"3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7\": container with ID starting with 3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7 not found: ID does not exist" Sep 08 04:01:50.416995 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.416917 2579 scope.go:117] "RemoveContainer" containerID="4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1" Sep 08 04:01:50.417171 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.417144 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1"} err="failed to get container status \"4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1\": rpc error: code = NotFound desc = could not find container \"4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1\": container with ID starting with 4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1 not found: ID does not exist" Sep 08 04:01:50.417244 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.417171 2579 scope.go:117] "RemoveContainer" containerID="e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3" Sep 08 04:01:50.417440 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.417405 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3"} err="failed to get container status \"e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3\": rpc error: code = NotFound desc = could not find container \"e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3\": container with ID starting with e19208a5e32aad1b09d446fa44875c1dea1429e809937778c6d5d00c301672b3 not found: ID does not exist" Sep 08 04:01:50.417440 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.417421 2579 scope.go:117] "RemoveContainer" containerID="0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94" Sep 08 04:01:50.417677 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.417652 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94"} err="failed to get container status \"0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94\": rpc error: code = NotFound desc = could not find container \"0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94\": container with ID starting with 0a2b0acd5d304cda916f2d228c6da9075774f0abbe88ddcfe2d8caccb0a2ce94 not found: ID does not exist" Sep 08 04:01:50.417677 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.417677 2579 scope.go:117] "RemoveContainer" containerID="6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844" Sep 08 04:01:50.417925 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.417908 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844"} err="failed to get container status \"6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844\": rpc error: code = NotFound desc = could not find container \"6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844\": container with ID starting with 6cd597f7d543018787642c82506afbc0a147ac22a25f049642b84f0d81b11844 not found: ID does not exist" Sep 08 04:01:50.417986 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.417924 2579 scope.go:117] "RemoveContainer" containerID="6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24" Sep 08 04:01:50.418204 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.418182 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24"} err="failed to get container status \"6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24\": rpc error: code = NotFound desc = could not find container \"6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24\": container with ID starting with 6d54b5adb8f170baa7e73875240585c0aa9fec3949f816cdfca0327fda4f3b24 not found: ID does not exist" Sep 08 04:01:50.418267 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.418207 2579 scope.go:117] "RemoveContainer" containerID="3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a" Sep 08 04:01:50.418527 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.418507 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a"} err="failed to get container status \"3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a\": rpc error: code = NotFound desc = could not find container \"3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a\": container with ID starting with 3245976104aa98c2838b673f8aed15cd98751366bcdf5fb1fc47293040e07a0a not found: ID does not exist" Sep 08 04:01:50.418527 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.418526 2579 scope.go:117] "RemoveContainer" containerID="3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7" Sep 08 04:01:50.418778 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.418758 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7"} err="failed to get container status \"3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7\": rpc error: code = NotFound desc = could not find container \"3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7\": container with ID starting with 3132aa692f4d64913a46d490537862e48e60bdc5bd54781ea8e1a55883b64de7 not found: ID does not exist" Sep 08 04:01:50.418778 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.418778 2579 scope.go:117] "RemoveContainer" containerID="4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1" Sep 08 04:01:50.419034 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.419008 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1"} err="failed to get container status \"4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1\": rpc error: code = NotFound desc = could not find container \"4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1\": container with ID starting with 4fbc667aa10f2c71990675ab43cdf44563832b809b9cf64a39cc3d2b841171d1 not found: ID does not exist" Sep 08 04:01:50.424208 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.424184 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 08 04:01:50.441404 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.441160 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7dbb7d85-4b09-4116-875b-8f96789a4eeb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.441404 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.441223 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7dbb7d85-4b09-4116-875b-8f96789a4eeb-config\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.441404 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.441288 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7dbb7d85-4b09-4116-875b-8f96789a4eeb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.441404 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.441328 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7dbb7d85-4b09-4116-875b-8f96789a4eeb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.441404 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.441359 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7dbb7d85-4b09-4116-875b-8f96789a4eeb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.441710 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.441445 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7dbb7d85-4b09-4116-875b-8f96789a4eeb-config-out\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.441710 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.441504 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7dbb7d85-4b09-4116-875b-8f96789a4eeb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.441710 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.441577 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7dbb7d85-4b09-4116-875b-8f96789a4eeb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.441710 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.441613 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7dbb7d85-4b09-4116-875b-8f96789a4eeb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.441710 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.441636 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7dbb7d85-4b09-4116-875b-8f96789a4eeb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.441710 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.441658 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7dbb7d85-4b09-4116-875b-8f96789a4eeb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.441710 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.441679 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7dbb7d85-4b09-4116-875b-8f96789a4eeb-web-config\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.442106 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.441749 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7dbb7d85-4b09-4116-875b-8f96789a4eeb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.442106 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.441779 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7dbb7d85-4b09-4116-875b-8f96789a4eeb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.442106 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.441805 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7dbb7d85-4b09-4116-875b-8f96789a4eeb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.442106 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.441832 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7dbb7d85-4b09-4116-875b-8f96789a4eeb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.442106 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.441856 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7dbb7d85-4b09-4116-875b-8f96789a4eeb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.442106 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.441929 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7dbb7d85-4b09-4116-875b-8f96789a4eeb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.442106 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.441963 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqvzs\" (UniqueName: \"kubernetes.io/projected/7dbb7d85-4b09-4116-875b-8f96789a4eeb-kube-api-access-bqvzs\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.442106 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.442004 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7dbb7d85-4b09-4116-875b-8f96789a4eeb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.542955 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.542905 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7dbb7d85-4b09-4116-875b-8f96789a4eeb-config\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.543236 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.542966 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7dbb7d85-4b09-4116-875b-8f96789a4eeb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.543236 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.543031 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7dbb7d85-4b09-4116-875b-8f96789a4eeb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.543236 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.543078 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7dbb7d85-4b09-4116-875b-8f96789a4eeb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.543236 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.543121 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7dbb7d85-4b09-4116-875b-8f96789a4eeb-config-out\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.543236 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.543154 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7dbb7d85-4b09-4116-875b-8f96789a4eeb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.543236 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.543213 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7dbb7d85-4b09-4116-875b-8f96789a4eeb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.543583 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.543241 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7dbb7d85-4b09-4116-875b-8f96789a4eeb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.543583 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.543264 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7dbb7d85-4b09-4116-875b-8f96789a4eeb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.543583 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.543288 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7dbb7d85-4b09-4116-875b-8f96789a4eeb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.543583 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.543313 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7dbb7d85-4b09-4116-875b-8f96789a4eeb-web-config\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.543583 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.543348 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7dbb7d85-4b09-4116-875b-8f96789a4eeb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.543583 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.543383 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7dbb7d85-4b09-4116-875b-8f96789a4eeb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.543583 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.543410 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7dbb7d85-4b09-4116-875b-8f96789a4eeb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.543583 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.543437 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7dbb7d85-4b09-4116-875b-8f96789a4eeb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.543583 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.543485 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7dbb7d85-4b09-4116-875b-8f96789a4eeb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.543583 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.543525 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7dbb7d85-4b09-4116-875b-8f96789a4eeb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.543583 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.543549 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqvzs\" (UniqueName: \"kubernetes.io/projected/7dbb7d85-4b09-4116-875b-8f96789a4eeb-kube-api-access-bqvzs\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.543583 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.543585 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7dbb7d85-4b09-4116-875b-8f96789a4eeb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.544199 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.543628 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7dbb7d85-4b09-4116-875b-8f96789a4eeb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.544199 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.543812 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7dbb7d85-4b09-4116-875b-8f96789a4eeb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.544199 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.544073 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7dbb7d85-4b09-4116-875b-8f96789a4eeb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.544199 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.544109 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7dbb7d85-4b09-4116-875b-8f96789a4eeb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.544402 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.544283 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7dbb7d85-4b09-4116-875b-8f96789a4eeb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.545672 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.545648 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7dbb7d85-4b09-4116-875b-8f96789a4eeb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.547159 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.546272 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7dbb7d85-4b09-4116-875b-8f96789a4eeb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.547159 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.546610 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7dbb7d85-4b09-4116-875b-8f96789a4eeb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.547159 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.547121 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7dbb7d85-4b09-4116-875b-8f96789a4eeb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.547880 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.547854 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7dbb7d85-4b09-4116-875b-8f96789a4eeb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.547966 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.547902 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7dbb7d85-4b09-4116-875b-8f96789a4eeb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.549441 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.549395 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7dbb7d85-4b09-4116-875b-8f96789a4eeb-config-out\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.549562 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.549520 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7dbb7d85-4b09-4116-875b-8f96789a4eeb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.549637 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.549593 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7dbb7d85-4b09-4116-875b-8f96789a4eeb-web-config\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.549742 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.549714 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7dbb7d85-4b09-4116-875b-8f96789a4eeb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.549843 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.549790 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7dbb7d85-4b09-4116-875b-8f96789a4eeb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.550204 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.550186 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7dbb7d85-4b09-4116-875b-8f96789a4eeb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.550381 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.550362 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7dbb7d85-4b09-4116-875b-8f96789a4eeb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.551049 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.551031 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7dbb7d85-4b09-4116-875b-8f96789a4eeb-config\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.551340 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.551319 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7dbb7d85-4b09-4116-875b-8f96789a4eeb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.554884 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.554862 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqvzs\" (UniqueName: \"kubernetes.io/projected/7dbb7d85-4b09-4116-875b-8f96789a4eeb-kube-api-access-bqvzs\") pod \"prometheus-k8s-0\" (UID: \"7dbb7d85-4b09-4116-875b-8f96789a4eeb\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.717920 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.717884 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:01:50.881109 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:50.881074 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86706a88-1623-4e82-8360-387ba40739e2" path="/var/lib/kubelet/pods/86706a88-1623-4e82-8360-387ba40739e2/volumes" Sep 08 04:01:51.019501 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:51.019470 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 08 04:01:51.425402 ip-10-0-128-141 kubenswrapper[2579]: W0908 04:01:51.425362 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dbb7d85_4b09_4116_875b_8f96789a4eeb.slice/crio-2e632921837caacc8b3a88076ea1e582571a568580dee78532b44e904c9285fa WatchSource:0}: Error finding container 2e632921837caacc8b3a88076ea1e582571a568580dee78532b44e904c9285fa: Status 404 returned error can't find the container with id 2e632921837caacc8b3a88076ea1e582571a568580dee78532b44e904c9285fa Sep 08 04:01:52.349978 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:52.349940 2579 generic.go:356] "Generic (PLEG): container finished" podID="7dbb7d85-4b09-4116-875b-8f96789a4eeb" containerID="21bbcd94f60f39c3a3ba5153e51c62bc51f7606bc7c0a7014db1c196a7a4e74f" exitCode=0 Sep 08 04:01:52.350451 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:52.349987 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7dbb7d85-4b09-4116-875b-8f96789a4eeb","Type":"ContainerDied","Data":"21bbcd94f60f39c3a3ba5153e51c62bc51f7606bc7c0a7014db1c196a7a4e74f"} Sep 08 04:01:52.350451 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:52.350071 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7dbb7d85-4b09-4116-875b-8f96789a4eeb","Type":"ContainerStarted","Data":"2e632921837caacc8b3a88076ea1e582571a568580dee78532b44e904c9285fa"} Sep 08 04:01:52.352306 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:52.352283 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" event={"ID":"64946685-65fe-4715-9125-26985b039d15","Type":"ContainerStarted","Data":"8a92d51c46594253a0daf2747bf264d3fa902f5c7c9d1a628558453aa3a11224"} Sep 08 04:01:52.352396 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:52.352335 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" event={"ID":"64946685-65fe-4715-9125-26985b039d15","Type":"ContainerStarted","Data":"384d72a57d92dfbc009b5ef73b892649fb61659da4b2d341c1873017b7061bd5"} Sep 08 04:01:52.352396 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:52.352355 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" event={"ID":"64946685-65fe-4715-9125-26985b039d15","Type":"ContainerStarted","Data":"e56f6ad5507205212cb8d288540ea3153c028b705f37d0402c84cd621893c2f5"} Sep 08 04:01:52.406672 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:52.406621 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7496599d74-sjlm8" podStartSLOduration=1.796830556 podStartE2EDuration="3.406603577s" totalImagesPullingTime="1.609773021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 04:01:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:01:52.404394367 +0000 UTC m=+372.025430335" watchObservedRunningTime="2026-09-08 04:01:52.406603577 +0000 UTC m=+372.027639543" Sep 08 04:01:53.365368 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:53.365329 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7dbb7d85-4b09-4116-875b-8f96789a4eeb","Type":"ContainerStarted","Data":"ce3b9d1bf0780de435e29d2e20bf40a8ae95beef54396f6f5c72b9d556b015dd"} Sep 08 04:01:53.365793 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:53.365417 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7dbb7d85-4b09-4116-875b-8f96789a4eeb","Type":"ContainerStarted","Data":"d0b0cd5259252a731fa92081c1f91574e4fb9b981fc9fc7cfd4ccfe3a4295f06"} Sep 08 04:01:53.365793 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:53.365447 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7dbb7d85-4b09-4116-875b-8f96789a4eeb","Type":"ContainerStarted","Data":"626d870ed2f2913ed09c2627a13f8a6353c1af245a554b40cbfd8bfbc4244cef"} Sep 08 04:01:53.365793 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:53.365482 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7dbb7d85-4b09-4116-875b-8f96789a4eeb","Type":"ContainerStarted","Data":"2647ea21801323d4fd225a98a089af3d9a43136c732381ee554fc3514fb728ed"} Sep 08 04:01:53.365793 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:53.365506 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7dbb7d85-4b09-4116-875b-8f96789a4eeb","Type":"ContainerStarted","Data":"b357cfc19d4d7bf3be2fc91d7b0364d6b39511e145f28569777d086a698c11be"} Sep 08 04:01:53.365793 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:53.365528 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7dbb7d85-4b09-4116-875b-8f96789a4eeb","Type":"ContainerStarted","Data":"8d085cb11fc46ebefebe22c19e2905b8c2a87b4b72060440ef87630b142e862e"} Sep 08 04:01:53.400054 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:53.399994 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.374431627 podStartE2EDuration="3.399973546s" totalImagesPullingTime="0s" totalInitContainerRuntime="25.541919ms" isStatefulPod=false podCreationTimestamp="2026-09-08 04:01:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:01:53.39835212 +0000 UTC m=+373.019388089" watchObservedRunningTime="2026-09-08 04:01:53.399973546 +0000 UTC m=+373.021009513" Sep 08 04:01:53.772902 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:53.772858 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs\") pod \"network-metrics-daemon-r4xfl\" (UID: \"b7239679-8052-415f-ae9a-1b20d61d1223\") " pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 04:01:53.775168 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:53.775145 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7239679-8052-415f-ae9a-1b20d61d1223-metrics-certs\") pod \"network-metrics-daemon-r4xfl\" (UID: \"b7239679-8052-415f-ae9a-1b20d61d1223\") " pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 04:01:53.874339 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:53.874301 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tzmtk\"" Sep 08 04:01:53.881886 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:53.881863 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xfl" Sep 08 04:01:54.154574 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:54.154540 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r4xfl"] Sep 08 04:01:54.156956 ip-10-0-128-141 kubenswrapper[2579]: W0908 04:01:54.156929 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7239679_8052_415f_ae9a_1b20d61d1223.slice/crio-376d1601834dc6d7b70ac98695f68f518200c7663b72e300eaf6393329497158 WatchSource:0}: Error finding container 376d1601834dc6d7b70ac98695f68f518200c7663b72e300eaf6393329497158: Status 404 returned error can't find the container with id 376d1601834dc6d7b70ac98695f68f518200c7663b72e300eaf6393329497158 Sep 08 04:01:54.371421 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:54.371381 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r4xfl" event={"ID":"b7239679-8052-415f-ae9a-1b20d61d1223","Type":"ContainerStarted","Data":"376d1601834dc6d7b70ac98695f68f518200c7663b72e300eaf6393329497158"} Sep 08 04:01:55.377112 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:55.377077 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r4xfl" event={"ID":"b7239679-8052-415f-ae9a-1b20d61d1223","Type":"ContainerStarted","Data":"8f64c7a3213a92818e30ea0b2e298743058df8a4e1dee9a3f90c514599e7a773"} Sep 08 04:01:55.377523 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:55.377129 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r4xfl" event={"ID":"b7239679-8052-415f-ae9a-1b20d61d1223","Type":"ContainerStarted","Data":"9ff214421cd21c1a8b07931fa22921ded74f223a0af1f9fe3269ea7f306d5436"} Sep 08 04:01:55.396318 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:55.396259 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-r4xfl" podStartSLOduration=373.397321773 podStartE2EDuration="6m14.396239786s" totalImagesPullingTime="998.918013ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 03:55:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:01:55.395996162 +0000 UTC m=+375.017032163" watchObservedRunningTime="2026-09-08 04:01:55.396239786 +0000 UTC m=+375.017275756" Sep 08 04:01:55.719003 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:01:55.718954 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:02:50.718775 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:02:50.718730 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:02:50.734449 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:02:50.734421 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:02:51.609780 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:02:51.609755 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 04:04:40.654748 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:40.654673 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-operator-56d489b4b8-xqxtv"] Sep 08 04:04:40.657542 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:40.657525 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-56d489b4b8-xqxtv" Sep 08 04:04:40.660210 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:40.660185 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a617ff2e-7bb3-43c5-950b-06e4b9075b92-tmp\") pod \"jobset-operator-56d489b4b8-xqxtv\" (UID: \"a617ff2e-7bb3-43c5-950b-06e4b9075b92\") " pod="openshift-jobset-operator/jobset-operator-56d489b4b8-xqxtv" Sep 08 04:04:40.660333 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:40.660237 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9z6p\" (UniqueName: \"kubernetes.io/projected/a617ff2e-7bb3-43c5-950b-06e4b9075b92-kube-api-access-x9z6p\") pod \"jobset-operator-56d489b4b8-xqxtv\" (UID: \"a617ff2e-7bb3-43c5-950b-06e4b9075b92\") " pod="openshift-jobset-operator/jobset-operator-56d489b4b8-xqxtv" Sep 08 04:04:40.661569 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:40.661546 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-operator-dockercfg-4qllb\"" Sep 08 04:04:40.661569 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:40.661563 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\"" Sep 08 04:04:40.661706 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:40.661616 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\"" Sep 08 04:04:40.668140 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:40.668117 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-56d489b4b8-xqxtv"] Sep 08 04:04:40.761198 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:40.761172 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a617ff2e-7bb3-43c5-950b-06e4b9075b92-tmp\") pod \"jobset-operator-56d489b4b8-xqxtv\" (UID: \"a617ff2e-7bb3-43c5-950b-06e4b9075b92\") " pod="openshift-jobset-operator/jobset-operator-56d489b4b8-xqxtv" Sep 08 04:04:40.761348 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:40.761225 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9z6p\" (UniqueName: \"kubernetes.io/projected/a617ff2e-7bb3-43c5-950b-06e4b9075b92-kube-api-access-x9z6p\") pod \"jobset-operator-56d489b4b8-xqxtv\" (UID: \"a617ff2e-7bb3-43c5-950b-06e4b9075b92\") " pod="openshift-jobset-operator/jobset-operator-56d489b4b8-xqxtv" Sep 08 04:04:40.761548 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:40.761528 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a617ff2e-7bb3-43c5-950b-06e4b9075b92-tmp\") pod \"jobset-operator-56d489b4b8-xqxtv\" (UID: \"a617ff2e-7bb3-43c5-950b-06e4b9075b92\") " pod="openshift-jobset-operator/jobset-operator-56d489b4b8-xqxtv" Sep 08 04:04:40.774970 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:40.774946 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\"" Sep 08 04:04:40.785279 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:40.785258 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\"" Sep 08 04:04:40.795169 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:40.795141 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9z6p\" (UniqueName: \"kubernetes.io/projected/a617ff2e-7bb3-43c5-950b-06e4b9075b92-kube-api-access-x9z6p\") pod \"jobset-operator-56d489b4b8-xqxtv\" (UID: \"a617ff2e-7bb3-43c5-950b-06e4b9075b92\") " pod="openshift-jobset-operator/jobset-operator-56d489b4b8-xqxtv" Sep 08 04:04:40.982953 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:40.982871 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-operator-dockercfg-4qllb\"" Sep 08 04:04:40.991091 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:40.991075 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-56d489b4b8-xqxtv" Sep 08 04:04:41.260176 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:41.260151 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-56d489b4b8-xqxtv"] Sep 08 04:04:41.262859 ip-10-0-128-141 kubenswrapper[2579]: W0908 04:04:41.262830 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda617ff2e_7bb3_43c5_950b_06e4b9075b92.slice/crio-670157d410efa89f558e76642c6c2802276f8c2737fac131d7d29aea38b2f1d4 WatchSource:0}: Error finding container 670157d410efa89f558e76642c6c2802276f8c2737fac131d7d29aea38b2f1d4: Status 404 returned error can't find the container with id 670157d410efa89f558e76642c6c2802276f8c2737fac131d7d29aea38b2f1d4 Sep 08 04:04:42.021022 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:42.020985 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-56d489b4b8-xqxtv" event={"ID":"a617ff2e-7bb3-43c5-950b-06e4b9075b92","Type":"ContainerStarted","Data":"670157d410efa89f558e76642c6c2802276f8c2737fac131d7d29aea38b2f1d4"} Sep 08 04:04:44.032054 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:44.032024 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-56d489b4b8-xqxtv" event={"ID":"a617ff2e-7bb3-43c5-950b-06e4b9075b92","Type":"ContainerStarted","Data":"97c64fe0b6bd846c52b04156780edbc138fbaf088fa1979d2e90d58a460366f0"} Sep 08 04:04:44.053265 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:44.053221 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-operator-56d489b4b8-xqxtv" podStartSLOduration=1.7008164479999999 podStartE2EDuration="4.053206451s" totalImagesPullingTime="2.352390003s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 04:04:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:04:44.052211754 +0000 UTC m=+543.673247723" watchObservedRunningTime="2026-09-08 04:04:44.053206451 +0000 UTC m=+543.674242418" Sep 08 04:04:57.734406 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:57.734365 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-k959d"] Sep 08 04:04:57.740936 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:57.740909 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-k959d" Sep 08 04:04:57.746107 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:57.746079 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 08 04:04:57.746804 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:57.746786 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 08 04:04:57.751053 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:57.751034 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-dk8d9\"" Sep 08 04:04:57.753415 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:57.753393 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-k959d"] Sep 08 04:04:57.797686 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:57.797487 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsgzg\" (UniqueName: \"kubernetes.io/projected/bc7d82a1-8ff6-433a-a8ca-fe709cc3bb48-kube-api-access-nsgzg\") pod \"cert-manager-webhook-6486ffd997-k959d\" (UID: \"bc7d82a1-8ff6-433a-a8ca-fe709cc3bb48\") " pod="cert-manager/cert-manager-webhook-6486ffd997-k959d" Sep 08 04:04:57.797686 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:57.797538 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc7d82a1-8ff6-433a-a8ca-fe709cc3bb48-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-k959d\" (UID: \"bc7d82a1-8ff6-433a-a8ca-fe709cc3bb48\") " pod="cert-manager/cert-manager-webhook-6486ffd997-k959d" Sep 08 04:04:57.898192 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:57.898158 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsgzg\" (UniqueName: \"kubernetes.io/projected/bc7d82a1-8ff6-433a-a8ca-fe709cc3bb48-kube-api-access-nsgzg\") pod \"cert-manager-webhook-6486ffd997-k959d\" (UID: \"bc7d82a1-8ff6-433a-a8ca-fe709cc3bb48\") " pod="cert-manager/cert-manager-webhook-6486ffd997-k959d" Sep 08 04:04:57.898192 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:57.898197 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc7d82a1-8ff6-433a-a8ca-fe709cc3bb48-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-k959d\" (UID: \"bc7d82a1-8ff6-433a-a8ca-fe709cc3bb48\") " pod="cert-manager/cert-manager-webhook-6486ffd997-k959d" Sep 08 04:04:57.910435 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:57.910396 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsgzg\" (UniqueName: \"kubernetes.io/projected/bc7d82a1-8ff6-433a-a8ca-fe709cc3bb48-kube-api-access-nsgzg\") pod \"cert-manager-webhook-6486ffd997-k959d\" (UID: \"bc7d82a1-8ff6-433a-a8ca-fe709cc3bb48\") " pod="cert-manager/cert-manager-webhook-6486ffd997-k959d" Sep 08 04:04:57.911662 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:57.911641 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc7d82a1-8ff6-433a-a8ca-fe709cc3bb48-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-k959d\" (UID: \"bc7d82a1-8ff6-433a-a8ca-fe709cc3bb48\") " pod="cert-manager/cert-manager-webhook-6486ffd997-k959d" Sep 08 04:04:58.050179 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:58.050081 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-k959d" Sep 08 04:04:58.325568 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:58.325481 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-k959d"] Sep 08 04:04:58.330202 ip-10-0-128-141 kubenswrapper[2579]: W0908 04:04:58.330175 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc7d82a1_8ff6_433a_a8ca_fe709cc3bb48.slice/crio-96020baa6b7e3a7ac1a35e998bb134ce1ae8c1a6dc1a9cdfcd77770a39a6674a WatchSource:0}: Error finding container 96020baa6b7e3a7ac1a35e998bb134ce1ae8c1a6dc1a9cdfcd77770a39a6674a: Status 404 returned error can't find the container with id 96020baa6b7e3a7ac1a35e998bb134ce1ae8c1a6dc1a9cdfcd77770a39a6674a Sep 08 04:04:59.091831 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:04:59.091799 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-k959d" event={"ID":"bc7d82a1-8ff6-433a-a8ca-fe709cc3bb48","Type":"ContainerStarted","Data":"96020baa6b7e3a7ac1a35e998bb134ce1ae8c1a6dc1a9cdfcd77770a39a6674a"} Sep 08 04:05:02.107728 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:02.107693 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-k959d" event={"ID":"bc7d82a1-8ff6-433a-a8ca-fe709cc3bb48","Type":"ContainerStarted","Data":"8b893b075e6b79029c952a0cdc6972ce831272229be1fc0f657e8986cabc6b77"} Sep 08 04:05:02.108130 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:02.107768 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-k959d" Sep 08 04:05:02.127650 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:02.127592 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-k959d" podStartSLOduration=1.914699573 podStartE2EDuration="5.127575048s" totalImagesPullingTime="3.212875475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 04:04:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:05:02.125042628 +0000 UTC m=+561.746078598" watchObservedRunningTime="2026-09-08 04:05:02.127575048 +0000 UTC m=+561.748611016" Sep 08 04:05:05.215019 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.214987 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7"] Sep 08 04:05:05.218435 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.218416 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7" Sep 08 04:05:05.224111 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.224082 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 08 04:05:05.224390 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.224371 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\"" Sep 08 04:05:05.224929 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.224793 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-wdq29\"" Sep 08 04:05:05.224929 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.224812 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\"" Sep 08 04:05:05.224929 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.224835 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 08 04:05:05.225258 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.225234 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\"" Sep 08 04:05:05.246030 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.245995 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7"] Sep 08 04:05:05.266674 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.266638 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8rcw\" (UniqueName: \"kubernetes.io/projected/00efd044-0152-4ad1-983f-85cc273f7cb3-kube-api-access-v8rcw\") pod \"opendatahub-operator-controller-manager-b8484c5b8-qfpx7\" (UID: \"00efd044-0152-4ad1-983f-85cc273f7cb3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7" Sep 08 04:05:05.266845 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.266702 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00efd044-0152-4ad1-983f-85cc273f7cb3-tmp\") pod \"opendatahub-operator-controller-manager-b8484c5b8-qfpx7\" (UID: \"00efd044-0152-4ad1-983f-85cc273f7cb3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7" Sep 08 04:05:05.266845 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.266772 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/00efd044-0152-4ad1-983f-85cc273f7cb3-metrics-cert\") pod \"opendatahub-operator-controller-manager-b8484c5b8-qfpx7\" (UID: \"00efd044-0152-4ad1-983f-85cc273f7cb3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7" Sep 08 04:05:05.266845 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.266802 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/00efd044-0152-4ad1-983f-85cc273f7cb3-webhook-cert\") pod \"opendatahub-operator-controller-manager-b8484c5b8-qfpx7\" (UID: \"00efd044-0152-4ad1-983f-85cc273f7cb3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7" Sep 08 04:05:05.266845 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.266839 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/00efd044-0152-4ad1-983f-85cc273f7cb3-manifests\") pod \"opendatahub-operator-controller-manager-b8484c5b8-qfpx7\" (UID: \"00efd044-0152-4ad1-983f-85cc273f7cb3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7" Sep 08 04:05:05.266995 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.266862 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/00efd044-0152-4ad1-983f-85cc273f7cb3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b8484c5b8-qfpx7\" (UID: \"00efd044-0152-4ad1-983f-85cc273f7cb3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7" Sep 08 04:05:05.367284 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.367247 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/00efd044-0152-4ad1-983f-85cc273f7cb3-metrics-cert\") pod \"opendatahub-operator-controller-manager-b8484c5b8-qfpx7\" (UID: \"00efd044-0152-4ad1-983f-85cc273f7cb3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7" Sep 08 04:05:05.367284 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.367287 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/00efd044-0152-4ad1-983f-85cc273f7cb3-webhook-cert\") pod \"opendatahub-operator-controller-manager-b8484c5b8-qfpx7\" (UID: \"00efd044-0152-4ad1-983f-85cc273f7cb3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7" Sep 08 04:05:05.367550 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.367329 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/00efd044-0152-4ad1-983f-85cc273f7cb3-manifests\") pod \"opendatahub-operator-controller-manager-b8484c5b8-qfpx7\" (UID: \"00efd044-0152-4ad1-983f-85cc273f7cb3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7" Sep 08 04:05:05.367550 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.367357 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/00efd044-0152-4ad1-983f-85cc273f7cb3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b8484c5b8-qfpx7\" (UID: \"00efd044-0152-4ad1-983f-85cc273f7cb3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7" Sep 08 04:05:05.367550 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.367403 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8rcw\" (UniqueName: \"kubernetes.io/projected/00efd044-0152-4ad1-983f-85cc273f7cb3-kube-api-access-v8rcw\") pod \"opendatahub-operator-controller-manager-b8484c5b8-qfpx7\" (UID: \"00efd044-0152-4ad1-983f-85cc273f7cb3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7" Sep 08 04:05:05.367550 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.367493 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00efd044-0152-4ad1-983f-85cc273f7cb3-tmp\") pod \"opendatahub-operator-controller-manager-b8484c5b8-qfpx7\" (UID: \"00efd044-0152-4ad1-983f-85cc273f7cb3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7" Sep 08 04:05:05.367975 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.367939 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/00efd044-0152-4ad1-983f-85cc273f7cb3-manifests\") pod \"opendatahub-operator-controller-manager-b8484c5b8-qfpx7\" (UID: \"00efd044-0152-4ad1-983f-85cc273f7cb3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7" Sep 08 04:05:05.369823 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.369791 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00efd044-0152-4ad1-983f-85cc273f7cb3-tmp\") pod \"opendatahub-operator-controller-manager-b8484c5b8-qfpx7\" (UID: \"00efd044-0152-4ad1-983f-85cc273f7cb3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7" Sep 08 04:05:05.370037 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.370020 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/00efd044-0152-4ad1-983f-85cc273f7cb3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b8484c5b8-qfpx7\" (UID: \"00efd044-0152-4ad1-983f-85cc273f7cb3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7" Sep 08 04:05:05.370142 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.370124 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/00efd044-0152-4ad1-983f-85cc273f7cb3-webhook-cert\") pod \"opendatahub-operator-controller-manager-b8484c5b8-qfpx7\" (UID: \"00efd044-0152-4ad1-983f-85cc273f7cb3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7" Sep 08 04:05:05.370189 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.370169 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/00efd044-0152-4ad1-983f-85cc273f7cb3-metrics-cert\") pod \"opendatahub-operator-controller-manager-b8484c5b8-qfpx7\" (UID: \"00efd044-0152-4ad1-983f-85cc273f7cb3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7" Sep 08 04:05:05.380201 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.380170 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8rcw\" (UniqueName: \"kubernetes.io/projected/00efd044-0152-4ad1-983f-85cc273f7cb3-kube-api-access-v8rcw\") pod \"opendatahub-operator-controller-manager-b8484c5b8-qfpx7\" (UID: \"00efd044-0152-4ad1-983f-85cc273f7cb3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7" Sep 08 04:05:05.529245 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.529157 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7" Sep 08 04:05:05.813742 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.813715 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7"] Sep 08 04:05:05.816422 ip-10-0-128-141 kubenswrapper[2579]: W0908 04:05:05.816396 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00efd044_0152_4ad1_983f_85cc273f7cb3.slice/crio-f495a194d2c00440d6f8934389d73126284dc514d5f129ab4a9e75106ef40951 WatchSource:0}: Error finding container f495a194d2c00440d6f8934389d73126284dc514d5f129ab4a9e75106ef40951: Status 404 returned error can't find the container with id f495a194d2c00440d6f8934389d73126284dc514d5f129ab4a9e75106ef40951 Sep 08 04:05:05.886643 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.886610 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-g7cmb"] Sep 08 04:05:05.891152 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.891134 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-g7cmb" Sep 08 04:05:05.893989 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.893966 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-z55bw\"" Sep 08 04:05:05.901594 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.901569 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-g7cmb"] Sep 08 04:05:05.973986 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.973955 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/04168ccc-a3f9-4cd9-b7fb-caa9cc602bc3-bound-sa-token\") pod \"cert-manager-55b4d954d5-g7cmb\" (UID: \"04168ccc-a3f9-4cd9-b7fb-caa9cc602bc3\") " pod="cert-manager/cert-manager-55b4d954d5-g7cmb" Sep 08 04:05:05.974162 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:05.974020 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctwnf\" (UniqueName: \"kubernetes.io/projected/04168ccc-a3f9-4cd9-b7fb-caa9cc602bc3-kube-api-access-ctwnf\") pod \"cert-manager-55b4d954d5-g7cmb\" (UID: \"04168ccc-a3f9-4cd9-b7fb-caa9cc602bc3\") " pod="cert-manager/cert-manager-55b4d954d5-g7cmb" Sep 08 04:05:06.074881 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:06.074782 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctwnf\" (UniqueName: \"kubernetes.io/projected/04168ccc-a3f9-4cd9-b7fb-caa9cc602bc3-kube-api-access-ctwnf\") pod \"cert-manager-55b4d954d5-g7cmb\" (UID: \"04168ccc-a3f9-4cd9-b7fb-caa9cc602bc3\") " pod="cert-manager/cert-manager-55b4d954d5-g7cmb" Sep 08 04:05:06.075052 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:06.074949 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/04168ccc-a3f9-4cd9-b7fb-caa9cc602bc3-bound-sa-token\") pod \"cert-manager-55b4d954d5-g7cmb\" (UID: \"04168ccc-a3f9-4cd9-b7fb-caa9cc602bc3\") " pod="cert-manager/cert-manager-55b4d954d5-g7cmb" Sep 08 04:05:06.084234 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:06.084195 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/04168ccc-a3f9-4cd9-b7fb-caa9cc602bc3-bound-sa-token\") pod \"cert-manager-55b4d954d5-g7cmb\" (UID: \"04168ccc-a3f9-4cd9-b7fb-caa9cc602bc3\") " pod="cert-manager/cert-manager-55b4d954d5-g7cmb" Sep 08 04:05:06.084378 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:06.084284 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctwnf\" (UniqueName: \"kubernetes.io/projected/04168ccc-a3f9-4cd9-b7fb-caa9cc602bc3-kube-api-access-ctwnf\") pod \"cert-manager-55b4d954d5-g7cmb\" (UID: \"04168ccc-a3f9-4cd9-b7fb-caa9cc602bc3\") " pod="cert-manager/cert-manager-55b4d954d5-g7cmb" Sep 08 04:05:06.132901 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:06.132864 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7" event={"ID":"00efd044-0152-4ad1-983f-85cc273f7cb3","Type":"ContainerStarted","Data":"f495a194d2c00440d6f8934389d73126284dc514d5f129ab4a9e75106ef40951"} Sep 08 04:05:06.201182 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:06.201145 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-g7cmb" Sep 08 04:05:06.476472 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:06.476405 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-g7cmb"] Sep 08 04:05:06.480978 ip-10-0-128-141 kubenswrapper[2579]: W0908 04:05:06.480949 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04168ccc_a3f9_4cd9_b7fb_caa9cc602bc3.slice/crio-7f5ec557e5079949fc5c48904133f979bbc79f84ca6ddd05535e544d3513a575 WatchSource:0}: Error finding container 7f5ec557e5079949fc5c48904133f979bbc79f84ca6ddd05535e544d3513a575: Status 404 returned error can't find the container with id 7f5ec557e5079949fc5c48904133f979bbc79f84ca6ddd05535e544d3513a575 Sep 08 04:05:07.141005 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:07.140660 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-g7cmb" event={"ID":"04168ccc-a3f9-4cd9-b7fb-caa9cc602bc3","Type":"ContainerStarted","Data":"858e263f9c8fb0d75815261fc46f519ad30ac03c977bdceaa8b7cac8fcf2dfce"} Sep 08 04:05:07.141005 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:07.140709 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-g7cmb" event={"ID":"04168ccc-a3f9-4cd9-b7fb-caa9cc602bc3","Type":"ContainerStarted","Data":"7f5ec557e5079949fc5c48904133f979bbc79f84ca6ddd05535e544d3513a575"} Sep 08 04:05:07.169618 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:07.169554 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-g7cmb" podStartSLOduration=2.169531132 podStartE2EDuration="2.169531132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 04:05:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:05:07.167339414 +0000 UTC m=+566.788375381" watchObservedRunningTime="2026-09-08 04:05:07.169531132 +0000 UTC m=+566.790567100" Sep 08 04:05:08.114613 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:08.114580 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-k959d" Sep 08 04:05:09.152388 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:09.152353 2579 generic.go:356] "Generic (PLEG): container finished" podID="00efd044-0152-4ad1-983f-85cc273f7cb3" containerID="f417d380ca677d744af73688bb94cdce6893668728153bec9907768786768f6e" exitCode=0 Sep 08 04:05:09.152737 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:09.152424 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7" event={"ID":"00efd044-0152-4ad1-983f-85cc273f7cb3","Type":"ContainerDied","Data":"f417d380ca677d744af73688bb94cdce6893668728153bec9907768786768f6e"} Sep 08 04:05:10.159889 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:10.159854 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7" event={"ID":"00efd044-0152-4ad1-983f-85cc273f7cb3","Type":"ContainerStarted","Data":"97e3dc34d820f54e97c5874c1e7e3be21d5e72f97530de966887d4506774fdc3"} Sep 08 04:05:10.160281 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:10.159973 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7" Sep 08 04:05:13.268002 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:13.267940 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7" podStartSLOduration=4.6523107679999995 podStartE2EDuration="8.267921173s" totalImagesPullingTime="3.206826449s" totalInitContainerRuntime="408.783956ms" isStatefulPod=false podCreationTimestamp="2026-09-08 04:05:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:05:10.188521252 +0000 UTC m=+569.809557219" watchObservedRunningTime="2026-09-08 04:05:13.267921173 +0000 UTC m=+572.888957140" Sep 08 04:05:13.268506 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:13.268307 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-5c6ll"] Sep 08 04:05:13.271893 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:13.271869 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-5c6ll" Sep 08 04:05:13.274879 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:13.274846 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-cvpm4\"" Sep 08 04:05:13.275100 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:13.275085 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Sep 08 04:05:13.276089 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:13.276073 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 08 04:05:13.276089 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:13.276084 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 08 04:05:13.276219 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:13.276120 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Sep 08 04:05:13.280144 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:13.280118 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-5c6ll"] Sep 08 04:05:13.345686 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:13.345646 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a643c680-e2b7-489b-b200-4bbf60097793-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-5c6ll\" (UID: \"a643c680-e2b7-489b-b200-4bbf60097793\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-5c6ll" Sep 08 04:05:13.345867 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:13.345698 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a643c680-e2b7-489b-b200-4bbf60097793-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-5c6ll\" (UID: \"a643c680-e2b7-489b-b200-4bbf60097793\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-5c6ll" Sep 08 04:05:13.345867 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:13.345717 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t68j\" (UniqueName: \"kubernetes.io/projected/a643c680-e2b7-489b-b200-4bbf60097793-kube-api-access-4t68j\") pod \"kuadrant-console-plugin-598f86468b-5c6ll\" (UID: \"a643c680-e2b7-489b-b200-4bbf60097793\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-5c6ll" Sep 08 04:05:13.446134 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:13.446092 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a643c680-e2b7-489b-b200-4bbf60097793-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-5c6ll\" (UID: \"a643c680-e2b7-489b-b200-4bbf60097793\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-5c6ll" Sep 08 04:05:13.446134 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:13.446141 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a643c680-e2b7-489b-b200-4bbf60097793-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-5c6ll\" (UID: \"a643c680-e2b7-489b-b200-4bbf60097793\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-5c6ll" Sep 08 04:05:13.446388 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:13.446162 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t68j\" (UniqueName: \"kubernetes.io/projected/a643c680-e2b7-489b-b200-4bbf60097793-kube-api-access-4t68j\") pod \"kuadrant-console-plugin-598f86468b-5c6ll\" (UID: \"a643c680-e2b7-489b-b200-4bbf60097793\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-5c6ll" Sep 08 04:05:13.446894 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:13.446873 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a643c680-e2b7-489b-b200-4bbf60097793-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-5c6ll\" (UID: \"a643c680-e2b7-489b-b200-4bbf60097793\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-5c6ll" Sep 08 04:05:13.448573 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:13.448553 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a643c680-e2b7-489b-b200-4bbf60097793-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-5c6ll\" (UID: \"a643c680-e2b7-489b-b200-4bbf60097793\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-5c6ll" Sep 08 04:05:13.457663 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:13.457643 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t68j\" (UniqueName: \"kubernetes.io/projected/a643c680-e2b7-489b-b200-4bbf60097793-kube-api-access-4t68j\") pod \"kuadrant-console-plugin-598f86468b-5c6ll\" (UID: \"a643c680-e2b7-489b-b200-4bbf60097793\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-5c6ll" Sep 08 04:05:13.583006 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:13.582921 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-5c6ll" Sep 08 04:05:13.855905 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:13.855839 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-5c6ll"] Sep 08 04:05:13.858298 ip-10-0-128-141 kubenswrapper[2579]: W0908 04:05:13.858274 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda643c680_e2b7_489b_b200_4bbf60097793.slice/crio-f92b50c2d6d4bae4d2183524f5e7fb8343a821f05f8c8a67f62bfce50c97b872 WatchSource:0}: Error finding container f92b50c2d6d4bae4d2183524f5e7fb8343a821f05f8c8a67f62bfce50c97b872: Status 404 returned error can't find the container with id f92b50c2d6d4bae4d2183524f5e7fb8343a821f05f8c8a67f62bfce50c97b872 Sep 08 04:05:14.178367 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:14.178336 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-5c6ll" event={"ID":"a643c680-e2b7-489b-b200-4bbf60097793","Type":"ContainerStarted","Data":"f92b50c2d6d4bae4d2183524f5e7fb8343a821f05f8c8a67f62bfce50c97b872"} Sep 08 04:05:19.206782 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:19.206701 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-5c6ll" event={"ID":"a643c680-e2b7-489b-b200-4bbf60097793","Type":"ContainerStarted","Data":"a8f382919cd59a2972d659ac7712eb65916549ad325210f15659d63d7908e218"} Sep 08 04:05:19.228049 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:19.228002 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-5c6ll" podStartSLOduration=1.206682674 podStartE2EDuration="6.227986224s" totalImagesPullingTime="5.02130355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 04:05:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:05:19.226146283 +0000 UTC m=+578.847182250" watchObservedRunningTime="2026-09-08 04:05:19.227986224 +0000 UTC m=+578.849022191" Sep 08 04:05:21.166707 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:21.166673 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-b8484c5b8-qfpx7" Sep 08 04:05:40.814175 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:40.814145 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tg7hb_45ae9afd-2dea-43b8-aa31-169b11e60fd2/ovn-acl-logging/0.log" Sep 08 04:05:40.814710 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:40.814561 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tg7hb_45ae9afd-2dea-43b8-aa31-169b11e60fd2/ovn-acl-logging/0.log" Sep 08 04:05:41.744594 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:41.744556 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-65bcfbd747-vzxhm"] Sep 08 04:05:41.747184 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:41.747160 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65bcfbd747-vzxhm" Sep 08 04:05:41.750406 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:41.750379 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-gf64n\"" Sep 08 04:05:41.750682 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:41.750663 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 08 04:05:41.750960 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:41.750943 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 08 04:05:41.766092 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:41.766059 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65bcfbd747-vzxhm"] Sep 08 04:05:41.779015 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:41.778979 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef20b471-07ea-4943-98b3-17ebf9a36b06-tls-certs\") pod \"kube-auth-proxy-65bcfbd747-vzxhm\" (UID: \"ef20b471-07ea-4943-98b3-17ebf9a36b06\") " pod="openshift-ingress/kube-auth-proxy-65bcfbd747-vzxhm" Sep 08 04:05:41.779345 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:41.779319 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbxjj\" (UniqueName: \"kubernetes.io/projected/ef20b471-07ea-4943-98b3-17ebf9a36b06-kube-api-access-dbxjj\") pod \"kube-auth-proxy-65bcfbd747-vzxhm\" (UID: \"ef20b471-07ea-4943-98b3-17ebf9a36b06\") " pod="openshift-ingress/kube-auth-proxy-65bcfbd747-vzxhm" Sep 08 04:05:41.779528 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:41.779511 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef20b471-07ea-4943-98b3-17ebf9a36b06-tmp\") pod \"kube-auth-proxy-65bcfbd747-vzxhm\" (UID: \"ef20b471-07ea-4943-98b3-17ebf9a36b06\") " pod="openshift-ingress/kube-auth-proxy-65bcfbd747-vzxhm" Sep 08 04:05:41.880886 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:41.880846 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbxjj\" (UniqueName: \"kubernetes.io/projected/ef20b471-07ea-4943-98b3-17ebf9a36b06-kube-api-access-dbxjj\") pod \"kube-auth-proxy-65bcfbd747-vzxhm\" (UID: \"ef20b471-07ea-4943-98b3-17ebf9a36b06\") " pod="openshift-ingress/kube-auth-proxy-65bcfbd747-vzxhm" Sep 08 04:05:41.881279 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:41.880920 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef20b471-07ea-4943-98b3-17ebf9a36b06-tmp\") pod \"kube-auth-proxy-65bcfbd747-vzxhm\" (UID: \"ef20b471-07ea-4943-98b3-17ebf9a36b06\") " pod="openshift-ingress/kube-auth-proxy-65bcfbd747-vzxhm" Sep 08 04:05:41.881279 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:41.880966 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef20b471-07ea-4943-98b3-17ebf9a36b06-tls-certs\") pod \"kube-auth-proxy-65bcfbd747-vzxhm\" (UID: \"ef20b471-07ea-4943-98b3-17ebf9a36b06\") " pod="openshift-ingress/kube-auth-proxy-65bcfbd747-vzxhm" Sep 08 04:05:41.881279 ip-10-0-128-141 kubenswrapper[2579]: E0908 04:05:41.881146 2579 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Sep 08 04:05:41.881447 ip-10-0-128-141 kubenswrapper[2579]: E0908 04:05:41.881353 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef20b471-07ea-4943-98b3-17ebf9a36b06-tls-certs podName:ef20b471-07ea-4943-98b3-17ebf9a36b06 nodeName:}" failed. No retries permitted until 2026-09-08 04:05:42.38130842 +0000 UTC m=+602.002344375 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ef20b471-07ea-4943-98b3-17ebf9a36b06-tls-certs") pod "kube-auth-proxy-65bcfbd747-vzxhm" (UID: "ef20b471-07ea-4943-98b3-17ebf9a36b06") : secret "kube-auth-proxy-tls" not found Sep 08 04:05:41.883504 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:41.883441 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef20b471-07ea-4943-98b3-17ebf9a36b06-tmp\") pod \"kube-auth-proxy-65bcfbd747-vzxhm\" (UID: \"ef20b471-07ea-4943-98b3-17ebf9a36b06\") " pod="openshift-ingress/kube-auth-proxy-65bcfbd747-vzxhm" Sep 08 04:05:41.897834 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:41.897801 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbxjj\" (UniqueName: \"kubernetes.io/projected/ef20b471-07ea-4943-98b3-17ebf9a36b06-kube-api-access-dbxjj\") pod \"kube-auth-proxy-65bcfbd747-vzxhm\" (UID: \"ef20b471-07ea-4943-98b3-17ebf9a36b06\") " pod="openshift-ingress/kube-auth-proxy-65bcfbd747-vzxhm" Sep 08 04:05:42.322396 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:42.322363 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-l8z4q"] Sep 08 04:05:42.325263 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:42.325240 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-l8z4q" Sep 08 04:05:42.332342 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:42.332318 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Sep 08 04:05:42.346485 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:42.346292 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-l8z4q"] Sep 08 04:05:42.387768 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:42.387738 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef20b471-07ea-4943-98b3-17ebf9a36b06-tls-certs\") pod \"kube-auth-proxy-65bcfbd747-vzxhm\" (UID: \"ef20b471-07ea-4943-98b3-17ebf9a36b06\") " pod="openshift-ingress/kube-auth-proxy-65bcfbd747-vzxhm" Sep 08 04:05:42.387960 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:42.387779 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7634c361-743e-416a-8179-639c86ce75e5-config-file\") pod \"limitador-limitador-859c4d96d6-l8z4q\" (UID: \"7634c361-743e-416a-8179-639c86ce75e5\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-l8z4q" Sep 08 04:05:42.387960 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:42.387807 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g86f2\" (UniqueName: \"kubernetes.io/projected/7634c361-743e-416a-8179-639c86ce75e5-kube-api-access-g86f2\") pod \"limitador-limitador-859c4d96d6-l8z4q\" (UID: \"7634c361-743e-416a-8179-639c86ce75e5\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-l8z4q" Sep 08 04:05:42.390601 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:42.390572 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef20b471-07ea-4943-98b3-17ebf9a36b06-tls-certs\") pod \"kube-auth-proxy-65bcfbd747-vzxhm\" (UID: \"ef20b471-07ea-4943-98b3-17ebf9a36b06\") " pod="openshift-ingress/kube-auth-proxy-65bcfbd747-vzxhm" Sep 08 04:05:42.437287 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:42.437231 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-l8z4q"] Sep 08 04:05:42.488538 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:42.488494 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7634c361-743e-416a-8179-639c86ce75e5-config-file\") pod \"limitador-limitador-859c4d96d6-l8z4q\" (UID: \"7634c361-743e-416a-8179-639c86ce75e5\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-l8z4q" Sep 08 04:05:42.488538 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:42.488542 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g86f2\" (UniqueName: \"kubernetes.io/projected/7634c361-743e-416a-8179-639c86ce75e5-kube-api-access-g86f2\") pod \"limitador-limitador-859c4d96d6-l8z4q\" (UID: \"7634c361-743e-416a-8179-639c86ce75e5\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-l8z4q" Sep 08 04:05:42.489200 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:42.489176 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7634c361-743e-416a-8179-639c86ce75e5-config-file\") pod \"limitador-limitador-859c4d96d6-l8z4q\" (UID: \"7634c361-743e-416a-8179-639c86ce75e5\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-l8z4q" Sep 08 04:05:42.520586 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:42.520551 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g86f2\" (UniqueName: \"kubernetes.io/projected/7634c361-743e-416a-8179-639c86ce75e5-kube-api-access-g86f2\") pod \"limitador-limitador-859c4d96d6-l8z4q\" (UID: \"7634c361-743e-416a-8179-639c86ce75e5\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-l8z4q" Sep 08 04:05:42.639573 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:42.639535 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-l8z4q" Sep 08 04:05:42.661111 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:42.661072 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65bcfbd747-vzxhm" Sep 08 04:05:42.933327 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:42.933258 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-l8z4q"] Sep 08 04:05:42.935594 ip-10-0-128-141 kubenswrapper[2579]: W0908 04:05:42.935567 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7634c361_743e_416a_8179_639c86ce75e5.slice/crio-215e6b4ca12c0758fc0267f90816fae8b53799f0a26e7b432f767f2d22126d5e WatchSource:0}: Error finding container 215e6b4ca12c0758fc0267f90816fae8b53799f0a26e7b432f767f2d22126d5e: Status 404 returned error can't find the container with id 215e6b4ca12c0758fc0267f90816fae8b53799f0a26e7b432f767f2d22126d5e Sep 08 04:05:42.972274 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:42.972231 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65bcfbd747-vzxhm"] Sep 08 04:05:42.975397 ip-10-0-128-141 kubenswrapper[2579]: W0908 04:05:42.975363 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef20b471_07ea_4943_98b3_17ebf9a36b06.slice/crio-c50a2f2262f845a095a6040c988619d7e3f67125737abd64ec34a6e0db66fe2d WatchSource:0}: Error finding container c50a2f2262f845a095a6040c988619d7e3f67125737abd64ec34a6e0db66fe2d: Status 404 returned error can't find the container with id c50a2f2262f845a095a6040c988619d7e3f67125737abd64ec34a6e0db66fe2d Sep 08 04:05:43.041498 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:43.041444 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-fhcmn"] Sep 08 04:05:43.044538 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:43.044517 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-fhcmn" Sep 08 04:05:43.056561 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:43.056532 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-fhcmn"] Sep 08 04:05:43.090678 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:43.090639 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-fhcmn"] Sep 08 04:05:43.095079 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:43.095047 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k68x8\" (UniqueName: \"kubernetes.io/projected/3e6aeb77-f443-4bce-8b72-df3687476747-kube-api-access-k68x8\") pod \"limitador-limitador-85d9db4d5d-fhcmn\" (UID: \"3e6aeb77-f443-4bce-8b72-df3687476747\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-fhcmn" Sep 08 04:05:43.095184 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:43.095106 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3e6aeb77-f443-4bce-8b72-df3687476747-config-file\") pod \"limitador-limitador-85d9db4d5d-fhcmn\" (UID: \"3e6aeb77-f443-4bce-8b72-df3687476747\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-fhcmn" Sep 08 04:05:43.196684 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:43.196602 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k68x8\" (UniqueName: \"kubernetes.io/projected/3e6aeb77-f443-4bce-8b72-df3687476747-kube-api-access-k68x8\") pod \"limitador-limitador-85d9db4d5d-fhcmn\" (UID: \"3e6aeb77-f443-4bce-8b72-df3687476747\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-fhcmn" Sep 08 04:05:43.196684 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:43.196647 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3e6aeb77-f443-4bce-8b72-df3687476747-config-file\") pod \"limitador-limitador-85d9db4d5d-fhcmn\" (UID: \"3e6aeb77-f443-4bce-8b72-df3687476747\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-fhcmn" Sep 08 04:05:43.197251 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:43.197232 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3e6aeb77-f443-4bce-8b72-df3687476747-config-file\") pod \"limitador-limitador-85d9db4d5d-fhcmn\" (UID: \"3e6aeb77-f443-4bce-8b72-df3687476747\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-fhcmn" Sep 08 04:05:43.205557 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:43.205520 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k68x8\" (UniqueName: \"kubernetes.io/projected/3e6aeb77-f443-4bce-8b72-df3687476747-kube-api-access-k68x8\") pod \"limitador-limitador-85d9db4d5d-fhcmn\" (UID: \"3e6aeb77-f443-4bce-8b72-df3687476747\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-fhcmn" Sep 08 04:05:43.326110 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:43.326070 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-l8z4q" event={"ID":"7634c361-743e-416a-8179-639c86ce75e5","Type":"ContainerStarted","Data":"215e6b4ca12c0758fc0267f90816fae8b53799f0a26e7b432f767f2d22126d5e"} Sep 08 04:05:43.327224 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:43.327199 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65bcfbd747-vzxhm" event={"ID":"ef20b471-07ea-4943-98b3-17ebf9a36b06","Type":"ContainerStarted","Data":"c50a2f2262f845a095a6040c988619d7e3f67125737abd64ec34a6e0db66fe2d"} Sep 08 04:05:43.356615 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:43.356579 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-fhcmn" Sep 08 04:05:43.672646 ip-10-0-128-141 kubenswrapper[2579]: W0908 04:05:43.672612 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e6aeb77_f443_4bce_8b72_df3687476747.slice/crio-2026852ea6c723814417415b4ac1b8e97d3877fae0523fe6126ecb688dfd5fb2 WatchSource:0}: Error finding container 2026852ea6c723814417415b4ac1b8e97d3877fae0523fe6126ecb688dfd5fb2: Status 404 returned error can't find the container with id 2026852ea6c723814417415b4ac1b8e97d3877fae0523fe6126ecb688dfd5fb2 Sep 08 04:05:43.675042 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:43.675018 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-fhcmn"] Sep 08 04:05:44.340225 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:44.340186 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-fhcmn" event={"ID":"3e6aeb77-f443-4bce-8b72-df3687476747","Type":"ContainerStarted","Data":"2026852ea6c723814417415b4ac1b8e97d3877fae0523fe6126ecb688dfd5fb2"} Sep 08 04:05:47.360870 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:47.360831 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-fhcmn" event={"ID":"3e6aeb77-f443-4bce-8b72-df3687476747","Type":"ContainerStarted","Data":"7fbfd093e7614c0b42a760000dd16c45e262437f2c98ce5b8184df525edb0787"} Sep 08 04:05:47.361345 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:47.360978 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-fhcmn" Sep 08 04:05:47.362677 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:47.362651 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-l8z4q" event={"ID":"7634c361-743e-416a-8179-639c86ce75e5","Type":"ContainerStarted","Data":"8277d816ef17c37c341da28df23620109627324b1b14a3b099c69a848a07f727"} Sep 08 04:05:47.362798 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:47.362718 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-l8z4q" Sep 08 04:05:47.364250 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:47.364229 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65bcfbd747-vzxhm" event={"ID":"ef20b471-07ea-4943-98b3-17ebf9a36b06","Type":"ContainerStarted","Data":"f3f3fedf2400100a77f41f02a672d9f0784b09301256e20e1c4d9abdeadc905f"} Sep 08 04:05:47.379970 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:47.379892 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-85d9db4d5d-fhcmn" podStartSLOduration=1.273111223 podStartE2EDuration="4.379870572s" totalImagesPullingTime="3.106759349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 04:05:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:05:47.376589654 +0000 UTC m=+606.997625633" watchObservedRunningTime="2026-09-08 04:05:47.379870572 +0000 UTC m=+607.000906540" Sep 08 04:05:47.394584 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:47.394513 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-65bcfbd747-vzxhm" podStartSLOduration=2.636248703 podStartE2EDuration="6.394494286s" totalImagesPullingTime="3.758245583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 04:05:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:05:47.393584012 +0000 UTC m=+607.014619982" watchObservedRunningTime="2026-09-08 04:05:47.394494286 +0000 UTC m=+607.015530254" Sep 08 04:05:47.412444 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:47.412385 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-l8z4q" podStartSLOduration=1.573421306 podStartE2EDuration="5.41236629s" totalImagesPullingTime="3.838944984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 04:05:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:05:47.410921262 +0000 UTC m=+607.031957232" watchObservedRunningTime="2026-09-08 04:05:47.41236629 +0000 UTC m=+607.033402258" Sep 08 04:05:58.370652 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:58.370620 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-fhcmn" Sep 08 04:05:58.371143 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:58.370692 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-l8z4q" Sep 08 04:05:58.428774 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:58.428741 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-l8z4q"] Sep 08 04:05:58.428995 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:58.428968 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-l8z4q" podUID="7634c361-743e-416a-8179-639c86ce75e5" containerName="limitador" containerID="cri-o://8277d816ef17c37c341da28df23620109627324b1b14a3b099c69a848a07f727" gracePeriod=30 Sep 08 04:05:59.371645 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:59.371622 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-l8z4q" Sep 08 04:05:59.423521 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:59.423487 2579 generic.go:356] "Generic (PLEG): container finished" podID="7634c361-743e-416a-8179-639c86ce75e5" containerID="8277d816ef17c37c341da28df23620109627324b1b14a3b099c69a848a07f727" exitCode=0 Sep 08 04:05:59.423693 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:59.423579 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-l8z4q" Sep 08 04:05:59.423693 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:59.423616 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-l8z4q" event={"ID":"7634c361-743e-416a-8179-639c86ce75e5","Type":"ContainerDied","Data":"8277d816ef17c37c341da28df23620109627324b1b14a3b099c69a848a07f727"} Sep 08 04:05:59.423693 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:59.423663 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-l8z4q" event={"ID":"7634c361-743e-416a-8179-639c86ce75e5","Type":"ContainerDied","Data":"215e6b4ca12c0758fc0267f90816fae8b53799f0a26e7b432f767f2d22126d5e"} Sep 08 04:05:59.423693 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:59.423688 2579 scope.go:117] "RemoveContainer" containerID="8277d816ef17c37c341da28df23620109627324b1b14a3b099c69a848a07f727" Sep 08 04:05:59.431784 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:59.431653 2579 scope.go:117] "RemoveContainer" containerID="8277d816ef17c37c341da28df23620109627324b1b14a3b099c69a848a07f727" Sep 08 04:05:59.431936 ip-10-0-128-141 kubenswrapper[2579]: E0908 04:05:59.431917 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8277d816ef17c37c341da28df23620109627324b1b14a3b099c69a848a07f727\": container with ID starting with 8277d816ef17c37c341da28df23620109627324b1b14a3b099c69a848a07f727 not found: ID does not exist" containerID="8277d816ef17c37c341da28df23620109627324b1b14a3b099c69a848a07f727" Sep 08 04:05:59.431983 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:59.431943 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8277d816ef17c37c341da28df23620109627324b1b14a3b099c69a848a07f727"} err="failed to get container status \"8277d816ef17c37c341da28df23620109627324b1b14a3b099c69a848a07f727\": rpc error: code = NotFound desc = could not find container \"8277d816ef17c37c341da28df23620109627324b1b14a3b099c69a848a07f727\": container with ID starting with 8277d816ef17c37c341da28df23620109627324b1b14a3b099c69a848a07f727 not found: ID does not exist" Sep 08 04:05:59.457691 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:59.457668 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7634c361-743e-416a-8179-639c86ce75e5-config-file\") pod \"7634c361-743e-416a-8179-639c86ce75e5\" (UID: \"7634c361-743e-416a-8179-639c86ce75e5\") " Sep 08 04:05:59.457786 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:59.457719 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g86f2\" (UniqueName: \"kubernetes.io/projected/7634c361-743e-416a-8179-639c86ce75e5-kube-api-access-g86f2\") pod \"7634c361-743e-416a-8179-639c86ce75e5\" (UID: \"7634c361-743e-416a-8179-639c86ce75e5\") " Sep 08 04:05:59.457984 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:59.457964 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7634c361-743e-416a-8179-639c86ce75e5-config-file" (OuterVolumeSpecName: "config-file") pod "7634c361-743e-416a-8179-639c86ce75e5" (UID: "7634c361-743e-416a-8179-639c86ce75e5"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 04:05:59.459725 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:59.459706 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7634c361-743e-416a-8179-639c86ce75e5-kube-api-access-g86f2" (OuterVolumeSpecName: "kube-api-access-g86f2") pod "7634c361-743e-416a-8179-639c86ce75e5" (UID: "7634c361-743e-416a-8179-639c86ce75e5"). InnerVolumeSpecName "kube-api-access-g86f2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 04:05:59.559341 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:59.559279 2579 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7634c361-743e-416a-8179-639c86ce75e5-config-file\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:05:59.559341 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:59.559302 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g86f2\" (UniqueName: \"kubernetes.io/projected/7634c361-743e-416a-8179-639c86ce75e5-kube-api-access-g86f2\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:05:59.744650 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:59.744621 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-l8z4q"] Sep 08 04:05:59.750161 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:05:59.750133 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-l8z4q"] Sep 08 04:06:00.875386 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:00.875351 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7634c361-743e-416a-8179-639c86ce75e5" path="/var/lib/kubelet/pods/7634c361-743e-416a-8179-639c86ce75e5/volumes" Sep 08 04:06:01.726726 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:01.726696 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7848477867-gnggp"] Sep 08 04:06:01.727070 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:01.727056 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7634c361-743e-416a-8179-639c86ce75e5" containerName="limitador" Sep 08 04:06:01.727133 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:01.727071 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7634c361-743e-416a-8179-639c86ce75e5" containerName="limitador" Sep 08 04:06:01.727185 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:01.727135 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7634c361-743e-416a-8179-639c86ce75e5" containerName="limitador" Sep 08 04:06:01.731247 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:01.731218 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7848477867-gnggp" Sep 08 04:06:01.734611 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:01.734590 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 08 04:06:01.735964 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:01.735945 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 08 04:06:01.736089 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:01.735968 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-gljz6\"" Sep 08 04:06:01.736089 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:01.736004 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 08 04:06:01.736089 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:01.735945 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 08 04:06:01.736089 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:01.735946 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 08 04:06:01.758967 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:01.758935 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7848477867-gnggp"] Sep 08 04:06:01.879471 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:01.879421 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0c9dbbb3-3427-4dbe-bf98-87bc3cd984be-webhook-cert\") pod \"maas-controller-7848477867-gnggp\" (UID: \"0c9dbbb3-3427-4dbe-bf98-87bc3cd984be\") " pod="opendatahub/maas-controller-7848477867-gnggp" Sep 08 04:06:01.879870 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:01.879515 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c9dbbb3-3427-4dbe-bf98-87bc3cd984be-metrics-certs\") pod \"maas-controller-7848477867-gnggp\" (UID: \"0c9dbbb3-3427-4dbe-bf98-87bc3cd984be\") " pod="opendatahub/maas-controller-7848477867-gnggp" Sep 08 04:06:01.879870 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:01.879563 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4fdq\" (UniqueName: \"kubernetes.io/projected/0c9dbbb3-3427-4dbe-bf98-87bc3cd984be-kube-api-access-m4fdq\") pod \"maas-controller-7848477867-gnggp\" (UID: \"0c9dbbb3-3427-4dbe-bf98-87bc3cd984be\") " pod="opendatahub/maas-controller-7848477867-gnggp" Sep 08 04:06:01.980303 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:01.980212 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0c9dbbb3-3427-4dbe-bf98-87bc3cd984be-webhook-cert\") pod \"maas-controller-7848477867-gnggp\" (UID: \"0c9dbbb3-3427-4dbe-bf98-87bc3cd984be\") " pod="opendatahub/maas-controller-7848477867-gnggp" Sep 08 04:06:01.980303 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:01.980302 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c9dbbb3-3427-4dbe-bf98-87bc3cd984be-metrics-certs\") pod \"maas-controller-7848477867-gnggp\" (UID: \"0c9dbbb3-3427-4dbe-bf98-87bc3cd984be\") " pod="opendatahub/maas-controller-7848477867-gnggp" Sep 08 04:06:01.980549 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:01.980343 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4fdq\" (UniqueName: \"kubernetes.io/projected/0c9dbbb3-3427-4dbe-bf98-87bc3cd984be-kube-api-access-m4fdq\") pod \"maas-controller-7848477867-gnggp\" (UID: \"0c9dbbb3-3427-4dbe-bf98-87bc3cd984be\") " pod="opendatahub/maas-controller-7848477867-gnggp" Sep 08 04:06:01.981096 ip-10-0-128-141 kubenswrapper[2579]: E0908 04:06:01.981070 2579 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found Sep 08 04:06:01.981212 ip-10-0-128-141 kubenswrapper[2579]: E0908 04:06:01.981140 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c9dbbb3-3427-4dbe-bf98-87bc3cd984be-metrics-certs podName:0c9dbbb3-3427-4dbe-bf98-87bc3cd984be nodeName:}" failed. No retries permitted until 2026-09-08 04:06:02.481120771 +0000 UTC m=+622.102156736 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0c9dbbb3-3427-4dbe-bf98-87bc3cd984be-metrics-certs") pod "maas-controller-7848477867-gnggp" (UID: "0c9dbbb3-3427-4dbe-bf98-87bc3cd984be") : secret "maas-controller-metrics-tls" not found Sep 08 04:06:01.983142 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:01.983110 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0c9dbbb3-3427-4dbe-bf98-87bc3cd984be-webhook-cert\") pod \"maas-controller-7848477867-gnggp\" (UID: \"0c9dbbb3-3427-4dbe-bf98-87bc3cd984be\") " pod="opendatahub/maas-controller-7848477867-gnggp" Sep 08 04:06:01.990171 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:01.990150 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4fdq\" (UniqueName: \"kubernetes.io/projected/0c9dbbb3-3427-4dbe-bf98-87bc3cd984be-kube-api-access-m4fdq\") pod \"maas-controller-7848477867-gnggp\" (UID: \"0c9dbbb3-3427-4dbe-bf98-87bc3cd984be\") " pod="opendatahub/maas-controller-7848477867-gnggp" Sep 08 04:06:02.485522 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:02.485454 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c9dbbb3-3427-4dbe-bf98-87bc3cd984be-metrics-certs\") pod \"maas-controller-7848477867-gnggp\" (UID: \"0c9dbbb3-3427-4dbe-bf98-87bc3cd984be\") " pod="opendatahub/maas-controller-7848477867-gnggp" Sep 08 04:06:02.487820 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:02.487794 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c9dbbb3-3427-4dbe-bf98-87bc3cd984be-metrics-certs\") pod \"maas-controller-7848477867-gnggp\" (UID: \"0c9dbbb3-3427-4dbe-bf98-87bc3cd984be\") " pod="opendatahub/maas-controller-7848477867-gnggp" Sep 08 04:06:02.645926 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:02.645887 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7848477867-gnggp" Sep 08 04:06:02.919598 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:02.919487 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7848477867-gnggp"] Sep 08 04:06:02.922244 ip-10-0-128-141 kubenswrapper[2579]: W0908 04:06:02.922216 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c9dbbb3_3427_4dbe_bf98_87bc3cd984be.slice/crio-bfbdf771cf4407c3d284c83ccfac11ca159b0de0b675f2a7bfb8e959403ed5f9 WatchSource:0}: Error finding container bfbdf771cf4407c3d284c83ccfac11ca159b0de0b675f2a7bfb8e959403ed5f9: Status 404 returned error can't find the container with id bfbdf771cf4407c3d284c83ccfac11ca159b0de0b675f2a7bfb8e959403ed5f9 Sep 08 04:06:03.446320 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:03.446281 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7848477867-gnggp" event={"ID":"0c9dbbb3-3427-4dbe-bf98-87bc3cd984be","Type":"ContainerStarted","Data":"bfbdf771cf4407c3d284c83ccfac11ca159b0de0b675f2a7bfb8e959403ed5f9"} Sep 08 04:06:04.452850 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:04.452747 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7848477867-gnggp" event={"ID":"0c9dbbb3-3427-4dbe-bf98-87bc3cd984be","Type":"ContainerStarted","Data":"e0a66457ec9e976b5e1f2fe07f985938aa461ca3cfef716fe9bf72bb92e8b281"} Sep 08 04:06:04.452850 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:04.452827 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7848477867-gnggp" Sep 08 04:06:04.474272 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:04.474217 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7848477867-gnggp" podStartSLOduration=2.197919544 podStartE2EDuration="3.474198207s" totalImagesPullingTime="1.276278663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 04:06:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:06:04.473173031 +0000 UTC m=+624.094208997" watchObservedRunningTime="2026-09-08 04:06:04.474198207 +0000 UTC m=+624.095234175" Sep 08 04:06:15.459241 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:15.459209 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7848477867-gnggp" Sep 08 04:06:15.672415 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:15.672379 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7848477867-gnggp"] Sep 08 04:06:15.672669 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:15.672626 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7848477867-gnggp" podUID="0c9dbbb3-3427-4dbe-bf98-87bc3cd984be" containerName="manager" containerID="cri-o://e0a66457ec9e976b5e1f2fe07f985938aa461ca3cfef716fe9bf72bb92e8b281" gracePeriod=30 Sep 08 04:06:15.917302 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:15.917273 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7848477867-gnggp" Sep 08 04:06:16.110247 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:16.110155 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4fdq\" (UniqueName: \"kubernetes.io/projected/0c9dbbb3-3427-4dbe-bf98-87bc3cd984be-kube-api-access-m4fdq\") pod \"0c9dbbb3-3427-4dbe-bf98-87bc3cd984be\" (UID: \"0c9dbbb3-3427-4dbe-bf98-87bc3cd984be\") " Sep 08 04:06:16.110396 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:16.110252 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0c9dbbb3-3427-4dbe-bf98-87bc3cd984be-webhook-cert\") pod \"0c9dbbb3-3427-4dbe-bf98-87bc3cd984be\" (UID: \"0c9dbbb3-3427-4dbe-bf98-87bc3cd984be\") " Sep 08 04:06:16.110396 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:16.110270 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c9dbbb3-3427-4dbe-bf98-87bc3cd984be-metrics-certs\") pod \"0c9dbbb3-3427-4dbe-bf98-87bc3cd984be\" (UID: \"0c9dbbb3-3427-4dbe-bf98-87bc3cd984be\") " Sep 08 04:06:16.112564 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:16.112527 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c9dbbb3-3427-4dbe-bf98-87bc3cd984be-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0c9dbbb3-3427-4dbe-bf98-87bc3cd984be" (UID: "0c9dbbb3-3427-4dbe-bf98-87bc3cd984be"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 04:06:16.112687 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:16.112563 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c9dbbb3-3427-4dbe-bf98-87bc3cd984be-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0c9dbbb3-3427-4dbe-bf98-87bc3cd984be" (UID: "0c9dbbb3-3427-4dbe-bf98-87bc3cd984be"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 04:06:16.112687 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:16.112541 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c9dbbb3-3427-4dbe-bf98-87bc3cd984be-kube-api-access-m4fdq" (OuterVolumeSpecName: "kube-api-access-m4fdq") pod "0c9dbbb3-3427-4dbe-bf98-87bc3cd984be" (UID: "0c9dbbb3-3427-4dbe-bf98-87bc3cd984be"). InnerVolumeSpecName "kube-api-access-m4fdq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 04:06:16.211346 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:16.211306 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4fdq\" (UniqueName: \"kubernetes.io/projected/0c9dbbb3-3427-4dbe-bf98-87bc3cd984be-kube-api-access-m4fdq\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:06:16.211346 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:16.211341 2579 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0c9dbbb3-3427-4dbe-bf98-87bc3cd984be-webhook-cert\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:06:16.211575 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:16.211358 2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c9dbbb3-3427-4dbe-bf98-87bc3cd984be-metrics-certs\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:06:16.513876 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:16.513840 2579 generic.go:356] "Generic (PLEG): container finished" podID="0c9dbbb3-3427-4dbe-bf98-87bc3cd984be" containerID="e0a66457ec9e976b5e1f2fe07f985938aa461ca3cfef716fe9bf72bb92e8b281" exitCode=0 Sep 08 04:06:16.513876 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:16.513878 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7848477867-gnggp" event={"ID":"0c9dbbb3-3427-4dbe-bf98-87bc3cd984be","Type":"ContainerDied","Data":"e0a66457ec9e976b5e1f2fe07f985938aa461ca3cfef716fe9bf72bb92e8b281"} Sep 08 04:06:16.514333 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:16.513885 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7848477867-gnggp" Sep 08 04:06:16.514333 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:16.513928 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7848477867-gnggp" event={"ID":"0c9dbbb3-3427-4dbe-bf98-87bc3cd984be","Type":"ContainerDied","Data":"bfbdf771cf4407c3d284c83ccfac11ca159b0de0b675f2a7bfb8e959403ed5f9"} Sep 08 04:06:16.514333 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:16.513952 2579 scope.go:117] "RemoveContainer" containerID="e0a66457ec9e976b5e1f2fe07f985938aa461ca3cfef716fe9bf72bb92e8b281" Sep 08 04:06:16.523265 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:16.523240 2579 scope.go:117] "RemoveContainer" containerID="e0a66457ec9e976b5e1f2fe07f985938aa461ca3cfef716fe9bf72bb92e8b281" Sep 08 04:06:16.523579 ip-10-0-128-141 kubenswrapper[2579]: E0908 04:06:16.523560 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0a66457ec9e976b5e1f2fe07f985938aa461ca3cfef716fe9bf72bb92e8b281\": container with ID starting with e0a66457ec9e976b5e1f2fe07f985938aa461ca3cfef716fe9bf72bb92e8b281 not found: ID does not exist" containerID="e0a66457ec9e976b5e1f2fe07f985938aa461ca3cfef716fe9bf72bb92e8b281" Sep 08 04:06:16.523632 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:16.523587 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0a66457ec9e976b5e1f2fe07f985938aa461ca3cfef716fe9bf72bb92e8b281"} err="failed to get container status \"e0a66457ec9e976b5e1f2fe07f985938aa461ca3cfef716fe9bf72bb92e8b281\": rpc error: code = NotFound desc = could not find container \"e0a66457ec9e976b5e1f2fe07f985938aa461ca3cfef716fe9bf72bb92e8b281\": container with ID starting with e0a66457ec9e976b5e1f2fe07f985938aa461ca3cfef716fe9bf72bb92e8b281 not found: ID does not exist" Sep 08 04:06:16.536835 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:16.536800 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7848477867-gnggp"] Sep 08 04:06:16.540942 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:16.540911 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7848477867-gnggp"] Sep 08 04:06:16.876370 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:16.876337 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c9dbbb3-3427-4dbe-bf98-87bc3cd984be" path="/var/lib/kubelet/pods/0c9dbbb3-3427-4dbe-bf98-87bc3cd984be/volumes" Sep 08 04:06:37.724760 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:37.724723 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8"] Sep 08 04:06:37.725279 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:37.725255 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c9dbbb3-3427-4dbe-bf98-87bc3cd984be" containerName="manager" Sep 08 04:06:37.725279 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:37.725274 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c9dbbb3-3427-4dbe-bf98-87bc3cd984be" containerName="manager" Sep 08 04:06:37.725430 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:37.725364 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c9dbbb3-3427-4dbe-bf98-87bc3cd984be" containerName="manager" Sep 08 04:06:37.729011 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:37.728990 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8" Sep 08 04:06:37.734814 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:37.734791 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 08 04:06:37.734942 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:37.734882 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 08 04:06:37.735000 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:37.734793 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Sep 08 04:06:37.736064 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:37.735835 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6hlmn\"" Sep 08 04:06:37.737806 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:37.737787 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 04:06:37.754551 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:37.754515 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8"] Sep 08 04:06:37.793406 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:37.793371 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9a33153-82f2-4117-a85e-a54723cbcb4e-metrics-certs\") pod \"maas-api-768897d8b6-gb7d8\" (UID: \"c9a33153-82f2-4117-a85e-a54723cbcb4e\") " pod="odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8" Sep 08 04:06:37.793406 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:37.793411 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9a33153-82f2-4117-a85e-a54723cbcb4e-openshift-service-ca\") pod \"maas-api-768897d8b6-gb7d8\" (UID: \"c9a33153-82f2-4117-a85e-a54723cbcb4e\") " pod="odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8" Sep 08 04:06:37.793681 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:37.793448 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7qbd\" (UniqueName: \"kubernetes.io/projected/c9a33153-82f2-4117-a85e-a54723cbcb4e-kube-api-access-z7qbd\") pod \"maas-api-768897d8b6-gb7d8\" (UID: \"c9a33153-82f2-4117-a85e-a54723cbcb4e\") " pod="odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8" Sep 08 04:06:37.793681 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:37.793574 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9a33153-82f2-4117-a85e-a54723cbcb4e-maas-api-tls\") pod \"maas-api-768897d8b6-gb7d8\" (UID: \"c9a33153-82f2-4117-a85e-a54723cbcb4e\") " pod="odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8" Sep 08 04:06:37.894847 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:37.894811 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9a33153-82f2-4117-a85e-a54723cbcb4e-maas-api-tls\") pod \"maas-api-768897d8b6-gb7d8\" (UID: \"c9a33153-82f2-4117-a85e-a54723cbcb4e\") " pod="odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8" Sep 08 04:06:37.895019 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:37.894873 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9a33153-82f2-4117-a85e-a54723cbcb4e-metrics-certs\") pod \"maas-api-768897d8b6-gb7d8\" (UID: \"c9a33153-82f2-4117-a85e-a54723cbcb4e\") " pod="odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8" Sep 08 04:06:37.895019 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:37.894894 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9a33153-82f2-4117-a85e-a54723cbcb4e-openshift-service-ca\") pod \"maas-api-768897d8b6-gb7d8\" (UID: \"c9a33153-82f2-4117-a85e-a54723cbcb4e\") " pod="odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8" Sep 08 04:06:37.895019 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:37.894918 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7qbd\" (UniqueName: \"kubernetes.io/projected/c9a33153-82f2-4117-a85e-a54723cbcb4e-kube-api-access-z7qbd\") pod \"maas-api-768897d8b6-gb7d8\" (UID: \"c9a33153-82f2-4117-a85e-a54723cbcb4e\") " pod="odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8" Sep 08 04:06:37.895677 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:37.895650 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9a33153-82f2-4117-a85e-a54723cbcb4e-openshift-service-ca\") pod \"maas-api-768897d8b6-gb7d8\" (UID: \"c9a33153-82f2-4117-a85e-a54723cbcb4e\") " pod="odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8" Sep 08 04:06:37.897285 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:37.897262 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9a33153-82f2-4117-a85e-a54723cbcb4e-metrics-certs\") pod \"maas-api-768897d8b6-gb7d8\" (UID: \"c9a33153-82f2-4117-a85e-a54723cbcb4e\") " pod="odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8" Sep 08 04:06:37.897365 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:37.897296 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9a33153-82f2-4117-a85e-a54723cbcb4e-maas-api-tls\") pod \"maas-api-768897d8b6-gb7d8\" (UID: \"c9a33153-82f2-4117-a85e-a54723cbcb4e\") " pod="odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8" Sep 08 04:06:37.905272 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:37.905247 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7qbd\" (UniqueName: \"kubernetes.io/projected/c9a33153-82f2-4117-a85e-a54723cbcb4e-kube-api-access-z7qbd\") pod \"maas-api-768897d8b6-gb7d8\" (UID: \"c9a33153-82f2-4117-a85e-a54723cbcb4e\") " pod="odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8" Sep 08 04:06:38.039349 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:38.039254 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8" Sep 08 04:06:38.353873 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:38.353843 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8"] Sep 08 04:06:38.355648 ip-10-0-128-141 kubenswrapper[2579]: W0908 04:06:38.355625 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9a33153_82f2_4117_a85e_a54723cbcb4e.slice/crio-26ee7ba3851716678744dcc0fb4cafd298397176542d0f6b06a175d023660ad2 WatchSource:0}: Error finding container 26ee7ba3851716678744dcc0fb4cafd298397176542d0f6b06a175d023660ad2: Status 404 returned error can't find the container with id 26ee7ba3851716678744dcc0fb4cafd298397176542d0f6b06a175d023660ad2 Sep 08 04:06:38.409685 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:38.409648 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-2qlfp"] Sep 08 04:06:38.415150 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:38.415128 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2qlfp" Sep 08 04:06:38.418982 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:38.418961 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 08 04:06:38.419086 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:38.419067 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-7jbw9\"" Sep 08 04:06:38.435550 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:38.435520 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-2qlfp"] Sep 08 04:06:38.500770 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:38.500731 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgv7c\" (UniqueName: \"kubernetes.io/projected/9f5151f2-bf8a-4222-b50e-38f00d33e36c-kube-api-access-mgv7c\") pod \"payload-pre-processing-59fbc47bf8-2qlfp\" (UID: \"9f5151f2-bf8a-4222-b50e-38f00d33e36c\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2qlfp" Sep 08 04:06:38.500923 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:38.500781 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f5151f2-bf8a-4222-b50e-38f00d33e36c-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-2qlfp\" (UID: \"9f5151f2-bf8a-4222-b50e-38f00d33e36c\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2qlfp" Sep 08 04:06:38.552848 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:38.552812 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-jpfsx"] Sep 08 04:06:38.557086 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:38.557068 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-jpfsx" Sep 08 04:06:38.573801 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:38.573778 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-jpfsx"] Sep 08 04:06:38.602651 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:38.602619 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f5151f2-bf8a-4222-b50e-38f00d33e36c-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-2qlfp\" (UID: \"9f5151f2-bf8a-4222-b50e-38f00d33e36c\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2qlfp" Sep 08 04:06:38.602839 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:38.602667 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a77b151-799d-4738-a061-2c30cae4655f-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-jpfsx\" (UID: \"3a77b151-799d-4738-a061-2c30cae4655f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-jpfsx" Sep 08 04:06:38.602839 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:38.602741 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f8nj\" (UniqueName: \"kubernetes.io/projected/3a77b151-799d-4738-a061-2c30cae4655f-kube-api-access-8f8nj\") pod \"payload-processing-5b54b7fbb7-jpfsx\" (UID: \"3a77b151-799d-4738-a061-2c30cae4655f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-jpfsx" Sep 08 04:06:38.602839 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:38.602801 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgv7c\" (UniqueName: \"kubernetes.io/projected/9f5151f2-bf8a-4222-b50e-38f00d33e36c-kube-api-access-mgv7c\") pod \"payload-pre-processing-59fbc47bf8-2qlfp\" (UID: \"9f5151f2-bf8a-4222-b50e-38f00d33e36c\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2qlfp" Sep 08 04:06:38.603374 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:38.603348 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f5151f2-bf8a-4222-b50e-38f00d33e36c-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-2qlfp\" (UID: \"9f5151f2-bf8a-4222-b50e-38f00d33e36c\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2qlfp" Sep 08 04:06:38.613808 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:38.613785 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgv7c\" (UniqueName: \"kubernetes.io/projected/9f5151f2-bf8a-4222-b50e-38f00d33e36c-kube-api-access-mgv7c\") pod \"payload-pre-processing-59fbc47bf8-2qlfp\" (UID: \"9f5151f2-bf8a-4222-b50e-38f00d33e36c\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2qlfp" Sep 08 04:06:38.623681 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:38.623657 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8" event={"ID":"c9a33153-82f2-4117-a85e-a54723cbcb4e","Type":"ContainerStarted","Data":"26ee7ba3851716678744dcc0fb4cafd298397176542d0f6b06a175d023660ad2"} Sep 08 04:06:38.703992 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:38.703962 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a77b151-799d-4738-a061-2c30cae4655f-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-jpfsx\" (UID: \"3a77b151-799d-4738-a061-2c30cae4655f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-jpfsx" Sep 08 04:06:38.704135 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:38.704022 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8f8nj\" (UniqueName: \"kubernetes.io/projected/3a77b151-799d-4738-a061-2c30cae4655f-kube-api-access-8f8nj\") pod \"payload-processing-5b54b7fbb7-jpfsx\" (UID: \"3a77b151-799d-4738-a061-2c30cae4655f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-jpfsx" Sep 08 04:06:38.704717 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:38.704696 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a77b151-799d-4738-a061-2c30cae4655f-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-jpfsx\" (UID: \"3a77b151-799d-4738-a061-2c30cae4655f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-jpfsx" Sep 08 04:06:38.713200 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:38.713180 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f8nj\" (UniqueName: \"kubernetes.io/projected/3a77b151-799d-4738-a061-2c30cae4655f-kube-api-access-8f8nj\") pod \"payload-processing-5b54b7fbb7-jpfsx\" (UID: \"3a77b151-799d-4738-a061-2c30cae4655f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-jpfsx" Sep 08 04:06:38.726006 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:38.725979 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2qlfp" Sep 08 04:06:38.867002 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:38.866917 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-jpfsx" Sep 08 04:06:39.001132 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:39.001101 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-2qlfp"] Sep 08 04:06:39.003290 ip-10-0-128-141 kubenswrapper[2579]: W0908 04:06:39.003262 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f5151f2_bf8a_4222_b50e_38f00d33e36c.slice/crio-472bfec065885579e752175523e9e2b15f4e98739f07a1e38a93d9cacb77c7eb WatchSource:0}: Error finding container 472bfec065885579e752175523e9e2b15f4e98739f07a1e38a93d9cacb77c7eb: Status 404 returned error can't find the container with id 472bfec065885579e752175523e9e2b15f4e98739f07a1e38a93d9cacb77c7eb Sep 08 04:06:39.154571 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:39.154539 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-jpfsx"] Sep 08 04:06:39.155633 ip-10-0-128-141 kubenswrapper[2579]: W0908 04:06:39.155595 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a77b151_799d_4738_a061_2c30cae4655f.slice/crio-4e8bd1088e505079750bc0c25e9b3bce0da905d2f613cd278155ea7cb45fba63 WatchSource:0}: Error finding container 4e8bd1088e505079750bc0c25e9b3bce0da905d2f613cd278155ea7cb45fba63: Status 404 returned error can't find the container with id 4e8bd1088e505079750bc0c25e9b3bce0da905d2f613cd278155ea7cb45fba63 Sep 08 04:06:39.429343 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:39.429263 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-fhcmn"] Sep 08 04:06:39.634155 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:39.634116 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2qlfp" event={"ID":"9f5151f2-bf8a-4222-b50e-38f00d33e36c","Type":"ContainerStarted","Data":"472bfec065885579e752175523e9e2b15f4e98739f07a1e38a93d9cacb77c7eb"} Sep 08 04:06:39.636454 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:39.636421 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-jpfsx" event={"ID":"3a77b151-799d-4738-a061-2c30cae4655f","Type":"ContainerStarted","Data":"4e8bd1088e505079750bc0c25e9b3bce0da905d2f613cd278155ea7cb45fba63"} Sep 08 04:06:40.583203 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:40.581421 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-7d548556bc-4nvg5"] Sep 08 04:06:40.588135 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:40.588113 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-7d548556bc-4nvg5" Sep 08 04:06:40.611528 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:40.596104 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-zkwm8\"" Sep 08 04:06:40.611528 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:40.596353 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Sep 08 04:06:40.611528 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:40.598599 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 08 04:06:40.618780 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:40.613202 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-7d548556bc-4nvg5"] Sep 08 04:06:40.618780 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:40.613332 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 08 04:06:40.673001 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:40.668411 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8" event={"ID":"c9a33153-82f2-4117-a85e-a54723cbcb4e","Type":"ContainerStarted","Data":"7ab32e401862274ec65654166b0ee3fe0885b30c2bf27546e3df1d3b0f7f784d"} Sep 08 04:06:40.673001 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:40.668702 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8" Sep 08 04:06:40.693479 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:40.692158 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8" podStartSLOduration=1.841275863 podStartE2EDuration="3.692137106s" totalImagesPullingTime="1.850861243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 04:06:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:06:40.691595181 +0000 UTC m=+660.312631162" watchObservedRunningTime="2026-09-08 04:06:40.692137106 +0000 UTC m=+660.313173069" Sep 08 04:06:40.727480 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:40.726928 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19c6e60f-a398-4a34-ad94-f544233af88f-cert\") pod \"llmisvc-controller-manager-7d548556bc-4nvg5\" (UID: \"19c6e60f-a398-4a34-ad94-f544233af88f\") " pod="opendatahub/llmisvc-controller-manager-7d548556bc-4nvg5" Sep 08 04:06:40.727480 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:40.726995 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9mhp\" (UniqueName: \"kubernetes.io/projected/19c6e60f-a398-4a34-ad94-f544233af88f-kube-api-access-b9mhp\") pod \"llmisvc-controller-manager-7d548556bc-4nvg5\" (UID: \"19c6e60f-a398-4a34-ad94-f544233af88f\") " pod="opendatahub/llmisvc-controller-manager-7d548556bc-4nvg5" Sep 08 04:06:40.828848 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:40.828811 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19c6e60f-a398-4a34-ad94-f544233af88f-cert\") pod \"llmisvc-controller-manager-7d548556bc-4nvg5\" (UID: \"19c6e60f-a398-4a34-ad94-f544233af88f\") " pod="opendatahub/llmisvc-controller-manager-7d548556bc-4nvg5" Sep 08 04:06:40.829016 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:40.828893 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9mhp\" (UniqueName: \"kubernetes.io/projected/19c6e60f-a398-4a34-ad94-f544233af88f-kube-api-access-b9mhp\") pod \"llmisvc-controller-manager-7d548556bc-4nvg5\" (UID: \"19c6e60f-a398-4a34-ad94-f544233af88f\") " pod="opendatahub/llmisvc-controller-manager-7d548556bc-4nvg5" Sep 08 04:06:40.834152 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:40.834043 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Sep 08 04:06:40.837851 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:40.837665 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 08 04:06:40.840236 ip-10-0-128-141 kubenswrapper[2579]: E0908 04:06:40.840114 2579 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Sep 08 04:06:40.840236 ip-10-0-128-141 kubenswrapper[2579]: E0908 04:06:40.840191 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19c6e60f-a398-4a34-ad94-f544233af88f-cert podName:19c6e60f-a398-4a34-ad94-f544233af88f nodeName:}" failed. No retries permitted until 2026-09-08 04:06:41.34017098 +0000 UTC m=+660.961206931 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/19c6e60f-a398-4a34-ad94-f544233af88f-cert") pod "llmisvc-controller-manager-7d548556bc-4nvg5" (UID: "19c6e60f-a398-4a34-ad94-f544233af88f") : secret "llmisvc-webhook-server-cert" not found Sep 08 04:06:40.848418 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:40.848392 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 08 04:06:40.868593 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:40.868532 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9mhp\" (UniqueName: \"kubernetes.io/projected/19c6e60f-a398-4a34-ad94-f544233af88f-kube-api-access-b9mhp\") pod \"llmisvc-controller-manager-7d548556bc-4nvg5\" (UID: \"19c6e60f-a398-4a34-ad94-f544233af88f\") " pod="opendatahub/llmisvc-controller-manager-7d548556bc-4nvg5" Sep 08 04:06:41.321631 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.321605 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx"] Sep 08 04:06:41.325931 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.325908 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" Sep 08 04:06:41.328975 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.328951 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 08 04:06:41.329095 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.328956 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 08 04:06:41.329321 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.329301 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 08 04:06:41.329420 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.329399 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-dzxnr\"" Sep 08 04:06:41.329709 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.329687 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 08 04:06:41.340818 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.340798 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx"] Sep 08 04:06:41.436665 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.436642 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19c6e60f-a398-4a34-ad94-f544233af88f-cert\") pod \"llmisvc-controller-manager-7d548556bc-4nvg5\" (UID: \"19c6e60f-a398-4a34-ad94-f544233af88f\") " pod="opendatahub/llmisvc-controller-manager-7d548556bc-4nvg5" Sep 08 04:06:41.436784 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.436704 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-gmrnx\" (UID: \"2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" Sep 08 04:06:41.436784 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.436728 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-gmrnx\" (UID: \"2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" Sep 08 04:06:41.436784 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.436745 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-gmrnx\" (UID: \"2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" Sep 08 04:06:41.436784 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.436775 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fl4gh\" (UniqueName: \"kubernetes.io/projected/2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063-kube-api-access-fl4gh\") pod \"istiod-openshift-gateway-5677ccb49b-gmrnx\" (UID: \"2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" Sep 08 04:06:41.436989 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.436913 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-gmrnx\" (UID: \"2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" Sep 08 04:06:41.436989 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.436956 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-gmrnx\" (UID: \"2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" Sep 08 04:06:41.436989 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.436980 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-gmrnx\" (UID: \"2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" Sep 08 04:06:41.439071 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.439050 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19c6e60f-a398-4a34-ad94-f544233af88f-cert\") pod \"llmisvc-controller-manager-7d548556bc-4nvg5\" (UID: \"19c6e60f-a398-4a34-ad94-f544233af88f\") " pod="opendatahub/llmisvc-controller-manager-7d548556bc-4nvg5" Sep 08 04:06:41.513297 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.513265 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-zkwm8\"" Sep 08 04:06:41.520666 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.520606 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-7d548556bc-4nvg5" Sep 08 04:06:41.537944 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.537918 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-gmrnx\" (UID: \"2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" Sep 08 04:06:41.538058 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.537958 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-gmrnx\" (UID: \"2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" Sep 08 04:06:41.538058 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.537985 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-gmrnx\" (UID: \"2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" Sep 08 04:06:41.538180 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.538068 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-gmrnx\" (UID: \"2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" Sep 08 04:06:41.538180 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.538099 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-gmrnx\" (UID: \"2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" Sep 08 04:06:41.538180 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.538125 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-gmrnx\" (UID: \"2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" Sep 08 04:06:41.538180 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.538167 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl4gh\" (UniqueName: \"kubernetes.io/projected/2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063-kube-api-access-fl4gh\") pod \"istiod-openshift-gateway-5677ccb49b-gmrnx\" (UID: \"2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" Sep 08 04:06:41.539036 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.539004 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-gmrnx\" (UID: \"2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" Sep 08 04:06:41.541140 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.541069 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-gmrnx\" (UID: \"2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" Sep 08 04:06:41.541237 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.541207 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-gmrnx\" (UID: \"2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" Sep 08 04:06:41.541569 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.541547 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-gmrnx\" (UID: \"2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" Sep 08 04:06:41.541685 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.541590 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-gmrnx\" (UID: \"2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" Sep 08 04:06:41.547941 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.547911 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-gmrnx\" (UID: \"2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" Sep 08 04:06:41.548030 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.548010 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fl4gh\" (UniqueName: \"kubernetes.io/projected/2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063-kube-api-access-fl4gh\") pod \"istiod-openshift-gateway-5677ccb49b-gmrnx\" (UID: \"2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" Sep 08 04:06:41.647958 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.647928 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" Sep 08 04:06:41.685131 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.685061 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-jpfsx" event={"ID":"3a77b151-799d-4738-a061-2c30cae4655f","Type":"ContainerStarted","Data":"64608142acd081741eeb257937b448e398b625f9fea617b6687201cbab3d1bab"} Sep 08 04:06:41.685324 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.685164 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-jpfsx" Sep 08 04:06:41.686922 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.686888 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2qlfp" event={"ID":"9f5151f2-bf8a-4222-b50e-38f00d33e36c","Type":"ContainerStarted","Data":"242cfd4b354f610150898babcd234087521f842ed1fb12e96f7328f8cfaf1e1d"} Sep 08 04:06:41.687115 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.687092 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2qlfp" Sep 08 04:06:41.708799 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.708643 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-jpfsx" podStartSLOduration=1.472639618 podStartE2EDuration="3.708610881s" totalImagesPullingTime="2.235971263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 04:06:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:06:41.70597908 +0000 UTC m=+661.327015047" watchObservedRunningTime="2026-09-08 04:06:41.708610881 +0000 UTC m=+661.329646853" Sep 08 04:06:41.724794 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.724739 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2qlfp" podStartSLOduration=1.411595795 podStartE2EDuration="3.724721772s" totalImagesPullingTime="2.313125977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 04:06:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:06:41.724055813 +0000 UTC m=+661.345091781" watchObservedRunningTime="2026-09-08 04:06:41.724721772 +0000 UTC m=+661.345757739" Sep 08 04:06:41.801675 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.801542 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-7d548556bc-4nvg5"] Sep 08 04:06:41.804431 ip-10-0-128-141 kubenswrapper[2579]: W0908 04:06:41.804401 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod19c6e60f_a398_4a34_ad94_f544233af88f.slice/crio-1777a545191362ae8a5cd789f95277f56649adfef29e57b8dad526f204eeb7de WatchSource:0}: Error finding container 1777a545191362ae8a5cd789f95277f56649adfef29e57b8dad526f204eeb7de: Status 404 returned error can't find the container with id 1777a545191362ae8a5cd789f95277f56649adfef29e57b8dad526f204eeb7de Sep 08 04:06:41.956959 ip-10-0-128-141 kubenswrapper[2579]: W0908 04:06:41.956919 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ff2bfc5_d93b_4e90_8aed_cbcc0e8df063.slice/crio-1a5ae946832b777d7518c4a8e2f5df7cf6ebceea10a5a2e8e220fb59c0b26162 WatchSource:0}: Error finding container 1a5ae946832b777d7518c4a8e2f5df7cf6ebceea10a5a2e8e220fb59c0b26162: Status 404 returned error can't find the container with id 1a5ae946832b777d7518c4a8e2f5df7cf6ebceea10a5a2e8e220fb59c0b26162 Sep 08 04:06:41.959283 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:41.959253 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx"] Sep 08 04:06:42.695359 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:42.695322 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" event={"ID":"2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063","Type":"ContainerStarted","Data":"1a5ae946832b777d7518c4a8e2f5df7cf6ebceea10a5a2e8e220fb59c0b26162"} Sep 08 04:06:42.696845 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:42.696778 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-7d548556bc-4nvg5" event={"ID":"19c6e60f-a398-4a34-ad94-f544233af88f","Type":"ContainerStarted","Data":"1777a545191362ae8a5cd789f95277f56649adfef29e57b8dad526f204eeb7de"} Sep 08 04:06:45.196305 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:45.196270 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 08 04:06:45.196653 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:45.196339 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 08 04:06:45.716300 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:45.716265 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" event={"ID":"2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063","Type":"ContainerStarted","Data":"510301bb9b0938f2042a4b20be36b3ad900d4788343351387f8ef63694aebc84"} Sep 08 04:06:45.716553 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:45.716334 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" Sep 08 04:06:45.717795 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:45.717774 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-7d548556bc-4nvg5" event={"ID":"19c6e60f-a398-4a34-ad94-f544233af88f","Type":"ContainerStarted","Data":"7daf760506006675650b8fdbce48f553e71c45ce1bab6510f5e06f2481b3e64b"} Sep 08 04:06:45.717886 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:45.717837 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-7d548556bc-4nvg5" Sep 08 04:06:45.754673 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:45.754612 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-7d548556bc-4nvg5" podStartSLOduration=2.3619423299999998 podStartE2EDuration="5.754595201s" totalImagesPullingTime="3.392652871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 04:06:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:06:45.753954456 +0000 UTC m=+665.374990422" watchObservedRunningTime="2026-09-08 04:06:45.754595201 +0000 UTC m=+665.375631167" Sep 08 04:06:45.754956 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:45.754928 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" podStartSLOduration=1.518052358 podStartE2EDuration="4.754918391s" totalImagesPullingTime="3.236866033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 04:06:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:06:45.738271669 +0000 UTC m=+665.359307637" watchObservedRunningTime="2026-09-08 04:06:45.754918391 +0000 UTC m=+665.375954363" Sep 08 04:06:46.693300 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:46.693273 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8" Sep 08 04:06:46.726514 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:46.726453 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gmrnx" Sep 08 04:06:52.699158 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:52.699128 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2qlfp" Sep 08 04:06:52.699744 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:52.699210 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-jpfsx" Sep 08 04:06:57.053132 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:57.053093 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8"] Sep 08 04:06:57.053587 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:57.053356 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8" podUID="c9a33153-82f2-4117-a85e-a54723cbcb4e" containerName="maas-api" containerID="cri-o://7ab32e401862274ec65654166b0ee3fe0885b30c2bf27546e3df1d3b0f7f784d" gracePeriod=30 Sep 08 04:06:57.297926 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:57.297899 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8" Sep 08 04:06:57.386478 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:57.386428 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9a33153-82f2-4117-a85e-a54723cbcb4e-openshift-service-ca\") pod \"c9a33153-82f2-4117-a85e-a54723cbcb4e\" (UID: \"c9a33153-82f2-4117-a85e-a54723cbcb4e\") " Sep 08 04:06:57.386707 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:57.386605 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9a33153-82f2-4117-a85e-a54723cbcb4e-metrics-certs\") pod \"c9a33153-82f2-4117-a85e-a54723cbcb4e\" (UID: \"c9a33153-82f2-4117-a85e-a54723cbcb4e\") " Sep 08 04:06:57.386707 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:57.386663 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7qbd\" (UniqueName: \"kubernetes.io/projected/c9a33153-82f2-4117-a85e-a54723cbcb4e-kube-api-access-z7qbd\") pod \"c9a33153-82f2-4117-a85e-a54723cbcb4e\" (UID: \"c9a33153-82f2-4117-a85e-a54723cbcb4e\") " Sep 08 04:06:57.386707 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:57.386685 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9a33153-82f2-4117-a85e-a54723cbcb4e-maas-api-tls\") pod \"c9a33153-82f2-4117-a85e-a54723cbcb4e\" (UID: \"c9a33153-82f2-4117-a85e-a54723cbcb4e\") " Sep 08 04:06:57.386861 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:57.386786 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9a33153-82f2-4117-a85e-a54723cbcb4e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c9a33153-82f2-4117-a85e-a54723cbcb4e" (UID: "c9a33153-82f2-4117-a85e-a54723cbcb4e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 04:06:57.386961 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:57.386946 2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9a33153-82f2-4117-a85e-a54723cbcb4e-openshift-service-ca\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:06:57.388894 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:57.388865 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9a33153-82f2-4117-a85e-a54723cbcb4e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c9a33153-82f2-4117-a85e-a54723cbcb4e" (UID: "c9a33153-82f2-4117-a85e-a54723cbcb4e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 04:06:57.388894 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:57.388862 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9a33153-82f2-4117-a85e-a54723cbcb4e-kube-api-access-z7qbd" (OuterVolumeSpecName: "kube-api-access-z7qbd") pod "c9a33153-82f2-4117-a85e-a54723cbcb4e" (UID: "c9a33153-82f2-4117-a85e-a54723cbcb4e"). InnerVolumeSpecName "kube-api-access-z7qbd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 04:06:57.389063 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:57.388928 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9a33153-82f2-4117-a85e-a54723cbcb4e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c9a33153-82f2-4117-a85e-a54723cbcb4e" (UID: "c9a33153-82f2-4117-a85e-a54723cbcb4e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 04:06:57.487642 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:57.487594 2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9a33153-82f2-4117-a85e-a54723cbcb4e-metrics-certs\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:06:57.487642 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:57.487632 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7qbd\" (UniqueName: \"kubernetes.io/projected/c9a33153-82f2-4117-a85e-a54723cbcb4e-kube-api-access-z7qbd\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:06:57.487642 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:57.487646 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9a33153-82f2-4117-a85e-a54723cbcb4e-maas-api-tls\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Sep 08 04:06:57.781985 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:57.781888 2579 generic.go:356] "Generic (PLEG): container finished" podID="c9a33153-82f2-4117-a85e-a54723cbcb4e" containerID="7ab32e401862274ec65654166b0ee3fe0885b30c2bf27546e3df1d3b0f7f784d" exitCode=0 Sep 08 04:06:57.781985 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:57.781935 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8" event={"ID":"c9a33153-82f2-4117-a85e-a54723cbcb4e","Type":"ContainerDied","Data":"7ab32e401862274ec65654166b0ee3fe0885b30c2bf27546e3df1d3b0f7f784d"} Sep 08 04:06:57.781985 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:57.781941 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8" Sep 08 04:06:57.781985 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:57.781987 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8" event={"ID":"c9a33153-82f2-4117-a85e-a54723cbcb4e","Type":"ContainerDied","Data":"26ee7ba3851716678744dcc0fb4cafd298397176542d0f6b06a175d023660ad2"} Sep 08 04:06:57.782300 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:57.782010 2579 scope.go:117] "RemoveContainer" containerID="7ab32e401862274ec65654166b0ee3fe0885b30c2bf27546e3df1d3b0f7f784d" Sep 08 04:06:57.791436 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:57.791414 2579 scope.go:117] "RemoveContainer" containerID="7ab32e401862274ec65654166b0ee3fe0885b30c2bf27546e3df1d3b0f7f784d" Sep 08 04:06:57.791785 ip-10-0-128-141 kubenswrapper[2579]: E0908 04:06:57.791761 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ab32e401862274ec65654166b0ee3fe0885b30c2bf27546e3df1d3b0f7f784d\": container with ID starting with 7ab32e401862274ec65654166b0ee3fe0885b30c2bf27546e3df1d3b0f7f784d not found: ID does not exist" containerID="7ab32e401862274ec65654166b0ee3fe0885b30c2bf27546e3df1d3b0f7f784d" Sep 08 04:06:57.791865 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:57.791795 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ab32e401862274ec65654166b0ee3fe0885b30c2bf27546e3df1d3b0f7f784d"} err="failed to get container status \"7ab32e401862274ec65654166b0ee3fe0885b30c2bf27546e3df1d3b0f7f784d\": rpc error: code = NotFound desc = could not find container \"7ab32e401862274ec65654166b0ee3fe0885b30c2bf27546e3df1d3b0f7f784d\": container with ID starting with 7ab32e401862274ec65654166b0ee3fe0885b30c2bf27546e3df1d3b0f7f784d not found: ID does not exist" Sep 08 04:06:57.804519 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:57.804489 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8"] Sep 08 04:06:57.808063 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:57.808036 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-768897d8b6-gb7d8"] Sep 08 04:06:58.876174 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:06:58.876141 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9a33153-82f2-4117-a85e-a54723cbcb4e" path="/var/lib/kubelet/pods/c9a33153-82f2-4117-a85e-a54723cbcb4e/volumes" Sep 08 04:07:16.726761 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:07:16.726729 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-7d548556bc-4nvg5" Sep 08 04:07:48.990114 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:07:48.990078 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-fhcmn"] Sep 08 04:08:00.975811 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:08:00.975771 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-fhcmn"] Sep 08 04:08:15.375794 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:08:15.375758 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-fhcmn"] Sep 08 04:08:25.786086 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:08:25.786048 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-fhcmn"] Sep 08 04:08:32.381934 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:08:32.381898 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-fhcmn"] Sep 08 04:08:41.273694 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:08:41.273661 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-fhcmn"] Sep 08 04:09:09.984662 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:09:09.984572 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-fhcmn"] Sep 08 04:10:40.850979 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:10:40.850945 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tg7hb_45ae9afd-2dea-43b8-aa31-169b11e60fd2/ovn-acl-logging/0.log" Sep 08 04:10:40.852699 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:10:40.852667 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tg7hb_45ae9afd-2dea-43b8-aa31-169b11e60fd2/ovn-acl-logging/0.log" Sep 08 04:15:31.793687 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:31.793651 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-7d548556bc-4nvg5_19c6e60f-a398-4a34-ad94-f544233af88f/manager/0.log" Sep 08 04:15:34.113977 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:34.113945 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-5c6ll_a643c680-e2b7-489b-b200-4bbf60097793/kuadrant-console-plugin/0.log" Sep 08 04:15:34.387247 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:34.387222 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-fhcmn_3e6aeb77-f443-4bce-8b72-df3687476747/limitador/0.log" Sep 08 04:15:35.068784 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:35.068754 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-gmrnx_2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063/discovery/0.log" Sep 08 04:15:35.326162 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:35.326086 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65bcfbd747-vzxhm_ef20b471-07ea-4943-98b3-17ebf9a36b06/kube-auth-proxy/0.log" Sep 08 04:15:35.719137 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:35.719089 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-2qlfp_9f5151f2-bf8a-4222-b50e-38f00d33e36c/payload-pre-processing/0.log" Sep 08 04:15:35.877114 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:35.877082 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-jpfsx_3a77b151-799d-4738-a061-2c30cae4655f/payload-processing/0.log" Sep 08 04:15:35.997420 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:35.997332 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-69bf99dbfb-78zrw_179fd61c-eeab-4eef-b59c-c75528b5c6ee/router/0.log" Sep 08 04:15:40.881629 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:40.881595 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tg7hb_45ae9afd-2dea-43b8-aa31-169b11e60fd2/ovn-acl-logging/0.log" Sep 08 04:15:40.884069 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:40.884046 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tg7hb_45ae9afd-2dea-43b8-aa31-169b11e60fd2/ovn-acl-logging/0.log" Sep 08 04:15:45.732638 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:45.732603 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hrmrn_26877828-beec-4ff6-a322-79cdf54e0039/global-pull-secret-syncer/0.log" Sep 08 04:15:45.799997 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:45.799964 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2rrvn_cc5a49e7-1980-4418-acf0-0be7ad18e2f5/konnectivity-agent/0.log" Sep 08 04:15:45.843970 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:45.843941 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-141.ec2.internal_f6f3f87e6fd344b94ef9c0dd58f8052f/haproxy/0.log" Sep 08 04:15:50.217375 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:50.217346 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-5c6ll_a643c680-e2b7-489b-b200-4bbf60097793/kuadrant-console-plugin/0.log" Sep 08 04:15:50.289687 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:50.289655 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-fhcmn_3e6aeb77-f443-4bce-8b72-df3687476747/limitador/0.log" Sep 08 04:15:52.041970 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:52.041936 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-dxhpn_3027dc6f-51e4-45de-adfc-3f426e71e9b9/cluster-monitoring-operator/0.log" Sep 08 04:15:52.072860 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:52.072827 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-kfqkf_d6b54952-ee54-48f0-a1ce-c0255b19fe37/kube-state-metrics/0.log" Sep 08 04:15:52.088445 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:52.088417 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-kfqkf_d6b54952-ee54-48f0-a1ce-c0255b19fe37/kube-rbac-proxy-main/0.log" Sep 08 04:15:52.104105 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:52.104085 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-kfqkf_d6b54952-ee54-48f0-a1ce-c0255b19fe37/kube-rbac-proxy-self/0.log" Sep 08 04:15:52.150083 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:52.150056 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-76j8x_927871d3-f692-417d-8cd1-024e6bd0b603/monitoring-plugin/0.log" Sep 08 04:15:52.303250 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:52.303175 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wm4h4_b100a94e-b7da-4c28-9fd4-0842380dc1f1/node-exporter/0.log" Sep 08 04:15:52.322643 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:52.322618 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wm4h4_b100a94e-b7da-4c28-9fd4-0842380dc1f1/kube-rbac-proxy/0.log" Sep 08 04:15:52.339056 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:52.339032 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wm4h4_b100a94e-b7da-4c28-9fd4-0842380dc1f1/init-textfile/0.log" Sep 08 04:15:52.375339 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:52.375313 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-m67qg_f9c080e9-aeba-4554-a0a2-66cf8881dce3/kube-rbac-proxy-main/0.log" Sep 08 04:15:52.397402 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:52.397377 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-m67qg_f9c080e9-aeba-4554-a0a2-66cf8881dce3/kube-rbac-proxy-self/0.log" Sep 08 04:15:52.416271 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:52.416232 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-m67qg_f9c080e9-aeba-4554-a0a2-66cf8881dce3/openshift-state-metrics/0.log" Sep 08 04:15:52.451744 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:52.451717 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7dbb7d85-4b09-4116-875b-8f96789a4eeb/prometheus/0.log" Sep 08 04:15:52.469454 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:52.469429 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7dbb7d85-4b09-4116-875b-8f96789a4eeb/config-reloader/0.log" Sep 08 04:15:52.486618 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:52.486596 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7dbb7d85-4b09-4116-875b-8f96789a4eeb/thanos-sidecar/0.log" Sep 08 04:15:52.504156 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:52.504131 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7dbb7d85-4b09-4116-875b-8f96789a4eeb/kube-rbac-proxy-web/0.log" Sep 08 04:15:52.526657 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:52.526632 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7dbb7d85-4b09-4116-875b-8f96789a4eeb/kube-rbac-proxy/0.log" Sep 08 04:15:52.544929 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:52.544888 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7dbb7d85-4b09-4116-875b-8f96789a4eeb/kube-rbac-proxy-thanos/0.log" Sep 08 04:15:52.565675 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:52.565595 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7dbb7d85-4b09-4116-875b-8f96789a4eeb/init-config-reloader/0.log" Sep 08 04:15:52.726702 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:52.726679 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7496599d74-sjlm8_64946685-65fe-4715-9125-26985b039d15/telemeter-client/0.log" Sep 08 04:15:52.748954 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:52.748934 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7496599d74-sjlm8_64946685-65fe-4715-9125-26985b039d15/reload/0.log" Sep 08 04:15:52.788644 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:52.788615 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7496599d74-sjlm8_64946685-65fe-4715-9125-26985b039d15/kube-rbac-proxy/0.log" Sep 08 04:15:55.655288 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:55.655252 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qbfdx/perf-node-gather-daemonset-mjvnj"] Sep 08 04:15:55.655720 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:55.655701 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9a33153-82f2-4117-a85e-a54723cbcb4e" containerName="maas-api" Sep 08 04:15:55.655767 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:55.655723 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9a33153-82f2-4117-a85e-a54723cbcb4e" containerName="maas-api" Sep 08 04:15:55.655867 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:55.655851 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9a33153-82f2-4117-a85e-a54723cbcb4e" containerName="maas-api" Sep 08 04:15:55.659001 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:55.658984 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qbfdx/perf-node-gather-daemonset-mjvnj" Sep 08 04:15:55.661624 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:55.661603 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qbfdx\"/\"openshift-service-ca.crt\"" Sep 08 04:15:55.661867 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:55.661852 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qbfdx\"/\"kube-root-ca.crt\"" Sep 08 04:15:55.662849 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:55.662814 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qbfdx\"/\"default-dockercfg-qwfzv\"" Sep 08 04:15:55.668658 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:55.668634 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qbfdx/perf-node-gather-daemonset-mjvnj"] Sep 08 04:15:55.778928 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:55.778887 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/15be02c5-baae-49e6-87a8-309c74ebeb20-lib-modules\") pod \"perf-node-gather-daemonset-mjvnj\" (UID: \"15be02c5-baae-49e6-87a8-309c74ebeb20\") " pod="openshift-must-gather-qbfdx/perf-node-gather-daemonset-mjvnj" Sep 08 04:15:55.778928 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:55.778930 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/15be02c5-baae-49e6-87a8-309c74ebeb20-sys\") pod \"perf-node-gather-daemonset-mjvnj\" (UID: \"15be02c5-baae-49e6-87a8-309c74ebeb20\") " pod="openshift-must-gather-qbfdx/perf-node-gather-daemonset-mjvnj" Sep 08 04:15:55.779241 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:55.778949 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlfm2\" (UniqueName: \"kubernetes.io/projected/15be02c5-baae-49e6-87a8-309c74ebeb20-kube-api-access-mlfm2\") pod \"perf-node-gather-daemonset-mjvnj\" (UID: \"15be02c5-baae-49e6-87a8-309c74ebeb20\") " pod="openshift-must-gather-qbfdx/perf-node-gather-daemonset-mjvnj" Sep 08 04:15:55.779241 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:55.779020 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/15be02c5-baae-49e6-87a8-309c74ebeb20-proc\") pod \"perf-node-gather-daemonset-mjvnj\" (UID: \"15be02c5-baae-49e6-87a8-309c74ebeb20\") " pod="openshift-must-gather-qbfdx/perf-node-gather-daemonset-mjvnj" Sep 08 04:15:55.779241 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:55.779071 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/15be02c5-baae-49e6-87a8-309c74ebeb20-podres\") pod \"perf-node-gather-daemonset-mjvnj\" (UID: \"15be02c5-baae-49e6-87a8-309c74ebeb20\") " pod="openshift-must-gather-qbfdx/perf-node-gather-daemonset-mjvnj" Sep 08 04:15:55.879914 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:55.879876 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/15be02c5-baae-49e6-87a8-309c74ebeb20-lib-modules\") pod \"perf-node-gather-daemonset-mjvnj\" (UID: \"15be02c5-baae-49e6-87a8-309c74ebeb20\") " pod="openshift-must-gather-qbfdx/perf-node-gather-daemonset-mjvnj" Sep 08 04:15:55.879914 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:55.879917 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/15be02c5-baae-49e6-87a8-309c74ebeb20-sys\") pod \"perf-node-gather-daemonset-mjvnj\" (UID: \"15be02c5-baae-49e6-87a8-309c74ebeb20\") " pod="openshift-must-gather-qbfdx/perf-node-gather-daemonset-mjvnj" Sep 08 04:15:55.880129 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:55.879936 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlfm2\" (UniqueName: \"kubernetes.io/projected/15be02c5-baae-49e6-87a8-309c74ebeb20-kube-api-access-mlfm2\") pod \"perf-node-gather-daemonset-mjvnj\" (UID: \"15be02c5-baae-49e6-87a8-309c74ebeb20\") " pod="openshift-must-gather-qbfdx/perf-node-gather-daemonset-mjvnj" Sep 08 04:15:55.880129 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:55.879966 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/15be02c5-baae-49e6-87a8-309c74ebeb20-proc\") pod \"perf-node-gather-daemonset-mjvnj\" (UID: \"15be02c5-baae-49e6-87a8-309c74ebeb20\") " pod="openshift-must-gather-qbfdx/perf-node-gather-daemonset-mjvnj" Sep 08 04:15:55.880129 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:55.880005 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/15be02c5-baae-49e6-87a8-309c74ebeb20-podres\") pod \"perf-node-gather-daemonset-mjvnj\" (UID: \"15be02c5-baae-49e6-87a8-309c74ebeb20\") " pod="openshift-must-gather-qbfdx/perf-node-gather-daemonset-mjvnj" Sep 08 04:15:55.880129 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:55.880034 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/15be02c5-baae-49e6-87a8-309c74ebeb20-sys\") pod \"perf-node-gather-daemonset-mjvnj\" (UID: \"15be02c5-baae-49e6-87a8-309c74ebeb20\") " pod="openshift-must-gather-qbfdx/perf-node-gather-daemonset-mjvnj" Sep 08 04:15:55.880129 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:55.880060 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/15be02c5-baae-49e6-87a8-309c74ebeb20-lib-modules\") pod \"perf-node-gather-daemonset-mjvnj\" (UID: \"15be02c5-baae-49e6-87a8-309c74ebeb20\") " pod="openshift-must-gather-qbfdx/perf-node-gather-daemonset-mjvnj" Sep 08 04:15:55.880129 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:55.880078 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/15be02c5-baae-49e6-87a8-309c74ebeb20-proc\") pod \"perf-node-gather-daemonset-mjvnj\" (UID: \"15be02c5-baae-49e6-87a8-309c74ebeb20\") " pod="openshift-must-gather-qbfdx/perf-node-gather-daemonset-mjvnj" Sep 08 04:15:55.880325 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:55.880135 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/15be02c5-baae-49e6-87a8-309c74ebeb20-podres\") pod \"perf-node-gather-daemonset-mjvnj\" (UID: \"15be02c5-baae-49e6-87a8-309c74ebeb20\") " pod="openshift-must-gather-qbfdx/perf-node-gather-daemonset-mjvnj" Sep 08 04:15:55.888118 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:55.888097 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlfm2\" (UniqueName: \"kubernetes.io/projected/15be02c5-baae-49e6-87a8-309c74ebeb20-kube-api-access-mlfm2\") pod \"perf-node-gather-daemonset-mjvnj\" (UID: \"15be02c5-baae-49e6-87a8-309c74ebeb20\") " pod="openshift-must-gather-qbfdx/perf-node-gather-daemonset-mjvnj" Sep 08 04:15:55.969819 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:55.969737 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qbfdx/perf-node-gather-daemonset-mjvnj" Sep 08 04:15:56.242979 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:56.242941 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qbfdx/perf-node-gather-daemonset-mjvnj"] Sep 08 04:15:56.245421 ip-10-0-128-141 kubenswrapper[2579]: W0908 04:15:56.245396 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod15be02c5_baae_49e6_87a8_309c74ebeb20.slice/crio-8d5e961d68c368290617472b673f1779c05d19de7b73dedb95db6e524fe37e2c WatchSource:0}: Error finding container 8d5e961d68c368290617472b673f1779c05d19de7b73dedb95db6e524fe37e2c: Status 404 returned error can't find the container with id 8d5e961d68c368290617472b673f1779c05d19de7b73dedb95db6e524fe37e2c Sep 08 04:15:56.358348 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:56.358326 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-7qzwn_5ceb21d6-1277-4ff4-a22d-8b4f4b218e45/volume-data-source-validator/0.log" Sep 08 04:15:56.383012 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:56.382988 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qbfdx/perf-node-gather-daemonset-mjvnj" event={"ID":"15be02c5-baae-49e6-87a8-309c74ebeb20","Type":"ContainerStarted","Data":"8e9caf3c0d8547f8fd6fc2c27452011decbb262b6852fa865432846007a191dc"} Sep 08 04:15:56.383144 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:56.383035 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qbfdx/perf-node-gather-daemonset-mjvnj" event={"ID":"15be02c5-baae-49e6-87a8-309c74ebeb20","Type":"ContainerStarted","Data":"8d5e961d68c368290617472b673f1779c05d19de7b73dedb95db6e524fe37e2c"} Sep 08 04:15:56.383144 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:56.383084 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qbfdx/perf-node-gather-daemonset-mjvnj" Sep 08 04:15:56.401171 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:56.401118 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qbfdx/perf-node-gather-daemonset-mjvnj" podStartSLOduration=1.401103515 podStartE2EDuration="1.401103515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 04:15:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 04:15:56.399140746 +0000 UTC m=+1216.020176712" watchObservedRunningTime="2026-09-08 04:15:56.401103515 +0000 UTC m=+1216.022139481" Sep 08 04:15:57.419393 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:57.419362 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2jhc2_77b984c3-52a4-442d-9356-f05a01aaa1bb/dns/0.log" Sep 08 04:15:57.437294 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:57.437263 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2jhc2_77b984c3-52a4-442d-9356-f05a01aaa1bb/kube-rbac-proxy/0.log" Sep 08 04:15:57.491750 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:57.491720 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-djmk2_ec702c02-1c7e-471b-9783-5593eefc92a3/dns-node-resolver/0.log" Sep 08 04:15:58.074161 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:58.074134 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wmrvz_308e3330-d662-4672-8e7f-9892c04a5491/node-ca/0.log" Sep 08 04:15:59.006235 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:59.006197 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-gmrnx_2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063/discovery/0.log" Sep 08 04:15:59.047035 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:59.047005 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65bcfbd747-vzxhm_ef20b471-07ea-4943-98b3-17ebf9a36b06/kube-auth-proxy/0.log" Sep 08 04:15:59.114681 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:59.114652 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-2qlfp_9f5151f2-bf8a-4222-b50e-38f00d33e36c/payload-pre-processing/0.log" Sep 08 04:15:59.134436 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:59.134395 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-jpfsx_3a77b151-799d-4738-a061-2c30cae4655f/payload-processing/0.log" Sep 08 04:15:59.160419 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:59.160392 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-69bf99dbfb-78zrw_179fd61c-eeab-4eef-b59c-c75528b5c6ee/router/0.log" Sep 08 04:15:59.784323 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:15:59.784293 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fdn8c_9d27b634-b977-4b2b-b5ad-0766b2b0e1e2/serve-healthcheck-canary/0.log" Sep 08 04:16:00.297843 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:00.297812 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-t55cp_ab64adad-879e-4ec6-8425-86a70f48615f/insights-operator/0.log" Sep 08 04:16:00.298222 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:00.297972 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-t55cp_ab64adad-879e-4ec6-8425-86a70f48615f/insights-operator/1.log" Sep 08 04:16:00.433584 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:00.433555 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qkkfr_dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62/kube-rbac-proxy/0.log" Sep 08 04:16:00.450362 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:00.450336 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qkkfr_dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62/exporter/0.log" Sep 08 04:16:00.467625 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:00.467601 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qkkfr_dd2c3b6c-f530-4e7c-b2a0-57877c3f5b62/extractor/0.log" Sep 08 04:16:02.397888 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:02.397860 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qbfdx/perf-node-gather-daemonset-mjvnj" Sep 08 04:16:02.793979 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:02.793895 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-b8484c5b8-qfpx7_00efd044-0152-4ad1-983f-85cc273f7cb3/manager/0.log" Sep 08 04:16:02.808798 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:02.808765 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-b8484c5b8-qfpx7_00efd044-0152-4ad1-983f-85cc273f7cb3/copy-manifests/0.log" Sep 08 04:16:03.919853 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:03.919820 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-jobset-operator_jobset-operator-56d489b4b8-xqxtv_a617ff2e-7bb3-43c5-950b-06e4b9075b92/jobset-operator/0.log" Sep 08 04:16:04.677905 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:04.677872 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-7d548556bc-4nvg5_19c6e60f-a398-4a34-ad94-f544233af88f/manager/0.log" Sep 08 04:16:04.887541 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:04.887512 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-gmrnx_2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063/discovery/0.log" Sep 08 04:16:04.913333 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:04.913307 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65bcfbd747-vzxhm_ef20b471-07ea-4943-98b3-17ebf9a36b06/kube-auth-proxy/0.log" Sep 08 04:16:04.966991 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:04.966913 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-2qlfp_9f5151f2-bf8a-4222-b50e-38f00d33e36c/payload-pre-processing/0.log" Sep 08 04:16:04.982353 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:04.982321 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-jpfsx_3a77b151-799d-4738-a061-2c30cae4655f/payload-processing/0.log" Sep 08 04:16:05.002225 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:05.002197 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-69bf99dbfb-78zrw_179fd61c-eeab-4eef-b59c-c75528b5c6ee/router/0.log" Sep 08 04:16:06.249429 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:06.249399 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-gmrnx_2ff2bfc5-d93b-4e90-8aed-cbcc0e8df063/discovery/0.log" Sep 08 04:16:06.273201 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:06.273170 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65bcfbd747-vzxhm_ef20b471-07ea-4943-98b3-17ebf9a36b06/kube-auth-proxy/0.log" Sep 08 04:16:06.323184 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:06.323155 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-2qlfp_9f5151f2-bf8a-4222-b50e-38f00d33e36c/payload-pre-processing/0.log" Sep 08 04:16:06.339479 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:06.339427 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-jpfsx_3a77b151-799d-4738-a061-2c30cae4655f/payload-processing/0.log" Sep 08 04:16:06.357179 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:06.357155 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-69bf99dbfb-78zrw_179fd61c-eeab-4eef-b59c-c75528b5c6ee/router/0.log" Sep 08 04:16:09.664361 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:09.664333 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-cc86d_166ea36c-7359-47fc-ace9-ad5d46c0f52a/migrator/0.log" Sep 08 04:16:09.680924 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:09.680897 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-cc86d_166ea36c-7359-47fc-ace9-ad5d46c0f52a/graceful-termination/0.log" Sep 08 04:16:10.025715 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:10.025639 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-79hzk_3cd2c98f-1800-40fa-a229-f835cae8915a/kube-storage-version-migrator-operator/1.log" Sep 08 04:16:10.026780 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:10.026761 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-79hzk_3cd2c98f-1800-40fa-a229-f835cae8915a/kube-storage-version-migrator-operator/0.log" Sep 08 04:16:10.918507 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:10.918451 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nft7t_9a603e06-e5e6-4bc5-b352-42bb6dcdbb85/kube-multus-additional-cni-plugins/0.log" Sep 08 04:16:10.935678 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:10.935653 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nft7t_9a603e06-e5e6-4bc5-b352-42bb6dcdbb85/egress-router-binary-copy/0.log" Sep 08 04:16:10.955439 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:10.955410 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nft7t_9a603e06-e5e6-4bc5-b352-42bb6dcdbb85/cni-plugins/0.log" Sep 08 04:16:10.970916 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:10.970888 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nft7t_9a603e06-e5e6-4bc5-b352-42bb6dcdbb85/bond-cni-plugin/0.log" Sep 08 04:16:10.986831 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:10.986798 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nft7t_9a603e06-e5e6-4bc5-b352-42bb6dcdbb85/routeoverride-cni/0.log" Sep 08 04:16:11.002586 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:11.002563 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nft7t_9a603e06-e5e6-4bc5-b352-42bb6dcdbb85/whereabouts-cni-bincopy/0.log" Sep 08 04:16:11.018037 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:11.018016 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nft7t_9a603e06-e5e6-4bc5-b352-42bb6dcdbb85/whereabouts-cni/0.log" Sep 08 04:16:11.285864 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:11.285787 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-l47nd_a60305a7-deaa-4247-b229-9ff28b1ad55b/kube-multus/0.log" Sep 08 04:16:11.403512 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:11.403481 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-r4xfl_b7239679-8052-415f-ae9a-1b20d61d1223/network-metrics-daemon/0.log" Sep 08 04:16:11.419615 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:11.419589 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-r4xfl_b7239679-8052-415f-ae9a-1b20d61d1223/kube-rbac-proxy/0.log" Sep 08 04:16:12.662438 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:12.662405 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tg7hb_45ae9afd-2dea-43b8-aa31-169b11e60fd2/ovn-controller/0.log" Sep 08 04:16:12.676286 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:12.676256 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tg7hb_45ae9afd-2dea-43b8-aa31-169b11e60fd2/ovn-acl-logging/1.log" Sep 08 04:16:12.676286 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:12.676264 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tg7hb_45ae9afd-2dea-43b8-aa31-169b11e60fd2/ovn-acl-logging/0.log" Sep 08 04:16:12.693969 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:12.693930 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tg7hb_45ae9afd-2dea-43b8-aa31-169b11e60fd2/kube-rbac-proxy-node/0.log" Sep 08 04:16:12.715915 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:12.715884 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tg7hb_45ae9afd-2dea-43b8-aa31-169b11e60fd2/kube-rbac-proxy-ovn-metrics/0.log" Sep 08 04:16:12.732680 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:12.732652 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tg7hb_45ae9afd-2dea-43b8-aa31-169b11e60fd2/northd/0.log" Sep 08 04:16:12.750633 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:12.750604 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tg7hb_45ae9afd-2dea-43b8-aa31-169b11e60fd2/nbdb/0.log" Sep 08 04:16:12.772733 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:12.772689 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tg7hb_45ae9afd-2dea-43b8-aa31-169b11e60fd2/sbdb/0.log" Sep 08 04:16:12.825341 ip-10-0-128-141 kubenswrapper[2579]: I0908 04:16:12.825302 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tg7hb_45ae9afd-2dea-43b8-aa31-169b11e60fd2/ovnkube-controller/0.log"