Sep 18 03:33:04.309193 ip-10-0-134-80 systemd[1]: Starting Kubernetes Kubelet... Sep 18 03:33:07.692050 ip-10-0-134-80 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 18 03:33:07.692050 ip-10-0-134-80 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 18 03:33:07.692050 ip-10-0-134-80 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 18 03:33:07.692050 ip-10-0-134-80 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 18 03:33:07.692050 ip-10-0-134-80 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 18 03:33:07.692881 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.692131 2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 18 03:33:07.696390 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696373 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 03:33:07.696390 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696389 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 03:33:07.696390 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696393 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 03:33:07.696491 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696397 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 03:33:07.696491 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696400 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 03:33:07.696491 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696403 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 03:33:07.696491 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696406 2568 feature_gate.go:328] unrecognized feature gate: Example Sep 18 03:33:07.696491 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696410 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 03:33:07.696491 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696413 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 03:33:07.696491 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696416 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 03:33:07.696491 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696419 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 03:33:07.696491 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696422 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 03:33:07.696491 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696424 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 03:33:07.696491 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696428 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 03:33:07.696491 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696431 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 03:33:07.696491 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696433 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 03:33:07.696491 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696436 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 03:33:07.696491 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696438 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 03:33:07.696491 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696442 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 03:33:07.696491 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696445 2568 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 03:33:07.696491 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696447 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 03:33:07.696491 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696450 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 03:33:07.696491 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696453 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 03:33:07.696985 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696455 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 03:33:07.696985 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696458 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 03:33:07.696985 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696460 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 03:33:07.696985 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696463 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 03:33:07.696985 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696465 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 03:33:07.696985 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696468 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 03:33:07.696985 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696470 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 03:33:07.696985 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696473 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 03:33:07.696985 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696476 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 03:33:07.696985 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696478 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 03:33:07.696985 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696481 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 03:33:07.696985 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696483 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 03:33:07.696985 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696487 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 03:33:07.696985 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696491 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 03:33:07.696985 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696495 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 03:33:07.696985 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696497 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 03:33:07.696985 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696501 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 03:33:07.696985 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696504 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 03:33:07.696985 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696507 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 03:33:07.696985 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696510 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 03:33:07.697482 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696513 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 03:33:07.697482 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696515 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 03:33:07.697482 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696520 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 03:33:07.697482 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696524 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 03:33:07.697482 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696526 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 03:33:07.697482 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696529 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 03:33:07.697482 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696532 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 03:33:07.697482 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696535 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 03:33:07.697482 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696538 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 03:33:07.697482 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696541 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 03:33:07.697482 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696544 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 03:33:07.697482 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696546 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 03:33:07.697482 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696549 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 03:33:07.697482 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696554 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 03:33:07.697482 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696557 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 03:33:07.697482 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696560 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 03:33:07.697482 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696563 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 03:33:07.697482 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696566 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 03:33:07.697482 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696569 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 03:33:07.697965 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696572 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 03:33:07.697965 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696574 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 03:33:07.697965 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696577 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 03:33:07.697965 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696580 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 03:33:07.697965 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696583 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 03:33:07.697965 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696586 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 03:33:07.697965 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696589 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 03:33:07.697965 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696591 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 03:33:07.697965 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696594 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 03:33:07.697965 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696602 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 03:33:07.697965 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696605 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 03:33:07.697965 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696608 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 03:33:07.697965 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696610 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 03:33:07.697965 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696613 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 03:33:07.697965 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696615 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 03:33:07.697965 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696618 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 03:33:07.697965 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696621 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 03:33:07.697965 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696623 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 03:33:07.697965 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696626 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 03:33:07.697965 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696628 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 03:33:07.698488 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696631 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 03:33:07.698488 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696633 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 03:33:07.698488 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696636 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 03:33:07.698488 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696639 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 03:33:07.698488 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696641 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 03:33:07.698488 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696644 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 03:33:07.698488 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696647 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 03:33:07.698488 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696650 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 03:33:07.698488 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696652 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 03:33:07.698488 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696655 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 03:33:07.698488 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696658 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 03:33:07.698488 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696662 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 03:33:07.698488 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696666 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 03:33:07.698488 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696669 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 03:33:07.698488 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696672 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 03:33:07.698488 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696674 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 03:33:07.698488 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.696677 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 03:33:07.698488 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697357 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 03:33:07.698488 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697364 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 03:33:07.698966 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697367 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 03:33:07.698966 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697371 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 03:33:07.698966 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697375 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 03:33:07.698966 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697380 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 03:33:07.698966 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697383 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 03:33:07.698966 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697386 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 03:33:07.698966 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697389 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 03:33:07.698966 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697392 2568 feature_gate.go:328] unrecognized feature gate: Example Sep 18 03:33:07.698966 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697395 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 03:33:07.698966 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697398 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 03:33:07.698966 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697401 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 03:33:07.698966 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697403 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 03:33:07.698966 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697406 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 03:33:07.698966 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697409 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 03:33:07.698966 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697411 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 03:33:07.698966 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697415 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 03:33:07.698966 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697417 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 03:33:07.698966 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697420 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 03:33:07.698966 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697423 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 03:33:07.699448 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697425 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 03:33:07.699448 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697428 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 03:33:07.699448 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697430 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 03:33:07.699448 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697434 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 03:33:07.699448 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697438 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 03:33:07.699448 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697441 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 03:33:07.699448 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697443 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 03:33:07.699448 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697446 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 03:33:07.699448 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697449 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 03:33:07.699448 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697451 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 03:33:07.699448 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697454 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 03:33:07.699448 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697456 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 03:33:07.699448 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697459 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 03:33:07.699448 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697462 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 03:33:07.699448 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697465 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 03:33:07.699448 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697467 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 03:33:07.699448 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697470 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 03:33:07.699448 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697473 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 03:33:07.699448 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697476 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 03:33:07.699919 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697478 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 03:33:07.699919 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697481 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 03:33:07.699919 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697484 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 03:33:07.699919 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697487 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 03:33:07.699919 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697489 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 03:33:07.699919 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697492 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 03:33:07.699919 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697495 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 03:33:07.699919 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697498 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 03:33:07.699919 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697500 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 03:33:07.699919 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697504 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 03:33:07.699919 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697507 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 03:33:07.699919 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697510 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 03:33:07.699919 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697512 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 03:33:07.699919 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697515 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 03:33:07.699919 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697517 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 03:33:07.699919 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697520 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 03:33:07.699919 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697523 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 03:33:07.699919 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697525 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 03:33:07.699919 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697528 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 03:33:07.699919 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697530 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 03:33:07.700432 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697533 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 03:33:07.700432 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697536 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 03:33:07.700432 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697538 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 03:33:07.700432 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697541 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 03:33:07.700432 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697543 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 03:33:07.700432 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697546 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 03:33:07.700432 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697549 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 03:33:07.700432 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697552 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 03:33:07.700432 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697554 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 03:33:07.700432 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697557 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 03:33:07.700432 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697559 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 03:33:07.700432 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697563 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 03:33:07.700432 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697565 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 03:33:07.700432 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697568 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 03:33:07.700432 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697571 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 03:33:07.700432 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697574 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 03:33:07.700432 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697576 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 03:33:07.700432 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697579 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 03:33:07.700432 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697581 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 03:33:07.700432 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697586 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 03:33:07.700922 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697589 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 03:33:07.700922 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697592 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 03:33:07.700922 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697594 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 03:33:07.700922 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697597 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 03:33:07.700922 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697600 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 03:33:07.700922 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697602 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 03:33:07.700922 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697605 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 03:33:07.700922 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697608 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 03:33:07.700922 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697610 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 03:33:07.700922 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697613 2568 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 03:33:07.700922 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697616 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 03:33:07.700922 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697618 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 03:33:07.700922 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697621 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 03:33:07.700922 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697624 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 03:33:07.700922 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697626 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 03:33:07.700922 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697629 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 03:33:07.700922 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697634 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 03:33:07.700922 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697636 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 03:33:07.700922 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.697639 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 03:33:07.700922 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698033 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 03:33:07.701419 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698038 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 03:33:07.701419 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698041 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 03:33:07.701419 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698051 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 03:33:07.701419 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698056 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 03:33:07.701419 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698059 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 03:33:07.701419 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698062 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 03:33:07.701419 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698065 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 03:33:07.701419 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698068 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 03:33:07.701419 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698071 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 03:33:07.701419 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698074 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 03:33:07.701419 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698077 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 03:33:07.701419 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698081 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 03:33:07.701419 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698084 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 03:33:07.701419 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698086 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 03:33:07.701419 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698089 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 03:33:07.701419 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698092 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 03:33:07.701419 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698094 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 03:33:07.701419 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698097 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 03:33:07.701419 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698100 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 03:33:07.701921 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698104 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 03:33:07.701921 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698108 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 03:33:07.701921 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698110 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 03:33:07.701921 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698113 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 03:33:07.701921 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698116 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 03:33:07.701921 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698119 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 03:33:07.701921 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698122 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 03:33:07.701921 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698124 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 03:33:07.701921 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698127 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 03:33:07.701921 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698131 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 03:33:07.701921 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698133 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 03:33:07.701921 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698136 2568 feature_gate.go:328] unrecognized feature gate: Example Sep 18 03:33:07.701921 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698139 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 03:33:07.701921 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698142 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 03:33:07.701921 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698144 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 03:33:07.701921 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698147 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 03:33:07.701921 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698151 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 03:33:07.701921 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698155 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 03:33:07.701921 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698159 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 03:33:07.702400 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698161 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 03:33:07.702400 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698164 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 03:33:07.702400 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698167 2568 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 03:33:07.702400 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698169 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 03:33:07.702400 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698172 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 03:33:07.702400 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698178 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 03:33:07.702400 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698180 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 03:33:07.702400 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698183 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 03:33:07.702400 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698186 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 03:33:07.702400 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698188 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 03:33:07.702400 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698191 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 03:33:07.702400 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698193 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 03:33:07.702400 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698196 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 03:33:07.702400 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698198 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 03:33:07.702400 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698201 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 03:33:07.702400 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698204 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 03:33:07.702400 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698206 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 03:33:07.702400 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698209 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 03:33:07.702400 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698212 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 03:33:07.702400 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698214 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 03:33:07.702890 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698217 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 03:33:07.702890 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698219 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 03:33:07.702890 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698223 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 03:33:07.702890 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698226 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 03:33:07.702890 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698228 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 03:33:07.702890 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698231 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 03:33:07.702890 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698233 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 03:33:07.702890 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698236 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 03:33:07.702890 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698238 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 03:33:07.702890 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698241 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 03:33:07.702890 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698243 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 03:33:07.702890 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698247 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 03:33:07.702890 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698249 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 03:33:07.702890 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698252 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 03:33:07.702890 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698254 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 03:33:07.702890 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698257 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 03:33:07.702890 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698260 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 03:33:07.702890 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698265 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 03:33:07.702890 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698268 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 03:33:07.702890 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698270 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 03:33:07.703435 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698273 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 03:33:07.703435 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698276 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 03:33:07.703435 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698278 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 03:33:07.703435 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698281 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 03:33:07.703435 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698284 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 03:33:07.703435 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698286 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 03:33:07.703435 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698288 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 03:33:07.703435 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698291 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 03:33:07.703435 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698293 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 03:33:07.703435 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698297 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 03:33:07.703435 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698300 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 03:33:07.703435 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698303 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 03:33:07.703435 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698306 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 03:33:07.703435 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698308 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 03:33:07.703435 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698312 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 03:33:07.703435 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698315 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 03:33:07.703435 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698317 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 03:33:07.703435 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698320 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 03:33:07.703435 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698323 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 03:33:07.703435 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.698325 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 03:33:07.703943 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698395 2568 flags.go:64] FLAG: --address="0.0.0.0" Sep 18 03:33:07.703943 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698435 2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 18 03:33:07.703943 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698441 2568 flags.go:64] FLAG: --anonymous-auth="true" Sep 18 03:33:07.703943 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698445 2568 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 18 03:33:07.703943 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698450 2568 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 18 03:33:07.703943 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698453 2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 18 03:33:07.703943 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698458 2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 18 03:33:07.703943 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698462 2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 18 03:33:07.703943 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698465 2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 18 03:33:07.703943 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698470 2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 18 03:33:07.703943 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698474 2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 18 03:33:07.703943 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698478 2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 18 03:33:07.703943 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698481 2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 18 03:33:07.703943 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698484 2568 flags.go:64] FLAG: --cgroup-root="" Sep 18 03:33:07.703943 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698487 2568 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 18 03:33:07.703943 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698490 2568 flags.go:64] FLAG: --client-ca-file="" Sep 18 03:33:07.703943 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698493 2568 flags.go:64] FLAG: --cloud-provider="external" Sep 18 03:33:07.703943 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698497 2568 flags.go:64] FLAG: --cluster-dns="[]" Sep 18 03:33:07.703943 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698501 2568 flags.go:64] FLAG: --cluster-domain="" Sep 18 03:33:07.703943 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698504 2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 18 03:33:07.703943 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698508 2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 18 03:33:07.703943 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698511 2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 18 03:33:07.703943 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698515 2568 flags.go:64] FLAG: --container-log-max-files="5" Sep 18 03:33:07.703943 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698519 2568 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 18 03:33:07.704536 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698522 2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 18 03:33:07.704536 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698525 2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 18 03:33:07.704536 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698530 2568 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 18 03:33:07.704536 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698533 2568 flags.go:64] FLAG: --contention-profiling="false" Sep 18 03:33:07.704536 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698536 2568 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 18 03:33:07.704536 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698539 2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 18 03:33:07.704536 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698543 2568 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 18 03:33:07.704536 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698546 2568 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 18 03:33:07.704536 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698550 2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 18 03:33:07.704536 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698554 2568 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 18 03:33:07.704536 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698557 2568 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 18 03:33:07.704536 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698559 2568 flags.go:64] FLAG: --enable-load-reader="false" Sep 18 03:33:07.704536 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698562 2568 flags.go:64] FLAG: --enable-server="true" Sep 18 03:33:07.704536 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698566 2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 18 03:33:07.704536 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698571 2568 flags.go:64] FLAG: --event-burst="100" Sep 18 03:33:07.704536 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698574 2568 flags.go:64] FLAG: --event-qps="50" Sep 18 03:33:07.704536 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698577 2568 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 18 03:33:07.704536 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698581 2568 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 18 03:33:07.704536 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698584 2568 flags.go:64] FLAG: --eviction-hard="" Sep 18 03:33:07.704536 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698587 2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 18 03:33:07.704536 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698590 2568 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 18 03:33:07.704536 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698593 2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 18 03:33:07.704536 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698597 2568 flags.go:64] FLAG: --eviction-soft="" Sep 18 03:33:07.704536 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698600 2568 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 18 03:33:07.704536 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698603 2568 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 18 03:33:07.705176 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698606 2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 18 03:33:07.705176 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698609 2568 flags.go:64] FLAG: --experimental-mounter-path="" Sep 18 03:33:07.705176 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698613 2568 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 18 03:33:07.705176 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698615 2568 flags.go:64] FLAG: --fail-swap-on="true" Sep 18 03:33:07.705176 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698618 2568 flags.go:64] FLAG: --feature-gates="" Sep 18 03:33:07.705176 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698622 2568 flags.go:64] FLAG: --file-check-frequency="20s" Sep 18 03:33:07.705176 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698625 2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 18 03:33:07.705176 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698628 2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 18 03:33:07.705176 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698631 2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 18 03:33:07.705176 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698634 2568 flags.go:64] FLAG: --healthz-port="10248" Sep 18 03:33:07.705176 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698638 2568 flags.go:64] FLAG: --help="false" Sep 18 03:33:07.705176 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698641 2568 flags.go:64] FLAG: --hostname-override="ip-10-0-134-80.ec2.internal" Sep 18 03:33:07.705176 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698644 2568 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 18 03:33:07.705176 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698647 2568 flags.go:64] FLAG: --http-check-frequency="20s" Sep 18 03:33:07.705176 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698650 2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 18 03:33:07.705176 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698653 2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 18 03:33:07.705176 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698657 2568 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 18 03:33:07.705176 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698659 2568 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 18 03:33:07.705176 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698663 2568 flags.go:64] FLAG: --image-service-endpoint="" Sep 18 03:33:07.705176 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698665 2568 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 18 03:33:07.705176 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698669 2568 flags.go:64] FLAG: --kube-api-burst="100" Sep 18 03:33:07.705176 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698672 2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 18 03:33:07.705176 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698676 2568 flags.go:64] FLAG: --kube-api-qps="50" Sep 18 03:33:07.705176 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698679 2568 flags.go:64] FLAG: --kube-reserved="" Sep 18 03:33:07.705774 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698682 2568 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 18 03:33:07.705774 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698685 2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 18 03:33:07.705774 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698688 2568 flags.go:64] FLAG: --kubelet-cgroups="" Sep 18 03:33:07.705774 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698691 2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 18 03:33:07.705774 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698694 2568 flags.go:64] FLAG: --lock-file="" Sep 18 03:33:07.705774 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698697 2568 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 18 03:33:07.705774 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698700 2568 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 18 03:33:07.705774 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698703 2568 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 18 03:33:07.705774 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698708 2568 flags.go:64] FLAG: --log-json-split-stream="false" Sep 18 03:33:07.705774 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698711 2568 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 18 03:33:07.705774 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698714 2568 flags.go:64] FLAG: --log-text-split-stream="false" Sep 18 03:33:07.705774 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698717 2568 flags.go:64] FLAG: --logging-format="text" Sep 18 03:33:07.705774 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698720 2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 18 03:33:07.705774 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698723 2568 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 18 03:33:07.705774 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698726 2568 flags.go:64] FLAG: --manifest-url="" Sep 18 03:33:07.705774 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698729 2568 flags.go:64] FLAG: --manifest-url-header="" Sep 18 03:33:07.705774 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698733 2568 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 18 03:33:07.705774 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698736 2568 flags.go:64] FLAG: --max-open-files="1000000" Sep 18 03:33:07.705774 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698740 2568 flags.go:64] FLAG: --max-pods="110" Sep 18 03:33:07.705774 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698743 2568 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 18 03:33:07.705774 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698746 2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 18 03:33:07.705774 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698749 2568 flags.go:64] FLAG: --memory-manager-policy="None" Sep 18 03:33:07.705774 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698752 2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 18 03:33:07.705774 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698755 2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 18 03:33:07.705774 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698758 2568 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 18 03:33:07.706439 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698761 2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 18 03:33:07.706439 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698768 2568 flags.go:64] FLAG: --node-status-max-images="50" Sep 18 03:33:07.706439 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698772 2568 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 18 03:33:07.706439 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698775 2568 flags.go:64] FLAG: --oom-score-adj="-999" Sep 18 03:33:07.706439 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698778 2568 flags.go:64] FLAG: --pod-cidr="" Sep 18 03:33:07.706439 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698781 2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc" Sep 18 03:33:07.706439 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698787 2568 flags.go:64] FLAG: --pod-manifest-path="" Sep 18 03:33:07.706439 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698790 2568 flags.go:64] FLAG: --pod-max-pids="-1" Sep 18 03:33:07.706439 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698793 2568 flags.go:64] FLAG: --pods-per-core="0" Sep 18 03:33:07.706439 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698796 2568 flags.go:64] FLAG: --port="10250" Sep 18 03:33:07.706439 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698799 2568 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 18 03:33:07.706439 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698802 2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f34a77daf16ede1a" Sep 18 03:33:07.706439 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698805 2568 flags.go:64] FLAG: --qos-reserved="" Sep 18 03:33:07.706439 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698808 2568 flags.go:64] FLAG: --read-only-port="10255" Sep 18 03:33:07.706439 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698811 2568 flags.go:64] FLAG: --register-node="true" Sep 18 03:33:07.706439 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698813 2568 flags.go:64] FLAG: --register-with-taints="" Sep 18 03:33:07.706439 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698817 2568 flags.go:64] FLAG: --registry-burst="10" Sep 18 03:33:07.706439 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698820 2568 flags.go:64] FLAG: --registry-qps="5" Sep 18 03:33:07.706439 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698823 2568 flags.go:64] FLAG: --reserved-cpus="" Sep 18 03:33:07.706439 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698825 2568 flags.go:64] FLAG: --reserved-memory="" Sep 18 03:33:07.706439 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698829 2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 18 03:33:07.706439 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698832 2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 18 03:33:07.706439 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698835 2568 flags.go:64] FLAG: --rotate-certificates="false" Sep 18 03:33:07.706439 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698838 2568 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 18 03:33:07.706439 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698841 2568 flags.go:64] FLAG: --runonce="false" Sep 18 03:33:07.707135 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698844 2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 18 03:33:07.707135 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698847 2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 18 03:33:07.707135 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698850 2568 flags.go:64] FLAG: --seccomp-default="false" Sep 18 03:33:07.707135 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698853 2568 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 18 03:33:07.707135 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698856 2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 18 03:33:07.707135 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698859 2568 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 18 03:33:07.707135 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698863 2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 18 03:33:07.707135 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698866 2568 flags.go:64] FLAG: --storage-driver-password="root" Sep 18 03:33:07.707135 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698869 2568 flags.go:64] FLAG: --storage-driver-secure="false" Sep 18 03:33:07.707135 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698872 2568 flags.go:64] FLAG: --storage-driver-table="stats" Sep 18 03:33:07.707135 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698874 2568 flags.go:64] FLAG: --storage-driver-user="root" Sep 18 03:33:07.707135 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698878 2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 18 03:33:07.707135 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698881 2568 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 18 03:33:07.707135 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698884 2568 flags.go:64] FLAG: --system-cgroups="" Sep 18 03:33:07.707135 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698887 2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 18 03:33:07.707135 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698893 2568 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 18 03:33:07.707135 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698896 2568 flags.go:64] FLAG: --tls-cert-file="" Sep 18 03:33:07.707135 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698898 2568 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 18 03:33:07.707135 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698903 2568 flags.go:64] FLAG: --tls-min-version="" Sep 18 03:33:07.707135 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698906 2568 flags.go:64] FLAG: --tls-private-key-file="" Sep 18 03:33:07.707135 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698908 2568 flags.go:64] FLAG: --topology-manager-policy="none" Sep 18 03:33:07.707135 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698911 2568 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 18 03:33:07.707135 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698914 2568 flags.go:64] FLAG: --topology-manager-scope="container" Sep 18 03:33:07.707135 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698917 2568 flags.go:64] FLAG: --v="2" Sep 18 03:33:07.707135 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698921 2568 flags.go:64] FLAG: --version="false" Sep 18 03:33:07.707748 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698925 2568 flags.go:64] FLAG: --vmodule="" Sep 18 03:33:07.707748 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698941 2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 18 03:33:07.707748 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.698945 2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 18 03:33:07.707748 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699058 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 03:33:07.707748 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699061 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 03:33:07.707748 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699065 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 03:33:07.707748 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699071 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 03:33:07.707748 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699082 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 03:33:07.707748 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699086 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 03:33:07.707748 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699090 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 03:33:07.707748 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699093 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 03:33:07.707748 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699096 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 03:33:07.707748 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699099 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 03:33:07.707748 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699102 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 03:33:07.707748 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699105 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 03:33:07.707748 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699108 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 03:33:07.707748 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699110 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 03:33:07.707748 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699113 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 03:33:07.707748 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699116 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 03:33:07.707748 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699119 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 03:33:07.708272 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699122 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 03:33:07.708272 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699125 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 03:33:07.708272 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699128 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 03:33:07.708272 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699131 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 03:33:07.708272 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699133 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 03:33:07.708272 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699136 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 03:33:07.708272 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699139 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 03:33:07.708272 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699142 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 03:33:07.708272 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699144 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 03:33:07.708272 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699147 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 03:33:07.708272 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699149 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 03:33:07.708272 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699152 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 03:33:07.708272 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699154 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 03:33:07.708272 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699157 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 03:33:07.708272 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699159 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 03:33:07.708272 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699162 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 03:33:07.708272 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699164 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 03:33:07.708272 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699167 2568 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 03:33:07.708272 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699172 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 03:33:07.708272 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699175 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 03:33:07.709091 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699178 2568 feature_gate.go:328] unrecognized feature gate: Example Sep 18 03:33:07.709091 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699181 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 03:33:07.709091 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699183 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 03:33:07.709091 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699186 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 03:33:07.709091 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699189 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 03:33:07.709091 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699191 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 03:33:07.709091 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699194 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 03:33:07.709091 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699197 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 03:33:07.709091 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699199 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 03:33:07.709091 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699202 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 03:33:07.709091 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699205 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 03:33:07.709091 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699207 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 03:33:07.709091 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699210 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 03:33:07.709091 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699212 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 03:33:07.709091 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699215 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 03:33:07.709091 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699218 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 03:33:07.709091 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699220 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 03:33:07.709091 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699223 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 03:33:07.709091 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699225 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 03:33:07.709091 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699228 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 03:33:07.709734 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699231 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 03:33:07.709734 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699233 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 03:33:07.709734 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699236 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 03:33:07.709734 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699238 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 03:33:07.709734 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699241 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 03:33:07.709734 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699244 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 03:33:07.709734 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699246 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 03:33:07.709734 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699249 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 03:33:07.709734 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699251 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 03:33:07.709734 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699254 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 03:33:07.709734 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699259 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 03:33:07.709734 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699261 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 03:33:07.709734 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699264 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 03:33:07.709734 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699267 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 03:33:07.709734 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699269 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 03:33:07.709734 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699272 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 03:33:07.709734 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699274 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 03:33:07.709734 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699277 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 03:33:07.709734 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699280 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 03:33:07.710301 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699282 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 03:33:07.710301 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699286 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 03:33:07.710301 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699290 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 03:33:07.710301 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699293 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 03:33:07.710301 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699295 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 03:33:07.710301 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699298 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 03:33:07.710301 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699301 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 03:33:07.710301 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699304 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 03:33:07.710301 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699307 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 03:33:07.710301 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699310 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 03:33:07.710301 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699312 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 03:33:07.710301 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699315 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 03:33:07.710301 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699317 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 03:33:07.710301 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699320 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 03:33:07.710301 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699323 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 03:33:07.710301 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699325 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 03:33:07.710301 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699328 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 03:33:07.710301 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699330 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 03:33:07.710301 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699333 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 03:33:07.710853 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699336 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 03:33:07.710853 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699338 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 03:33:07.710853 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699342 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 03:33:07.710853 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.699346 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 03:33:07.710853 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.699817 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 18 03:33:07.710853 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.707407 2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 18 03:33:07.710853 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.707422 2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 18 03:33:07.710853 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.707441 2568 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 18 03:33:07.710853 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.707447 2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 18 03:33:07.710853 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707501 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 03:33:07.710853 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707505 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 03:33:07.710853 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707509 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 03:33:07.710853 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707512 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 03:33:07.710853 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707515 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 03:33:07.710853 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707518 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 03:33:07.711291 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707520 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 03:33:07.711291 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707523 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 03:33:07.711291 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707526 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 03:33:07.711291 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707528 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 03:33:07.711291 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707531 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 03:33:07.711291 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707533 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 03:33:07.711291 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707538 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 03:33:07.711291 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707542 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 03:33:07.711291 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707545 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 03:33:07.711291 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707548 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 03:33:07.711291 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707550 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 03:33:07.711291 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707553 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 03:33:07.711291 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707555 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 03:33:07.711291 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707558 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 03:33:07.711291 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707561 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 03:33:07.711291 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707563 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 03:33:07.711291 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707566 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 03:33:07.711291 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707569 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 03:33:07.711291 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707571 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 03:33:07.711813 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707574 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 03:33:07.711813 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707577 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 03:33:07.711813 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707580 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 03:33:07.711813 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707582 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 03:33:07.711813 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707587 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 03:33:07.711813 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707590 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 03:33:07.711813 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707592 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 03:33:07.711813 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707595 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 03:33:07.711813 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707597 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 03:33:07.711813 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707600 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 03:33:07.711813 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707603 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 03:33:07.711813 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707605 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 03:33:07.711813 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707608 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 03:33:07.711813 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707611 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 03:33:07.711813 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707613 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 03:33:07.711813 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707616 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 03:33:07.711813 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707618 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 03:33:07.711813 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707621 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 03:33:07.711813 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707623 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 03:33:07.712353 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707626 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 03:33:07.712353 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707629 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 03:33:07.712353 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707632 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 03:33:07.712353 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707634 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 03:33:07.712353 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707637 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 03:33:07.712353 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707639 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 03:33:07.712353 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707642 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 03:33:07.712353 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707644 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 03:33:07.712353 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707648 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 03:33:07.712353 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707651 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 03:33:07.712353 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707654 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 03:33:07.712353 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707657 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 03:33:07.712353 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707660 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 03:33:07.712353 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707663 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 03:33:07.712353 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707666 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 03:33:07.712353 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707669 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 03:33:07.712353 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707672 2568 feature_gate.go:328] unrecognized feature gate: Example Sep 18 03:33:07.712353 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707675 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 03:33:07.712353 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707679 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 03:33:07.712353 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707682 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 03:33:07.712888 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707685 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 03:33:07.712888 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707688 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 03:33:07.712888 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707690 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 03:33:07.712888 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707694 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 03:33:07.712888 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707696 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 03:33:07.712888 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707699 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 03:33:07.712888 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707702 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 03:33:07.712888 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707706 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 03:33:07.712888 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707709 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 03:33:07.712888 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707711 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 03:33:07.712888 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707714 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 03:33:07.712888 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707716 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 03:33:07.712888 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707718 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 03:33:07.712888 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707721 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 03:33:07.712888 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707724 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 03:33:07.712888 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707726 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 03:33:07.712888 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707729 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 03:33:07.712888 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707731 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 03:33:07.712888 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707734 2568 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 03:33:07.712888 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707736 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 03:33:07.713446 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707739 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 03:33:07.713446 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707741 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 03:33:07.713446 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707744 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 03:33:07.713446 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707747 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 03:33:07.713446 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707750 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 03:33:07.713446 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707753 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 03:33:07.713446 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707755 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 03:33:07.713446 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707758 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 03:33:07.713446 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707761 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 03:33:07.713446 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707764 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 03:33:07.713446 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707767 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 03:33:07.713446 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707770 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 03:33:07.713446 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707773 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 03:33:07.713446 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707775 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 03:33:07.713446 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707778 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 03:33:07.713839 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.707782 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 18 03:33:07.713839 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707898 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 03:33:07.713839 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707904 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 03:33:07.713839 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707907 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 03:33:07.713839 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707909 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 03:33:07.713839 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707912 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 03:33:07.713839 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707915 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 03:33:07.713839 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707918 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 03:33:07.713839 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707921 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 03:33:07.713839 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707923 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 03:33:07.713839 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707926 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 03:33:07.713839 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707943 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 03:33:07.713839 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707947 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 03:33:07.713839 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707950 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 03:33:07.713839 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707953 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 03:33:07.714337 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707956 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 03:33:07.714337 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707959 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 03:33:07.714337 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707961 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 03:33:07.714337 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707964 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 03:33:07.714337 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707967 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 03:33:07.714337 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707969 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 03:33:07.714337 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707972 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 03:33:07.714337 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707974 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 03:33:07.714337 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707977 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 03:33:07.714337 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707980 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 03:33:07.714337 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707982 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 03:33:07.714337 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707986 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 03:33:07.714337 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707989 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 03:33:07.714337 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707991 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 03:33:07.714337 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707995 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 03:33:07.714337 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.707997 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 03:33:07.714337 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708000 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 03:33:07.714337 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708003 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 03:33:07.714337 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708005 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 03:33:07.714337 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708008 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 03:33:07.714819 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708010 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 03:33:07.714819 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708013 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 03:33:07.714819 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708015 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 03:33:07.714819 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708018 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 03:33:07.714819 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708020 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 03:33:07.714819 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708023 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 03:33:07.714819 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708025 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 03:33:07.714819 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708028 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 03:33:07.714819 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708030 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 03:33:07.714819 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708033 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 03:33:07.714819 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708035 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 03:33:07.714819 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708038 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 03:33:07.714819 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708041 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 03:33:07.714819 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708043 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 03:33:07.714819 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708045 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 03:33:07.714819 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708049 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 03:33:07.714819 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708053 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 03:33:07.714819 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708056 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 03:33:07.714819 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708059 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 03:33:07.715304 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708062 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 03:33:07.715304 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708065 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 03:33:07.715304 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708067 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 03:33:07.715304 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708071 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 03:33:07.715304 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708074 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 03:33:07.715304 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708077 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 03:33:07.715304 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708080 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 03:33:07.715304 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708083 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 03:33:07.715304 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708086 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 03:33:07.715304 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708088 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 03:33:07.715304 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708091 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 03:33:07.715304 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708093 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 03:33:07.715304 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708096 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 03:33:07.715304 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708098 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 03:33:07.715304 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708101 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 03:33:07.715304 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708103 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 03:33:07.715304 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708106 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 03:33:07.715304 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708108 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 03:33:07.715304 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708111 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 03:33:07.715775 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708113 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 03:33:07.715775 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708116 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 03:33:07.715775 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708118 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 03:33:07.715775 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708121 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 03:33:07.715775 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708123 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 03:33:07.715775 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708126 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 03:33:07.715775 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708128 2568 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 03:33:07.715775 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708131 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 03:33:07.715775 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708133 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 03:33:07.715775 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708136 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 03:33:07.715775 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708139 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 03:33:07.715775 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708141 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 03:33:07.715775 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708143 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 03:33:07.715775 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708146 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 03:33:07.715775 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708148 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 03:33:07.715775 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708151 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 03:33:07.715775 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708155 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 03:33:07.715775 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708159 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 03:33:07.715775 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708162 2568 feature_gate.go:328] unrecognized feature gate: Example Sep 18 03:33:07.716260 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708165 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 03:33:07.716260 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708167 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 03:33:07.716260 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708170 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 03:33:07.716260 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708209 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 03:33:07.716260 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708312 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 03:33:07.716260 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708318 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 03:33:07.716260 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708323 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 03:33:07.716260 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:07.708327 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 03:33:07.716260 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.708336 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 18 03:33:07.716260 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.708610 2568 server.go:962] "Client rotation is on, will bootstrap in background" Sep 18 03:33:07.716260 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.711218 2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 18 03:33:07.716260 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.712238 2568 server.go:1019] "Starting client certificate rotation" Sep 18 03:33:07.716260 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.712337 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 18 03:33:07.716260 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.712364 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 18 03:33:07.716615 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.714579 2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 18 03:33:07.716680 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.716615 2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 18 03:33:07.721606 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.721587 2568 log.go:25] "Validated CRI v1 runtime API" Sep 18 03:33:07.723243 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.723228 2568 log.go:25] "Validated CRI v1 image API" Sep 18 03:33:07.723987 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.723972 2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 18 03:33:07.730819 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.730799 2568 fs.go:135] Filesystem UUIDs: map[161b3ec2-2f1b-4bd0-ab3c-08edc2df60b3:/dev/nvme0n1p4 7202fb94-5534-4675-af22-74a54a1f2c02:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Sep 18 03:33:07.730891 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.730816 2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 18 03:33:07.735042 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.734919 2568 manager.go:216] Machine: {Timestamp:2026-09-18 03:33:07.734447456 +0000 UTC m=+2.017209310 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097953 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec229df060b63cafdff3d9143d16a4e6 SystemUUID:ec229df0-60b6-3caf-dff3-d9143d16a4e6 BootID:e6d093cb-44d0-4075-867d-74ba64c3f742 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e5:d5:90:49 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e5:d5:90:49 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:76:a2:e8:0d:e4:67 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 18 03:33:07.735042 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.735036 2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 18 03:33:07.735152 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.735143 2568 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 18 03:33:07.736687 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.736665 2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 18 03:33:07.736811 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.736689 2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-80.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 18 03:33:07.738019 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.738007 2568 topology_manager.go:138] "Creating topology manager with none policy" Sep 18 03:33:07.738067 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.738020 2568 container_manager_linux.go:309] "Creating device plugin manager" Sep 18 03:33:07.738067 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.738027 2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 18 03:33:07.738067 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.738037 2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 18 03:33:07.739591 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.739581 2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 18 03:33:07.743322 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.743310 2568 state_mem.go:36] "Initialized new in-memory state store" Sep 18 03:33:07.743432 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.743423 2568 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 18 03:33:07.743469 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.743458 2568 kubelet.go:486] "Attempting to sync node with API server" Sep 18 03:33:07.743469 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.743468 2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 18 03:33:07.743523 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.743482 2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 18 03:33:07.743523 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.743491 2568 kubelet.go:398] "Adding apiserver pod source" Sep 18 03:33:07.743523 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.743501 2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 18 03:33:07.743830 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.743819 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 18 03:33:07.743873 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.743844 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 18 03:33:07.751390 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.751261 2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1" Sep 18 03:33:07.753324 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.753304 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 18 03:33:07.753960 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.753945 2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 18 03:33:07.754010 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.753977 2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 18 03:33:07.754167 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.754155 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 18 03:33:07.754204 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.754176 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 18 03:33:07.754204 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.754188 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 18 03:33:07.754204 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.754197 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 18 03:33:07.754287 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.754209 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 18 03:33:07.754287 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.754218 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 18 03:33:07.754656 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.754647 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 18 03:33:07.754685 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.754657 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 18 03:33:07.754685 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.754664 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 18 03:33:07.754685 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.754669 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 18 03:33:07.754685 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.754677 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 18 03:33:07.754685 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.754685 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 18 03:33:07.754807 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.754709 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 18 03:33:07.754807 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.754714 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 18 03:33:07.757196 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.757178 2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-80.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 18 03:33:07.757326 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:07.757302 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-80.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 18 03:33:07.757379 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:07.757343 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 18 03:33:07.758300 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.758286 2568 server.go:1268] "Started kubelet" Sep 18 03:33:07.758412 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.758379 2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 18 03:33:07.758512 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.758344 2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 18 03:33:07.758589 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.758575 2568 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 18 03:33:07.758831 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.758819 2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 18 03:33:07.759084 ip-10-0-134-80 systemd[1]: Started Kubernetes Kubelet. Sep 18 03:33:07.762989 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.762975 2568 server.go:310] "Adding debug handlers to kubelet server" Sep 18 03:33:07.764259 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:07.764230 2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 18 03:33:07.771490 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.771473 2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 18 03:33:07.771560 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.771487 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 18 03:33:07.771721 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:07.771704 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-80.ec2.internal\" not found" Sep 18 03:33:07.771988 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.771973 2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 18 03:33:07.772093 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.771973 2568 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 18 03:33:07.772520 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.772508 2568 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 18 03:33:07.772641 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.772626 2568 reconstruct.go:97] "Volume reconstruction finished" Sep 18 03:33:07.772707 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.772643 2568 reconciler.go:29] "Reconciler: start to sync state" Sep 18 03:33:07.772707 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.772647 2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 18 03:33:07.772707 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.772664 2568 factory.go:55] Registering systemd factory Sep 18 03:33:07.772707 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.772673 2568 factory.go:223] Registration of the systemd container factory successfully Sep 18 03:33:07.773031 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.773012 2568 factory.go:153] Registering CRI-O factory Sep 18 03:33:07.773031 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.773033 2568 factory.go:223] Registration of the crio container factory successfully Sep 18 03:33:07.773143 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.773053 2568 factory.go:103] Registering Raw factory Sep 18 03:33:07.773143 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.773066 2568 manager.go:1196] Started watching for new ooms in manager Sep 18 03:33:07.773143 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.773011 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xh59t" Sep 18 03:33:07.773454 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.773441 2568 manager.go:349] Starting recovery of all containers Sep 18 03:33:07.773590 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:07.773569 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 18 03:33:07.773694 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:07.773677 2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-80.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 18 03:33:07.789998 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.789972 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xh59t" Sep 18 03:33:07.792634 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.792619 2568 manager.go:354] Recovery completed Sep 18 03:33:07.796472 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.796456 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 03:33:07.798327 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.798313 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-80.ec2.internal" event="NodeHasSufficientMemory" Sep 18 03:33:07.798386 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.798342 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-80.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 03:33:07.798386 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.798352 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-80.ec2.internal" event="NodeHasSufficientPID" Sep 18 03:33:07.798807 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.798792 2568 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 18 03:33:07.798807 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.798804 2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 18 03:33:07.798949 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.798812 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 03:33:07.798949 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.798822 2568 state_mem.go:36] "Initialized new in-memory state store" Sep 18 03:33:07.799585 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.799570 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-80.ec2.internal" event="NodeHasSufficientMemory" Sep 18 03:33:07.799666 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.799599 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-80.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 03:33:07.799666 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.799613 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-80.ec2.internal" event="NodeHasSufficientPID" Sep 18 03:33:07.799919 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.799907 2568 policy_none.go:49] "None policy: Start" Sep 18 03:33:07.799984 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.799922 2568 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 18 03:33:07.799984 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.799971 2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 18 03:33:07.800987 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.800973 2568 policy_none.go:47] "Start" Sep 18 03:33:07.831587 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.831568 2568 manager.go:337] "Starting Device Plugin manager" Sep 18 03:33:07.840912 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:07.831610 2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 18 03:33:07.840912 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.831624 2568 server.go:87] "Starting device plugin registration server" Sep 18 03:33:07.840912 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.831855 2568 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 18 03:33:07.840912 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.831875 2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 18 03:33:07.840912 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.831969 2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 18 03:33:07.840912 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.832042 2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 18 03:33:07.840912 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.832050 2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 18 03:33:07.840912 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:07.832979 2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 18 03:33:07.840912 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:07.833017 2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-80.ec2.internal\" not found" Sep 18 03:33:07.874642 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.874612 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 18 03:33:07.875714 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.875699 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 18 03:33:07.875799 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.875740 2568 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 18 03:33:07.875799 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.875758 2568 kubelet.go:2448] "Starting kubelet main sync loop" Sep 18 03:33:07.875799 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:07.875793 2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 18 03:33:07.877649 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.877631 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 03:33:07.932322 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.932298 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 03:33:07.933157 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.933142 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-80.ec2.internal" event="NodeHasSufficientMemory" Sep 18 03:33:07.933238 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.933164 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-80.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 03:33:07.933238 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.933173 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-80.ec2.internal" event="NodeHasSufficientPID" Sep 18 03:33:07.933238 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.933194 2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-80.ec2.internal" Sep 18 03:33:07.939780 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.939765 2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-80.ec2.internal" Sep 18 03:33:07.939859 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:07.939785 2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-80.ec2.internal\": node \"ip-10-0-134-80.ec2.internal\" not found" Sep 18 03:33:07.954187 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:07.954133 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-80.ec2.internal\" not found" Sep 18 03:33:07.976336 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.976318 2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-80.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-80.ec2.internal"] Sep 18 03:33:07.976415 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.976388 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 03:33:07.977093 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.977080 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-80.ec2.internal" event="NodeHasSufficientMemory" Sep 18 03:33:07.977161 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.977105 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-80.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 03:33:07.977161 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.977120 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-80.ec2.internal" event="NodeHasSufficientPID" Sep 18 03:33:07.978306 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.978294 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 03:33:07.978441 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.978424 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-80.ec2.internal" Sep 18 03:33:07.978486 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.978467 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 03:33:07.978977 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.978960 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-80.ec2.internal" event="NodeHasSufficientMemory" Sep 18 03:33:07.979060 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.978962 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-80.ec2.internal" event="NodeHasSufficientMemory" Sep 18 03:33:07.979060 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.978989 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-80.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 03:33:07.979060 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.979023 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-80.ec2.internal" event="NodeHasSufficientPID" Sep 18 03:33:07.979060 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.979041 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-80.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 03:33:07.979060 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.979056 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-80.ec2.internal" event="NodeHasSufficientPID" Sep 18 03:33:07.980213 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.980198 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-80.ec2.internal" Sep 18 03:33:07.980279 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.980232 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 03:33:07.980872 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.980856 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-80.ec2.internal" event="NodeHasSufficientMemory" Sep 18 03:33:07.980967 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.980888 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-80.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 03:33:07.980967 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:07.980903 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-80.ec2.internal" event="NodeHasSufficientPID" Sep 18 03:33:08.005431 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:08.005414 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-80.ec2.internal\" not found" node="ip-10-0-134-80.ec2.internal" Sep 18 03:33:08.009497 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:08.009483 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-80.ec2.internal\" not found" node="ip-10-0-134-80.ec2.internal" Sep 18 03:33:08.055072 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:08.055052 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-80.ec2.internal\" not found" Sep 18 03:33:08.074198 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:08.074177 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c05fdffdb3fc054c2046976206becf4f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-80.ec2.internal\" (UID: \"c05fdffdb3fc054c2046976206becf4f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-80.ec2.internal" Sep 18 03:33:08.074268 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:08.074206 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c05fdffdb3fc054c2046976206becf4f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-80.ec2.internal\" (UID: \"c05fdffdb3fc054c2046976206becf4f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-80.ec2.internal" Sep 18 03:33:08.074268 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:08.074228 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/20ab69c906b5fa2b928622de52fe5786-config\") pod \"kube-apiserver-proxy-ip-10-0-134-80.ec2.internal\" (UID: \"20ab69c906b5fa2b928622de52fe5786\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-80.ec2.internal" Sep 18 03:33:08.155279 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:08.155251 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-80.ec2.internal\" not found" Sep 18 03:33:08.174624 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:08.174596 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c05fdffdb3fc054c2046976206becf4f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-80.ec2.internal\" (UID: \"c05fdffdb3fc054c2046976206becf4f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-80.ec2.internal" Sep 18 03:33:08.174698 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:08.174629 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c05fdffdb3fc054c2046976206becf4f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-80.ec2.internal\" (UID: \"c05fdffdb3fc054c2046976206becf4f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-80.ec2.internal" Sep 18 03:33:08.174698 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:08.174647 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/20ab69c906b5fa2b928622de52fe5786-config\") pod \"kube-apiserver-proxy-ip-10-0-134-80.ec2.internal\" (UID: \"20ab69c906b5fa2b928622de52fe5786\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-80.ec2.internal" Sep 18 03:33:08.174698 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:08.174681 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c05fdffdb3fc054c2046976206becf4f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-80.ec2.internal\" (UID: \"c05fdffdb3fc054c2046976206becf4f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-80.ec2.internal" Sep 18 03:33:08.174811 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:08.174706 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c05fdffdb3fc054c2046976206becf4f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-80.ec2.internal\" (UID: \"c05fdffdb3fc054c2046976206becf4f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-80.ec2.internal" Sep 18 03:33:08.174811 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:08.174688 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/20ab69c906b5fa2b928622de52fe5786-config\") pod \"kube-apiserver-proxy-ip-10-0-134-80.ec2.internal\" (UID: \"20ab69c906b5fa2b928622de52fe5786\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-80.ec2.internal" Sep 18 03:33:08.256047 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:08.255967 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-80.ec2.internal\" not found" Sep 18 03:33:08.309442 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:08.309408 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-80.ec2.internal" Sep 18 03:33:08.314091 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:08.313951 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-80.ec2.internal" Sep 18 03:33:08.356609 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:08.356579 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-80.ec2.internal\" not found" Sep 18 03:33:08.457084 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:08.457047 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-80.ec2.internal\" not found" Sep 18 03:33:08.557611 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:08.557543 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-80.ec2.internal\" not found" Sep 18 03:33:08.658081 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:08.658042 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-80.ec2.internal\" not found" Sep 18 03:33:08.712511 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:08.712489 2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 18 03:33:08.712887 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:08.712637 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 18 03:33:08.758174 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:08.758140 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-80.ec2.internal\" not found" Sep 18 03:33:08.795376 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:08.795320 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-17 03:28:07 +0000 UTC" deadline="2028-03-07 08:19:21.742280484 +0000 UTC" Sep 18 03:33:08.795376 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:08.795368 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12868h46m12.946916326s" Sep 18 03:33:08.835718 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:08.835661 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20ab69c906b5fa2b928622de52fe5786.slice/crio-3f86eeef6d3a139352214b6fa85a74fd73d85fa0e1e5fbc913f8d8c4dee19a9c WatchSource:0}: Error finding container 3f86eeef6d3a139352214b6fa85a74fd73d85fa0e1e5fbc913f8d8c4dee19a9c: Status 404 returned error can't find the container with id 3f86eeef6d3a139352214b6fa85a74fd73d85fa0e1e5fbc913f8d8c4dee19a9c Sep 18 03:33:08.836109 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:08.836085 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc05fdffdb3fc054c2046976206becf4f.slice/crio-826ab96cbfc939308379b4ff0196e58e152642abf23f0e36bce98700c78c0290 WatchSource:0}: Error finding container 826ab96cbfc939308379b4ff0196e58e152642abf23f0e36bce98700c78c0290: Status 404 returned error can't find the container with id 826ab96cbfc939308379b4ff0196e58e152642abf23f0e36bce98700c78c0290 Sep 18 03:33:08.840822 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:08.840799 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 18 03:33:08.858798 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:08.858774 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-80.ec2.internal\" not found" Sep 18 03:33:08.879246 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:08.879210 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-80.ec2.internal" event={"ID":"20ab69c906b5fa2b928622de52fe5786","Type":"ContainerStarted","Data":"3f86eeef6d3a139352214b6fa85a74fd73d85fa0e1e5fbc913f8d8c4dee19a9c"} Sep 18 03:33:08.880269 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:08.880245 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-80.ec2.internal" event={"ID":"c05fdffdb3fc054c2046976206becf4f","Type":"ContainerStarted","Data":"826ab96cbfc939308379b4ff0196e58e152642abf23f0e36bce98700c78c0290"} Sep 18 03:33:08.959499 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:08.959472 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-80.ec2.internal\" not found" Sep 18 03:33:09.059985 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:09.059961 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-80.ec2.internal\" not found" Sep 18 03:33:09.160509 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:09.160425 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-80.ec2.internal\" not found" Sep 18 03:33:09.215660 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.215631 2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 03:33:09.272538 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.272517 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-80.ec2.internal" Sep 18 03:33:09.277507 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.277489 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 03:33:09.283207 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.283187 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 18 03:33:09.284773 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.284760 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-80.ec2.internal" Sep 18 03:33:09.294853 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.294833 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 18 03:33:09.332564 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.332543 2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 03:33:09.658060 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.657969 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 03:33:09.744832 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.744800 2568 apiserver.go:52] "Watching apiserver" Sep 18 03:33:09.752509 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.752470 2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 18 03:33:09.752849 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.752821 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-vsdh4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-80.ec2.internal","openshift-multus/multus-tlvwc","openshift-multus/network-metrics-daemon-tf6sq","openshift-network-operator/iptables-alerter-ct74c","openshift-ovn-kubernetes/ovnkube-node-psksv","kube-system/global-pull-secret-syncer-hv4gk","kube-system/konnectivity-agent-hs82j","kube-system/kube-apiserver-proxy-ip-10-0-134-80.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv","openshift-dns/node-resolver-87l8w","openshift-image-registry/node-ca-54ct2","openshift-multus/multus-additional-cni-plugins-fhzm9","openshift-network-diagnostics/network-check-target-jbxqp"] Sep 18 03:33:09.754367 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.754346 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.755422 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.755401 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:33:09.755543 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:09.755476 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbxqp" podUID="4437fdab-4af1-45cb-b944-acc7cbff5d20" Sep 18 03:33:09.756656 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.756464 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 18 03:33:09.756656 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.756479 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 18 03:33:09.756656 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.756530 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mzzbs\"" Sep 18 03:33:09.756847 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.756716 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.757835 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.757813 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fhzm9" Sep 18 03:33:09.758551 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.758531 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 18 03:33:09.759245 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.758803 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 18 03:33:09.759245 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.759019 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-87l8w" Sep 18 03:33:09.759245 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.759028 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 18 03:33:09.759421 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.759272 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pgk9b\"" Sep 18 03:33:09.759476 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.759419 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 18 03:33:09.759859 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.759842 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 18 03:33:09.759925 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.759862 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 18 03:33:09.759925 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.759913 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dbck9\"" Sep 18 03:33:09.760528 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.760472 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.761723 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.761592 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 18 03:33:09.761723 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.761728 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:09.761898 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:09.761809 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hv4gk" podUID="34b777d9-80fc-4885-a29b-0b1f1e7d7415" Sep 18 03:33:09.762318 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.762113 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 18 03:33:09.762318 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.762162 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5k8vl\"" Sep 18 03:33:09.763134 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.763110 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 18 03:33:09.763213 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.763134 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 18 03:33:09.763729 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.763420 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hs82j" Sep 18 03:33:09.763729 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.763427 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 18 03:33:09.763729 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.763458 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 18 03:33:09.763729 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.763494 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-t2s2j\"" Sep 18 03:33:09.763729 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.763644 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 18 03:33:09.764033 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.763733 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 18 03:33:09.764619 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.764600 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" Sep 18 03:33:09.765461 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.765443 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 18 03:33:09.765615 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.765598 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 18 03:33:09.765852 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.765823 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7bmjs\"" Sep 18 03:33:09.766259 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.766180 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ct74c" Sep 18 03:33:09.766746 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.766593 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 18 03:33:09.766746 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.766637 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 18 03:33:09.766746 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.766719 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 18 03:33:09.766995 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.766784 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ct42r\"" Sep 18 03:33:09.767569 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.767551 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-54ct2" Sep 18 03:33:09.768146 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.768130 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 18 03:33:09.768457 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.768427 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-p7tsm\"" Sep 18 03:33:09.768545 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.768500 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 18 03:33:09.768545 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.768535 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 18 03:33:09.769041 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.769023 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:09.769143 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:09.769116 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tf6sq" podUID="9fdcb7aa-5f4c-4d16-a250-b272513204da" Sep 18 03:33:09.769488 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.769475 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 18 03:33:09.769712 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.769695 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zwbdd\"" Sep 18 03:33:09.769807 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.769719 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 18 03:33:09.769807 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.769794 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 18 03:33:09.773210 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.773191 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 18 03:33:09.773306 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.773285 2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 18 03:33:09.783439 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.783415 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx9nm\" (UniqueName: \"kubernetes.io/projected/1013fa23-5170-4d1f-8722-86bc4b8b8742-kube-api-access-bx9nm\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.783513 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.783457 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4eb89543-6d24-4d8c-aa8b-d55e3a078d21-socket-dir\") pod \"aws-ebs-csi-driver-node-d6cdv\" (UID: \"4eb89543-6d24-4d8c-aa8b-d55e3a078d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" Sep 18 03:33:09.783513 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.783484 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4eb89543-6d24-4d8c-aa8b-d55e3a078d21-registration-dir\") pod \"aws-ebs-csi-driver-node-d6cdv\" (UID: \"4eb89543-6d24-4d8c-aa8b-d55e3a078d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" Sep 18 03:33:09.783584 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.783516 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-var-lib-kubelet\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.783584 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.783539 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-cnibin\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.783584 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.783553 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-hostroot\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.783670 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.783567 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nks5f\" (UniqueName: \"kubernetes.io/projected/afa7f816-ccc9-406c-bd67-55e4d933d144-kube-api-access-nks5f\") pod \"multus-additional-cni-plugins-fhzm9\" (UID: \"afa7f816-ccc9-406c-bd67-55e4d933d144\") " pod="openshift-multus/multus-additional-cni-plugins-fhzm9" Sep 18 03:33:09.783670 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.783622 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-host-run-netns\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.783670 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.783642 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-run-systemd\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.783670 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.783655 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-system-cni-dir\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.783839 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.783670 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-multus-socket-dir-parent\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.783839 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.783683 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-multus-conf-dir\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.783839 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.783703 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs\") pod \"network-metrics-daemon-tf6sq\" (UID: \"9fdcb7aa-5f4c-4d16-a250-b272513204da\") " pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:09.783839 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.783771 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl2lq\" (UniqueName: \"kubernetes.io/projected/80a2d237-3e90-4289-8d91-46a883cb2001-kube-api-access-hl2lq\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.783839 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.783805 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-host-kubelet\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.783839 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.783832 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-host-run-ovn-kubernetes\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.784178 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.783859 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d9a7ff5b-3ca7-43f4-907d-3eff238968f1-konnectivity-ca\") pod \"konnectivity-agent-hs82j\" (UID: \"d9a7ff5b-3ca7-43f4-907d-3eff238968f1\") " pod="kube-system/konnectivity-agent-hs82j" Sep 18 03:33:09.784178 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.783883 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4eb89543-6d24-4d8c-aa8b-d55e3a078d21-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d6cdv\" (UID: \"4eb89543-6d24-4d8c-aa8b-d55e3a078d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" Sep 18 03:33:09.784178 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.783906 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8khqd\" (UniqueName: \"kubernetes.io/projected/4437fdab-4af1-45cb-b944-acc7cbff5d20-kube-api-access-8khqd\") pod \"network-check-target-jbxqp\" (UID: \"4437fdab-4af1-45cb-b944-acc7cbff5d20\") " pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:33:09.784178 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.783945 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-log-socket\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.784178 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.783968 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w6kc\" (UniqueName: \"kubernetes.io/projected/fcf69b36-5d73-458d-9324-6da53a9f1c73-kube-api-access-5w6kc\") pod \"node-resolver-87l8w\" (UID: \"fcf69b36-5d73-458d-9324-6da53a9f1c73\") " pod="openshift-dns/node-resolver-87l8w" Sep 18 03:33:09.784178 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.783995 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/34b777d9-80fc-4885-a29b-0b1f1e7d7415-kubelet-config\") pod \"global-pull-secret-syncer-hv4gk\" (UID: \"34b777d9-80fc-4885-a29b-0b1f1e7d7415\") " pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:09.784178 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784015 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/34b777d9-80fc-4885-a29b-0b1f1e7d7415-dbus\") pod \"global-pull-secret-syncer-hv4gk\" (UID: \"34b777d9-80fc-4885-a29b-0b1f1e7d7415\") " pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:09.784178 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784038 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-etc-sysctl-conf\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.784178 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784060 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1013fa23-5170-4d1f-8722-86bc4b8b8742-ovn-node-metrics-cert\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.784178 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784082 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4eb89543-6d24-4d8c-aa8b-d55e3a078d21-sys-fs\") pod \"aws-ebs-csi-driver-node-d6cdv\" (UID: \"4eb89543-6d24-4d8c-aa8b-d55e3a078d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" Sep 18 03:33:09.784178 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784102 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-etc-systemd\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.784178 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784123 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-node-log\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.784178 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784162 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-host-var-lib-kubelet\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.784739 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784202 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/afa7f816-ccc9-406c-bd67-55e4d933d144-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fhzm9\" (UID: \"afa7f816-ccc9-406c-bd67-55e4d933d144\") " pod="openshift-multus/multus-additional-cni-plugins-fhzm9" Sep 18 03:33:09.784739 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784229 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcf69b36-5d73-458d-9324-6da53a9f1c73-tmp-dir\") pod \"node-resolver-87l8w\" (UID: \"fcf69b36-5d73-458d-9324-6da53a9f1c73\") " pod="openshift-dns/node-resolver-87l8w" Sep 18 03:33:09.784739 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784254 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-etc-kubernetes\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.784739 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784277 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-run\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.784739 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784302 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-host\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.784739 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784323 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-multus-cni-dir\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.784739 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784346 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/afa7f816-ccc9-406c-bd67-55e4d933d144-system-cni-dir\") pod \"multus-additional-cni-plugins-fhzm9\" (UID: \"afa7f816-ccc9-406c-bd67-55e4d933d144\") " pod="openshift-multus/multus-additional-cni-plugins-fhzm9" Sep 18 03:33:09.784739 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784372 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-etc-modprobe-d\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.784739 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784394 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4eb89543-6d24-4d8c-aa8b-d55e3a078d21-etc-selinux\") pod \"aws-ebs-csi-driver-node-d6cdv\" (UID: \"4eb89543-6d24-4d8c-aa8b-d55e3a078d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" Sep 18 03:33:09.784739 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784416 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-multus-daemon-config\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.784739 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784438 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-host-run-multus-certs\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.784739 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784461 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fcf69b36-5d73-458d-9324-6da53a9f1c73-hosts-file\") pod \"node-resolver-87l8w\" (UID: \"fcf69b36-5d73-458d-9324-6da53a9f1c73\") " pod="openshift-dns/node-resolver-87l8w" Sep 18 03:33:09.784739 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784497 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s96j\" (UniqueName: \"kubernetes.io/projected/9fdcb7aa-5f4c-4d16-a250-b272513204da-kube-api-access-2s96j\") pod \"network-metrics-daemon-tf6sq\" (UID: \"9fdcb7aa-5f4c-4d16-a250-b272513204da\") " pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:09.784739 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784547 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-var-lib-openvswitch\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.784739 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784575 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1013fa23-5170-4d1f-8722-86bc4b8b8742-env-overrides\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.784739 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784617 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-host-var-lib-cni-multus\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.785414 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784654 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-host-cni-netd\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.785414 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784678 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d9a7ff5b-3ca7-43f4-907d-3eff238968f1-agent-certs\") pod \"konnectivity-agent-hs82j\" (UID: \"d9a7ff5b-3ca7-43f4-907d-3eff238968f1\") " pod="kube-system/konnectivity-agent-hs82j" Sep 18 03:33:09.785414 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784697 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8zzf\" (UniqueName: \"kubernetes.io/projected/d5694211-42dd-4f1c-984b-fc2a748e8759-kube-api-access-c8zzf\") pod \"node-ca-54ct2\" (UID: \"d5694211-42dd-4f1c-984b-fc2a748e8759\") " pod="openshift-image-registry/node-ca-54ct2" Sep 18 03:33:09.785414 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784724 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-host-run-netns\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.785414 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784748 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-host-var-lib-cni-bin\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.785414 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784771 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/afa7f816-ccc9-406c-bd67-55e4d933d144-cni-binary-copy\") pod \"multus-additional-cni-plugins-fhzm9\" (UID: \"afa7f816-ccc9-406c-bd67-55e4d933d144\") " pod="openshift-multus/multus-additional-cni-plugins-fhzm9" Sep 18 03:33:09.785414 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784796 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/afa7f816-ccc9-406c-bd67-55e4d933d144-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fhzm9\" (UID: \"afa7f816-ccc9-406c-bd67-55e4d933d144\") " pod="openshift-multus/multus-additional-cni-plugins-fhzm9" Sep 18 03:33:09.785414 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784820 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/34b777d9-80fc-4885-a29b-0b1f1e7d7415-original-pull-secret\") pod \"global-pull-secret-syncer-hv4gk\" (UID: \"34b777d9-80fc-4885-a29b-0b1f1e7d7415\") " pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:09.785414 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784858 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v5pj\" (UniqueName: \"kubernetes.io/projected/4eb89543-6d24-4d8c-aa8b-d55e3a078d21-kube-api-access-8v5pj\") pod \"aws-ebs-csi-driver-node-d6cdv\" (UID: \"4eb89543-6d24-4d8c-aa8b-d55e3a078d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" Sep 18 03:33:09.785414 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784899 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d5694211-42dd-4f1c-984b-fc2a748e8759-serviceca\") pod \"node-ca-54ct2\" (UID: \"d5694211-42dd-4f1c-984b-fc2a748e8759\") " pod="openshift-image-registry/node-ca-54ct2" Sep 18 03:33:09.785414 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784948 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-cni-binary-copy\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.785414 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784973 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/afa7f816-ccc9-406c-bd67-55e4d933d144-cnibin\") pod \"multus-additional-cni-plugins-fhzm9\" (UID: \"afa7f816-ccc9-406c-bd67-55e4d933d144\") " pod="openshift-multus/multus-additional-cni-plugins-fhzm9" Sep 18 03:33:09.785414 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.784998 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f243ab41-9b0c-47d6-ab6f-7b300a516bb7-host-slash\") pod \"iptables-alerter-ct74c\" (UID: \"f243ab41-9b0c-47d6-ab6f-7b300a516bb7\") " pod="openshift-network-operator/iptables-alerter-ct74c" Sep 18 03:33:09.785414 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.785018 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-lib-modules\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.785414 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.785039 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-host-slash\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.785414 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.785061 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-etc-openvswitch\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.786096 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.785078 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-host-cni-bin\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.786096 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.785093 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/afa7f816-ccc9-406c-bd67-55e4d933d144-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fhzm9\" (UID: \"afa7f816-ccc9-406c-bd67-55e4d933d144\") " pod="openshift-multus/multus-additional-cni-plugins-fhzm9" Sep 18 03:33:09.786096 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.785116 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.786096 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.785141 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-host-run-k8s-cni-cncf-io\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.786096 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.785164 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5n77\" (UniqueName: \"kubernetes.io/projected/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-kube-api-access-c5n77\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.786096 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.785186 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-etc-sysconfig\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.786096 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.785227 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-etc-sysctl-d\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.786096 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.785255 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/80a2d237-3e90-4289-8d91-46a883cb2001-tmp\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.786096 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.785279 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-systemd-units\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.786096 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.785301 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-run-openvswitch\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.786096 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.785323 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-run-ovn\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.786096 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.785363 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1013fa23-5170-4d1f-8722-86bc4b8b8742-ovnkube-script-lib\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.786096 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.785394 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d5694211-42dd-4f1c-984b-fc2a748e8759-host\") pod \"node-ca-54ct2\" (UID: \"d5694211-42dd-4f1c-984b-fc2a748e8759\") " pod="openshift-image-registry/node-ca-54ct2" Sep 18 03:33:09.786096 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.785423 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1013fa23-5170-4d1f-8722-86bc4b8b8742-ovnkube-config\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.786096 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.785444 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4eb89543-6d24-4d8c-aa8b-d55e3a078d21-device-dir\") pod \"aws-ebs-csi-driver-node-d6cdv\" (UID: \"4eb89543-6d24-4d8c-aa8b-d55e3a078d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" Sep 18 03:33:09.786096 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.785458 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-etc-kubernetes\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.786534 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.785472 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f243ab41-9b0c-47d6-ab6f-7b300a516bb7-iptables-alerter-script\") pod \"iptables-alerter-ct74c\" (UID: \"f243ab41-9b0c-47d6-ab6f-7b300a516bb7\") " pod="openshift-network-operator/iptables-alerter-ct74c" Sep 18 03:33:09.786534 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.785495 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phxrz\" (UniqueName: \"kubernetes.io/projected/f243ab41-9b0c-47d6-ab6f-7b300a516bb7-kube-api-access-phxrz\") pod \"iptables-alerter-ct74c\" (UID: \"f243ab41-9b0c-47d6-ab6f-7b300a516bb7\") " pod="openshift-network-operator/iptables-alerter-ct74c" Sep 18 03:33:09.786534 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.785515 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-sys\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.786534 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.785528 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/80a2d237-3e90-4289-8d91-46a883cb2001-etc-tuned\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.787061 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.787047 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 18 03:33:09.802974 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.802960 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qtvmf" Sep 18 03:33:09.810359 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.810343 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qtvmf" Sep 18 03:33:09.886404 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.886374 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8zzf\" (UniqueName: \"kubernetes.io/projected/d5694211-42dd-4f1c-984b-fc2a748e8759-kube-api-access-c8zzf\") pod \"node-ca-54ct2\" (UID: \"d5694211-42dd-4f1c-984b-fc2a748e8759\") " pod="openshift-image-registry/node-ca-54ct2" Sep 18 03:33:09.886552 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.886415 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-host-run-netns\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.886552 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.886443 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-host-var-lib-cni-bin\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.886552 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.886468 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/afa7f816-ccc9-406c-bd67-55e4d933d144-cni-binary-copy\") pod \"multus-additional-cni-plugins-fhzm9\" (UID: \"afa7f816-ccc9-406c-bd67-55e4d933d144\") " pod="openshift-multus/multus-additional-cni-plugins-fhzm9" Sep 18 03:33:09.886552 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.886512 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/afa7f816-ccc9-406c-bd67-55e4d933d144-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fhzm9\" (UID: \"afa7f816-ccc9-406c-bd67-55e4d933d144\") " pod="openshift-multus/multus-additional-cni-plugins-fhzm9" Sep 18 03:33:09.886552 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.886541 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/34b777d9-80fc-4885-a29b-0b1f1e7d7415-original-pull-secret\") pod \"global-pull-secret-syncer-hv4gk\" (UID: \"34b777d9-80fc-4885-a29b-0b1f1e7d7415\") " pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:09.886867 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.886549 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-host-var-lib-cni-bin\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.886867 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.886566 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v5pj\" (UniqueName: \"kubernetes.io/projected/4eb89543-6d24-4d8c-aa8b-d55e3a078d21-kube-api-access-8v5pj\") pod \"aws-ebs-csi-driver-node-d6cdv\" (UID: \"4eb89543-6d24-4d8c-aa8b-d55e3a078d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" Sep 18 03:33:09.887227 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.886579 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-host-run-netns\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.887297 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887238 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d5694211-42dd-4f1c-984b-fc2a748e8759-serviceca\") pod \"node-ca-54ct2\" (UID: \"d5694211-42dd-4f1c-984b-fc2a748e8759\") " pod="openshift-image-registry/node-ca-54ct2" Sep 18 03:33:09.887297 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:09.887276 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 03:33:09.887297 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887288 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-cni-binary-copy\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.887440 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887325 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/afa7f816-ccc9-406c-bd67-55e4d933d144-cnibin\") pod \"multus-additional-cni-plugins-fhzm9\" (UID: \"afa7f816-ccc9-406c-bd67-55e4d933d144\") " pod="openshift-multus/multus-additional-cni-plugins-fhzm9" Sep 18 03:33:09.887440 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:09.887356 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34b777d9-80fc-4885-a29b-0b1f1e7d7415-original-pull-secret podName:34b777d9-80fc-4885-a29b-0b1f1e7d7415 nodeName:}" failed. No retries permitted until 2026-09-18 03:33:10.38732 +0000 UTC m=+4.670081843 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/34b777d9-80fc-4885-a29b-0b1f1e7d7415-original-pull-secret") pod "global-pull-secret-syncer-hv4gk" (UID: "34b777d9-80fc-4885-a29b-0b1f1e7d7415") : object "kube-system"/"original-pull-secret" not registered Sep 18 03:33:09.887440 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887376 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/afa7f816-ccc9-406c-bd67-55e4d933d144-cnibin\") pod \"multus-additional-cni-plugins-fhzm9\" (UID: \"afa7f816-ccc9-406c-bd67-55e4d933d144\") " pod="openshift-multus/multus-additional-cni-plugins-fhzm9" Sep 18 03:33:09.887440 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887390 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f243ab41-9b0c-47d6-ab6f-7b300a516bb7-host-slash\") pod \"iptables-alerter-ct74c\" (UID: \"f243ab41-9b0c-47d6-ab6f-7b300a516bb7\") " pod="openshift-network-operator/iptables-alerter-ct74c" Sep 18 03:33:09.887440 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887416 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-lib-modules\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.887440 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887427 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d5694211-42dd-4f1c-984b-fc2a748e8759-serviceca\") pod \"node-ca-54ct2\" (UID: \"d5694211-42dd-4f1c-984b-fc2a748e8759\") " pod="openshift-image-registry/node-ca-54ct2" Sep 18 03:33:09.887440 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887438 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-host-slash\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.887796 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887461 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-etc-openvswitch\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.887796 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887485 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-host-cni-bin\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.887796 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887508 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/afa7f816-ccc9-406c-bd67-55e4d933d144-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fhzm9\" (UID: \"afa7f816-ccc9-406c-bd67-55e4d933d144\") " pod="openshift-multus/multus-additional-cni-plugins-fhzm9" Sep 18 03:33:09.887796 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887532 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.887796 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887541 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/afa7f816-ccc9-406c-bd67-55e4d933d144-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fhzm9\" (UID: \"afa7f816-ccc9-406c-bd67-55e4d933d144\") " pod="openshift-multus/multus-additional-cni-plugins-fhzm9" Sep 18 03:33:09.887796 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887557 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-host-run-k8s-cni-cncf-io\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.887796 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887590 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-lib-modules\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.887796 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887596 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5n77\" (UniqueName: \"kubernetes.io/projected/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-kube-api-access-c5n77\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.887796 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887602 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-host-run-k8s-cni-cncf-io\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.887796 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887636 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-etc-sysconfig\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.887796 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887659 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-host-slash\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.887796 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887663 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-etc-sysctl-d\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.887796 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887694 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/80a2d237-3e90-4289-8d91-46a883cb2001-tmp\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.887796 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887717 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-systemd-units\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.887796 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887740 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-run-openvswitch\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.887796 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887764 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-run-ovn\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.887796 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887790 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1013fa23-5170-4d1f-8722-86bc4b8b8742-ovnkube-script-lib\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.888618 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887811 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d5694211-42dd-4f1c-984b-fc2a748e8759-host\") pod \"node-ca-54ct2\" (UID: \"d5694211-42dd-4f1c-984b-fc2a748e8759\") " pod="openshift-image-registry/node-ca-54ct2" Sep 18 03:33:09.888618 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887833 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1013fa23-5170-4d1f-8722-86bc4b8b8742-ovnkube-config\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.888618 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887839 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-etc-sysconfig\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.888618 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887856 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4eb89543-6d24-4d8c-aa8b-d55e3a078d21-device-dir\") pod \"aws-ebs-csi-driver-node-d6cdv\" (UID: \"4eb89543-6d24-4d8c-aa8b-d55e3a078d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" Sep 18 03:33:09.888618 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887878 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-etc-kubernetes\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.888618 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887886 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f243ab41-9b0c-47d6-ab6f-7b300a516bb7-host-slash\") pod \"iptables-alerter-ct74c\" (UID: \"f243ab41-9b0c-47d6-ab6f-7b300a516bb7\") " pod="openshift-network-operator/iptables-alerter-ct74c" Sep 18 03:33:09.888618 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887886 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-cni-binary-copy\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.888618 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887902 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f243ab41-9b0c-47d6-ab6f-7b300a516bb7-iptables-alerter-script\") pod \"iptables-alerter-ct74c\" (UID: \"f243ab41-9b0c-47d6-ab6f-7b300a516bb7\") " pod="openshift-network-operator/iptables-alerter-ct74c" Sep 18 03:33:09.888618 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887959 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-run-openvswitch\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.888618 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887983 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d5694211-42dd-4f1c-984b-fc2a748e8759-host\") pod \"node-ca-54ct2\" (UID: \"d5694211-42dd-4f1c-984b-fc2a748e8759\") " pod="openshift-image-registry/node-ca-54ct2" Sep 18 03:33:09.888618 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.888007 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-etc-openvswitch\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.888618 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.888045 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-host-cni-bin\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.888618 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.888180 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/afa7f816-ccc9-406c-bd67-55e4d933d144-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fhzm9\" (UID: \"afa7f816-ccc9-406c-bd67-55e4d933d144\") " pod="openshift-multus/multus-additional-cni-plugins-fhzm9" Sep 18 03:33:09.888618 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.888221 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.888618 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.888246 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/afa7f816-ccc9-406c-bd67-55e4d933d144-cni-binary-copy\") pod \"multus-additional-cni-plugins-fhzm9\" (UID: \"afa7f816-ccc9-406c-bd67-55e4d933d144\") " pod="openshift-multus/multus-additional-cni-plugins-fhzm9" Sep 18 03:33:09.888618 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.888314 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4eb89543-6d24-4d8c-aa8b-d55e3a078d21-device-dir\") pod \"aws-ebs-csi-driver-node-d6cdv\" (UID: \"4eb89543-6d24-4d8c-aa8b-d55e3a078d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" Sep 18 03:33:09.888618 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.887766 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-etc-sysctl-d\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.889445 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.888370 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-run-ovn\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.889445 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.888485 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phxrz\" (UniqueName: \"kubernetes.io/projected/f243ab41-9b0c-47d6-ab6f-7b300a516bb7-kube-api-access-phxrz\") pod \"iptables-alerter-ct74c\" (UID: \"f243ab41-9b0c-47d6-ab6f-7b300a516bb7\") " pod="openshift-network-operator/iptables-alerter-ct74c" Sep 18 03:33:09.889445 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.888493 2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 18 03:33:09.889445 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.888513 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f243ab41-9b0c-47d6-ab6f-7b300a516bb7-iptables-alerter-script\") pod \"iptables-alerter-ct74c\" (UID: \"f243ab41-9b0c-47d6-ab6f-7b300a516bb7\") " pod="openshift-network-operator/iptables-alerter-ct74c" Sep 18 03:33:09.889445 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.888518 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1013fa23-5170-4d1f-8722-86bc4b8b8742-ovnkube-config\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.889445 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.888525 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-sys\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.889445 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.888557 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/80a2d237-3e90-4289-8d91-46a883cb2001-etc-tuned\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.889445 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.888574 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-systemd-units\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.889445 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.888579 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-etc-kubernetes\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.889445 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.888583 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx9nm\" (UniqueName: \"kubernetes.io/projected/1013fa23-5170-4d1f-8722-86bc4b8b8742-kube-api-access-bx9nm\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.889445 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.888607 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4eb89543-6d24-4d8c-aa8b-d55e3a078d21-socket-dir\") pod \"aws-ebs-csi-driver-node-d6cdv\" (UID: \"4eb89543-6d24-4d8c-aa8b-d55e3a078d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" Sep 18 03:33:09.889445 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.888632 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4eb89543-6d24-4d8c-aa8b-d55e3a078d21-registration-dir\") pod \"aws-ebs-csi-driver-node-d6cdv\" (UID: \"4eb89543-6d24-4d8c-aa8b-d55e3a078d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" Sep 18 03:33:09.889445 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.888654 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-var-lib-kubelet\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.889445 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.888677 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-cnibin\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.889445 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.888698 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-hostroot\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.889445 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.888836 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-sys\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.889445 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.888845 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1013fa23-5170-4d1f-8722-86bc4b8b8742-ovnkube-script-lib\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.889445 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.888884 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4eb89543-6d24-4d8c-aa8b-d55e3a078d21-registration-dir\") pod \"aws-ebs-csi-driver-node-d6cdv\" (UID: \"4eb89543-6d24-4d8c-aa8b-d55e3a078d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" Sep 18 03:33:09.890352 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.888927 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-cnibin\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.890352 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889007 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-var-lib-kubelet\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.890352 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889013 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-hostroot\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.890352 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889020 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4eb89543-6d24-4d8c-aa8b-d55e3a078d21-socket-dir\") pod \"aws-ebs-csi-driver-node-d6cdv\" (UID: \"4eb89543-6d24-4d8c-aa8b-d55e3a078d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" Sep 18 03:33:09.890352 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.888720 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nks5f\" (UniqueName: \"kubernetes.io/projected/afa7f816-ccc9-406c-bd67-55e4d933d144-kube-api-access-nks5f\") pod \"multus-additional-cni-plugins-fhzm9\" (UID: \"afa7f816-ccc9-406c-bd67-55e4d933d144\") " pod="openshift-multus/multus-additional-cni-plugins-fhzm9" Sep 18 03:33:09.890352 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889239 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-host-run-netns\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.890352 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889263 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-run-systemd\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.890352 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889276 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-host-run-netns\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.890352 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889286 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-system-cni-dir\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.890352 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889331 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-run-systemd\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.890352 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889338 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-system-cni-dir\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.890352 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889351 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-multus-socket-dir-parent\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.890352 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889381 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-multus-conf-dir\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.890352 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889405 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs\") pod \"network-metrics-daemon-tf6sq\" (UID: \"9fdcb7aa-5f4c-4d16-a250-b272513204da\") " pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:09.890352 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889430 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl2lq\" (UniqueName: \"kubernetes.io/projected/80a2d237-3e90-4289-8d91-46a883cb2001-kube-api-access-hl2lq\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.890352 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889452 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-host-kubelet\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.890352 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889476 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-host-run-ovn-kubernetes\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.890352 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889500 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d9a7ff5b-3ca7-43f4-907d-3eff238968f1-konnectivity-ca\") pod \"konnectivity-agent-hs82j\" (UID: \"d9a7ff5b-3ca7-43f4-907d-3eff238968f1\") " pod="kube-system/konnectivity-agent-hs82j" Sep 18 03:33:09.891242 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889518 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-multus-socket-dir-parent\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.891242 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889546 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4eb89543-6d24-4d8c-aa8b-d55e3a078d21-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d6cdv\" (UID: \"4eb89543-6d24-4d8c-aa8b-d55e3a078d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" Sep 18 03:33:09.891242 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889571 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8khqd\" (UniqueName: \"kubernetes.io/projected/4437fdab-4af1-45cb-b944-acc7cbff5d20-kube-api-access-8khqd\") pod \"network-check-target-jbxqp\" (UID: \"4437fdab-4af1-45cb-b944-acc7cbff5d20\") " pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:33:09.891242 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889599 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-log-socket\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.891242 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889623 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w6kc\" (UniqueName: \"kubernetes.io/projected/fcf69b36-5d73-458d-9324-6da53a9f1c73-kube-api-access-5w6kc\") pod \"node-resolver-87l8w\" (UID: \"fcf69b36-5d73-458d-9324-6da53a9f1c73\") " pod="openshift-dns/node-resolver-87l8w" Sep 18 03:33:09.891242 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889680 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/34b777d9-80fc-4885-a29b-0b1f1e7d7415-kubelet-config\") pod \"global-pull-secret-syncer-hv4gk\" (UID: \"34b777d9-80fc-4885-a29b-0b1f1e7d7415\") " pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:09.891242 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889704 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/34b777d9-80fc-4885-a29b-0b1f1e7d7415-dbus\") pod \"global-pull-secret-syncer-hv4gk\" (UID: \"34b777d9-80fc-4885-a29b-0b1f1e7d7415\") " pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:09.891242 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889730 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-etc-sysctl-conf\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.891242 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889733 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-multus-conf-dir\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.891242 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889754 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1013fa23-5170-4d1f-8722-86bc4b8b8742-ovn-node-metrics-cert\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.891242 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889779 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4eb89543-6d24-4d8c-aa8b-d55e3a078d21-sys-fs\") pod \"aws-ebs-csi-driver-node-d6cdv\" (UID: \"4eb89543-6d24-4d8c-aa8b-d55e3a078d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" Sep 18 03:33:09.891242 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889802 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-etc-systemd\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.891242 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889827 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-node-log\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.891242 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889840 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-host-kubelet\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.891242 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889850 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-host-var-lib-kubelet\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.891242 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889875 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-host-run-ovn-kubernetes\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.891242 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889877 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/afa7f816-ccc9-406c-bd67-55e4d933d144-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fhzm9\" (UID: \"afa7f816-ccc9-406c-bd67-55e4d933d144\") " pod="openshift-multus/multus-additional-cni-plugins-fhzm9" Sep 18 03:33:09.892073 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889902 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcf69b36-5d73-458d-9324-6da53a9f1c73-tmp-dir\") pod \"node-resolver-87l8w\" (UID: \"fcf69b36-5d73-458d-9324-6da53a9f1c73\") " pod="openshift-dns/node-resolver-87l8w" Sep 18 03:33:09.892073 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889925 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-etc-kubernetes\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.892073 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889965 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-run\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.892073 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889977 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-etc-systemd\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.892073 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889987 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-host\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.892073 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:09.889804 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:33:09.892073 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.890021 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-multus-cni-dir\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.892073 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.890045 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/afa7f816-ccc9-406c-bd67-55e4d933d144-system-cni-dir\") pod \"multus-additional-cni-plugins-fhzm9\" (UID: \"afa7f816-ccc9-406c-bd67-55e4d933d144\") " pod="openshift-multus/multus-additional-cni-plugins-fhzm9" Sep 18 03:33:09.892073 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:09.890070 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs podName:9fdcb7aa-5f4c-4d16-a250-b272513204da nodeName:}" failed. No retries permitted until 2026-09-18 03:33:10.390045344 +0000 UTC m=+4.672807200 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs") pod "network-metrics-daemon-tf6sq" (UID: "9fdcb7aa-5f4c-4d16-a250-b272513204da") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:33:09.892073 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.890083 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/afa7f816-ccc9-406c-bd67-55e4d933d144-system-cni-dir\") pod \"multus-additional-cni-plugins-fhzm9\" (UID: \"afa7f816-ccc9-406c-bd67-55e4d933d144\") " pod="openshift-multus/multus-additional-cni-plugins-fhzm9" Sep 18 03:33:09.892073 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.890119 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-etc-modprobe-d\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.892073 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.890134 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-log-socket\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.892073 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.890146 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4eb89543-6d24-4d8c-aa8b-d55e3a078d21-etc-selinux\") pod \"aws-ebs-csi-driver-node-d6cdv\" (UID: \"4eb89543-6d24-4d8c-aa8b-d55e3a078d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" Sep 18 03:33:09.892073 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.890244 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4eb89543-6d24-4d8c-aa8b-d55e3a078d21-etc-selinux\") pod \"aws-ebs-csi-driver-node-d6cdv\" (UID: \"4eb89543-6d24-4d8c-aa8b-d55e3a078d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" Sep 18 03:33:09.892073 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.890303 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-node-log\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.892073 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.890344 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-host-var-lib-kubelet\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.892073 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.890532 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcf69b36-5d73-458d-9324-6da53a9f1c73-tmp-dir\") pod \"node-resolver-87l8w\" (UID: \"fcf69b36-5d73-458d-9324-6da53a9f1c73\") " pod="openshift-dns/node-resolver-87l8w" Sep 18 03:33:09.892875 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.890569 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d9a7ff5b-3ca7-43f4-907d-3eff238968f1-konnectivity-ca\") pod \"konnectivity-agent-hs82j\" (UID: \"d9a7ff5b-3ca7-43f4-907d-3eff238968f1\") " pod="kube-system/konnectivity-agent-hs82j" Sep 18 03:33:09.892875 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.890634 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/34b777d9-80fc-4885-a29b-0b1f1e7d7415-kubelet-config\") pod \"global-pull-secret-syncer-hv4gk\" (UID: \"34b777d9-80fc-4885-a29b-0b1f1e7d7415\") " pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:09.892875 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.890630 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-etc-kubernetes\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.892875 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.889925 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4eb89543-6d24-4d8c-aa8b-d55e3a078d21-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d6cdv\" (UID: \"4eb89543-6d24-4d8c-aa8b-d55e3a078d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" Sep 18 03:33:09.892875 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.890703 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-run\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.892875 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.890737 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/34b777d9-80fc-4885-a29b-0b1f1e7d7415-dbus\") pod \"global-pull-secret-syncer-hv4gk\" (UID: \"34b777d9-80fc-4885-a29b-0b1f1e7d7415\") " pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:09.892875 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.890744 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/afa7f816-ccc9-406c-bd67-55e4d933d144-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fhzm9\" (UID: \"afa7f816-ccc9-406c-bd67-55e4d933d144\") " pod="openshift-multus/multus-additional-cni-plugins-fhzm9" Sep 18 03:33:09.892875 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.890776 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-multus-daemon-config\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.892875 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.890789 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-etc-modprobe-d\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.892875 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.890801 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-host-run-multus-certs\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.892875 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.890823 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fcf69b36-5d73-458d-9324-6da53a9f1c73-hosts-file\") pod \"node-resolver-87l8w\" (UID: \"fcf69b36-5d73-458d-9324-6da53a9f1c73\") " pod="openshift-dns/node-resolver-87l8w" Sep 18 03:33:09.892875 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.890847 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s96j\" (UniqueName: \"kubernetes.io/projected/9fdcb7aa-5f4c-4d16-a250-b272513204da-kube-api-access-2s96j\") pod \"network-metrics-daemon-tf6sq\" (UID: \"9fdcb7aa-5f4c-4d16-a250-b272513204da\") " pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:09.892875 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.890861 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-host\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.892875 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.890849 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/80a2d237-3e90-4289-8d91-46a883cb2001-etc-sysctl-conf\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.892875 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.890898 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4eb89543-6d24-4d8c-aa8b-d55e3a078d21-sys-fs\") pod \"aws-ebs-csi-driver-node-d6cdv\" (UID: \"4eb89543-6d24-4d8c-aa8b-d55e3a078d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" Sep 18 03:33:09.892875 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.890953 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-host-run-multus-certs\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.892875 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.890963 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-multus-cni-dir\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.892875 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.891003 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fcf69b36-5d73-458d-9324-6da53a9f1c73-hosts-file\") pod \"node-resolver-87l8w\" (UID: \"fcf69b36-5d73-458d-9324-6da53a9f1c73\") " pod="openshift-dns/node-resolver-87l8w" Sep 18 03:33:09.893571 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.891155 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-var-lib-openvswitch\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.893571 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.891185 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1013fa23-5170-4d1f-8722-86bc4b8b8742-env-overrides\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.893571 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.893482 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-multus-daemon-config\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.893704 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.893651 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-var-lib-openvswitch\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.893759 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.893728 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-host-var-lib-cni-multus\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.894117 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.893985 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1013fa23-5170-4d1f-8722-86bc4b8b8742-env-overrides\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.895251 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.891209 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-host-var-lib-cni-multus\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.895251 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.894171 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-host-cni-netd\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.895251 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.894220 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d9a7ff5b-3ca7-43f4-907d-3eff238968f1-agent-certs\") pod \"konnectivity-agent-hs82j\" (UID: \"d9a7ff5b-3ca7-43f4-907d-3eff238968f1\") " pod="kube-system/konnectivity-agent-hs82j" Sep 18 03:33:09.895251 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.894416 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1013fa23-5170-4d1f-8722-86bc4b8b8742-host-cni-netd\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.895251 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.895026 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/80a2d237-3e90-4289-8d91-46a883cb2001-etc-tuned\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.895251 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:09.895246 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 03:33:09.895607 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:09.895269 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 03:33:09.895607 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:09.895282 2568 projected.go:196] Error preparing data for projected volume kube-api-access-8khqd for pod openshift-network-diagnostics/network-check-target-jbxqp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:33:09.895607 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.895333 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1013fa23-5170-4d1f-8722-86bc4b8b8742-ovn-node-metrics-cert\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.895607 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:09.895343 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4437fdab-4af1-45cb-b944-acc7cbff5d20-kube-api-access-8khqd podName:4437fdab-4af1-45cb-b944-acc7cbff5d20 nodeName:}" failed. No retries permitted until 2026-09-18 03:33:10.395326413 +0000 UTC m=+4.678088267 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8khqd" (UniqueName: "kubernetes.io/projected/4437fdab-4af1-45cb-b944-acc7cbff5d20-kube-api-access-8khqd") pod "network-check-target-jbxqp" (UID: "4437fdab-4af1-45cb-b944-acc7cbff5d20") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:33:09.898882 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.898859 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v5pj\" (UniqueName: \"kubernetes.io/projected/4eb89543-6d24-4d8c-aa8b-d55e3a078d21-kube-api-access-8v5pj\") pod \"aws-ebs-csi-driver-node-d6cdv\" (UID: \"4eb89543-6d24-4d8c-aa8b-d55e3a078d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" Sep 18 03:33:09.900885 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.900568 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5n77\" (UniqueName: \"kubernetes.io/projected/2fbf3341-a0d6-4123-8bc2-799fb6b913ae-kube-api-access-c5n77\") pod \"multus-tlvwc\" (UID: \"2fbf3341-a0d6-4123-8bc2-799fb6b913ae\") " pod="openshift-multus/multus-tlvwc" Sep 18 03:33:09.901469 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.901333 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nks5f\" (UniqueName: \"kubernetes.io/projected/afa7f816-ccc9-406c-bd67-55e4d933d144-kube-api-access-nks5f\") pod \"multus-additional-cni-plugins-fhzm9\" (UID: \"afa7f816-ccc9-406c-bd67-55e4d933d144\") " pod="openshift-multus/multus-additional-cni-plugins-fhzm9" Sep 18 03:33:09.901647 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.901623 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx9nm\" (UniqueName: \"kubernetes.io/projected/1013fa23-5170-4d1f-8722-86bc4b8b8742-kube-api-access-bx9nm\") pod \"ovnkube-node-psksv\" (UID: \"1013fa23-5170-4d1f-8722-86bc4b8b8742\") " pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:09.901778 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.901519 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8zzf\" (UniqueName: \"kubernetes.io/projected/d5694211-42dd-4f1c-984b-fc2a748e8759-kube-api-access-c8zzf\") pod \"node-ca-54ct2\" (UID: \"d5694211-42dd-4f1c-984b-fc2a748e8759\") " pod="openshift-image-registry/node-ca-54ct2" Sep 18 03:33:09.902208 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.902070 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl2lq\" (UniqueName: \"kubernetes.io/projected/80a2d237-3e90-4289-8d91-46a883cb2001-kube-api-access-hl2lq\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.902310 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.902223 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phxrz\" (UniqueName: \"kubernetes.io/projected/f243ab41-9b0c-47d6-ab6f-7b300a516bb7-kube-api-access-phxrz\") pod \"iptables-alerter-ct74c\" (UID: \"f243ab41-9b0c-47d6-ab6f-7b300a516bb7\") " pod="openshift-network-operator/iptables-alerter-ct74c" Sep 18 03:33:09.902310 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.902276 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/80a2d237-3e90-4289-8d91-46a883cb2001-tmp\") pod \"tuned-vsdh4\" (UID: \"80a2d237-3e90-4289-8d91-46a883cb2001\") " pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:09.902500 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.902482 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s96j\" (UniqueName: \"kubernetes.io/projected/9fdcb7aa-5f4c-4d16-a250-b272513204da-kube-api-access-2s96j\") pod \"network-metrics-daemon-tf6sq\" (UID: \"9fdcb7aa-5f4c-4d16-a250-b272513204da\") " pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:09.903420 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.903388 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w6kc\" (UniqueName: \"kubernetes.io/projected/fcf69b36-5d73-458d-9324-6da53a9f1c73-kube-api-access-5w6kc\") pod \"node-resolver-87l8w\" (UID: \"fcf69b36-5d73-458d-9324-6da53a9f1c73\") " pod="openshift-dns/node-resolver-87l8w" Sep 18 03:33:09.904387 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:09.904368 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d9a7ff5b-3ca7-43f4-907d-3eff238968f1-agent-certs\") pod \"konnectivity-agent-hs82j\" (UID: \"d9a7ff5b-3ca7-43f4-907d-3eff238968f1\") " pod="kube-system/konnectivity-agent-hs82j" Sep 18 03:33:10.066679 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:10.066641 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" Sep 18 03:33:10.074978 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:10.074945 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tlvwc" Sep 18 03:33:10.083878 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:10.083855 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fhzm9" Sep 18 03:33:10.087806 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:10.087780 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-87l8w" Sep 18 03:33:10.093685 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:10.093666 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:10.099425 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:10.099405 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hs82j" Sep 18 03:33:10.104209 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:10.104187 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" Sep 18 03:33:10.109898 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:10.109881 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ct74c" Sep 18 03:33:10.115521 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:10.115502 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-54ct2" Sep 18 03:33:10.398916 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:10.398822 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs\") pod \"network-metrics-daemon-tf6sq\" (UID: \"9fdcb7aa-5f4c-4d16-a250-b272513204da\") " pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:10.398916 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:10.398880 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8khqd\" (UniqueName: \"kubernetes.io/projected/4437fdab-4af1-45cb-b944-acc7cbff5d20-kube-api-access-8khqd\") pod \"network-check-target-jbxqp\" (UID: \"4437fdab-4af1-45cb-b944-acc7cbff5d20\") " pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:33:10.399129 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:10.398925 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/34b777d9-80fc-4885-a29b-0b1f1e7d7415-original-pull-secret\") pod \"global-pull-secret-syncer-hv4gk\" (UID: \"34b777d9-80fc-4885-a29b-0b1f1e7d7415\") " pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:10.399129 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:10.398967 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:33:10.399129 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:10.399021 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 03:33:10.399129 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:10.399041 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 03:33:10.399129 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:10.399048 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs podName:9fdcb7aa-5f4c-4d16-a250-b272513204da nodeName:}" failed. No retries permitted until 2026-09-18 03:33:11.399025802 +0000 UTC m=+5.681787669 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs") pod "network-metrics-daemon-tf6sq" (UID: "9fdcb7aa-5f4c-4d16-a250-b272513204da") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:33:10.399129 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:10.399058 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 03:33:10.399129 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:10.399071 2568 projected.go:196] Error preparing data for projected volume kube-api-access-8khqd for pod openshift-network-diagnostics/network-check-target-jbxqp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:33:10.399129 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:10.399073 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34b777d9-80fc-4885-a29b-0b1f1e7d7415-original-pull-secret podName:34b777d9-80fc-4885-a29b-0b1f1e7d7415 nodeName:}" failed. No retries permitted until 2026-09-18 03:33:11.399059004 +0000 UTC m=+5.681820867 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/34b777d9-80fc-4885-a29b-0b1f1e7d7415-original-pull-secret") pod "global-pull-secret-syncer-hv4gk" (UID: "34b777d9-80fc-4885-a29b-0b1f1e7d7415") : object "kube-system"/"original-pull-secret" not registered Sep 18 03:33:10.399444 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:10.399145 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4437fdab-4af1-45cb-b944-acc7cbff5d20-kube-api-access-8khqd podName:4437fdab-4af1-45cb-b944-acc7cbff5d20 nodeName:}" failed. No retries permitted until 2026-09-18 03:33:11.399132445 +0000 UTC m=+5.681894286 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8khqd" (UniqueName: "kubernetes.io/projected/4437fdab-4af1-45cb-b944-acc7cbff5d20-kube-api-access-8khqd") pod "network-check-target-jbxqp" (UID: "4437fdab-4af1-45cb-b944-acc7cbff5d20") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:33:10.813146 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:10.813061 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 03:28:09 +0000 UTC" deadline="2028-05-28 00:48:50.095955608 +0000 UTC" Sep 18 03:33:10.813146 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:10.813093 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14829h15m39.282866885s" Sep 18 03:33:10.900868 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:10.900781 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafa7f816_ccc9_406c_bd67_55e4d933d144.slice/crio-9cd110fc9e49f4dc20c2b978ae8beafa63d691f4839523e1f6b35d1759ac2788 WatchSource:0}: Error finding container 9cd110fc9e49f4dc20c2b978ae8beafa63d691f4839523e1f6b35d1759ac2788: Status 404 returned error can't find the container with id 9cd110fc9e49f4dc20c2b978ae8beafa63d691f4839523e1f6b35d1759ac2788 Sep 18 03:33:10.901509 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:10.901480 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcf69b36_5d73_458d_9324_6da53a9f1c73.slice/crio-5b85d3aea666e4e7491d58ddeb9d9bc171d31d1b7f6bbeb68fb7cac1ebaa6fd3 WatchSource:0}: Error finding container 5b85d3aea666e4e7491d58ddeb9d9bc171d31d1b7f6bbeb68fb7cac1ebaa6fd3: Status 404 returned error can't find the container with id 5b85d3aea666e4e7491d58ddeb9d9bc171d31d1b7f6bbeb68fb7cac1ebaa6fd3 Sep 18 03:33:10.902084 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:10.901975 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1013fa23_5170_4d1f_8722_86bc4b8b8742.slice/crio-0ffbfdca2410b8f4a18f15f2636bae6248ecc5d3250d352e6314a1bdbba28b75 WatchSource:0}: Error finding container 0ffbfdca2410b8f4a18f15f2636bae6248ecc5d3250d352e6314a1bdbba28b75: Status 404 returned error can't find the container with id 0ffbfdca2410b8f4a18f15f2636bae6248ecc5d3250d352e6314a1bdbba28b75 Sep 18 03:33:10.905345 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:10.905322 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf243ab41_9b0c_47d6_ab6f_7b300a516bb7.slice/crio-92af6ff97a142f9858ef385dbd5fb6f334ca8bd34c800d7fb478d395eed7c5c7 WatchSource:0}: Error finding container 92af6ff97a142f9858ef385dbd5fb6f334ca8bd34c800d7fb478d395eed7c5c7: Status 404 returned error can't find the container with id 92af6ff97a142f9858ef385dbd5fb6f334ca8bd34c800d7fb478d395eed7c5c7 Sep 18 03:33:10.906314 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:10.906292 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9a7ff5b_3ca7_43f4_907d_3eff238968f1.slice/crio-300bd08569c3569bc1c9aaed950077f2d9592748ff10bf775b3bd2d7304851f7 WatchSource:0}: Error finding container 300bd08569c3569bc1c9aaed950077f2d9592748ff10bf775b3bd2d7304851f7: Status 404 returned error can't find the container with id 300bd08569c3569bc1c9aaed950077f2d9592748ff10bf775b3bd2d7304851f7 Sep 18 03:33:10.907866 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:10.907698 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fbf3341_a0d6_4123_8bc2_799fb6b913ae.slice/crio-a4e853987e38da6a8843d5a7a8e494db9d7f050a7daddedf5a9d7b7583d2093f WatchSource:0}: Error finding container a4e853987e38da6a8843d5a7a8e494db9d7f050a7daddedf5a9d7b7583d2093f: Status 404 returned error can't find the container with id a4e853987e38da6a8843d5a7a8e494db9d7f050a7daddedf5a9d7b7583d2093f Sep 18 03:33:10.909636 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:10.909333 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eb89543_6d24_4d8c_aa8b_d55e3a078d21.slice/crio-a6f1e5723487878669a9cce891aa2d331e40c1fa2fbeabb801881f0e67e10666 WatchSource:0}: Error finding container a6f1e5723487878669a9cce891aa2d331e40c1fa2fbeabb801881f0e67e10666: Status 404 returned error can't find the container with id a6f1e5723487878669a9cce891aa2d331e40c1fa2fbeabb801881f0e67e10666 Sep 18 03:33:10.910844 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:10.910821 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5694211_42dd_4f1c_984b_fc2a748e8759.slice/crio-9bb4b73aef170bbd1c36e357bd13ea24079bb573af8b216dfa4553d97b2a9b7d WatchSource:0}: Error finding container 9bb4b73aef170bbd1c36e357bd13ea24079bb573af8b216dfa4553d97b2a9b7d: Status 404 returned error can't find the container with id 9bb4b73aef170bbd1c36e357bd13ea24079bb573af8b216dfa4553d97b2a9b7d Sep 18 03:33:10.911688 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:10.911583 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80a2d237_3e90_4289_8d91_46a883cb2001.slice/crio-b183e18929dc9403f5a7a0d2136a81eb16dc4a25bc6f592cfdcdee21c7aa8cad WatchSource:0}: Error finding container b183e18929dc9403f5a7a0d2136a81eb16dc4a25bc6f592cfdcdee21c7aa8cad: Status 404 returned error can't find the container with id b183e18929dc9403f5a7a0d2136a81eb16dc4a25bc6f592cfdcdee21c7aa8cad Sep 18 03:33:11.405839 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:11.405756 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/34b777d9-80fc-4885-a29b-0b1f1e7d7415-original-pull-secret\") pod \"global-pull-secret-syncer-hv4gk\" (UID: \"34b777d9-80fc-4885-a29b-0b1f1e7d7415\") " pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:11.405839 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:11.405822 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs\") pod \"network-metrics-daemon-tf6sq\" (UID: \"9fdcb7aa-5f4c-4d16-a250-b272513204da\") " pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:11.406084 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:11.405851 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8khqd\" (UniqueName: \"kubernetes.io/projected/4437fdab-4af1-45cb-b944-acc7cbff5d20-kube-api-access-8khqd\") pod \"network-check-target-jbxqp\" (UID: \"4437fdab-4af1-45cb-b944-acc7cbff5d20\") " pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:33:11.406084 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:11.406000 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 03:33:11.406084 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:11.406022 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 03:33:11.406084 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:11.406034 2568 projected.go:196] Error preparing data for projected volume kube-api-access-8khqd for pod openshift-network-diagnostics/network-check-target-jbxqp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:33:11.406301 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:11.406093 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4437fdab-4af1-45cb-b944-acc7cbff5d20-kube-api-access-8khqd podName:4437fdab-4af1-45cb-b944-acc7cbff5d20 nodeName:}" failed. No retries permitted until 2026-09-18 03:33:13.406076295 +0000 UTC m=+7.688838143 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8khqd" (UniqueName: "kubernetes.io/projected/4437fdab-4af1-45cb-b944-acc7cbff5d20-kube-api-access-8khqd") pod "network-check-target-jbxqp" (UID: "4437fdab-4af1-45cb-b944-acc7cbff5d20") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:33:11.406487 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:11.406469 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 03:33:11.406543 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:11.406524 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34b777d9-80fc-4885-a29b-0b1f1e7d7415-original-pull-secret podName:34b777d9-80fc-4885-a29b-0b1f1e7d7415 nodeName:}" failed. No retries permitted until 2026-09-18 03:33:13.406509649 +0000 UTC m=+7.689271496 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/34b777d9-80fc-4885-a29b-0b1f1e7d7415-original-pull-secret") pod "global-pull-secret-syncer-hv4gk" (UID: "34b777d9-80fc-4885-a29b-0b1f1e7d7415") : object "kube-system"/"original-pull-secret" not registered Sep 18 03:33:11.406603 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:11.406596 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:33:11.406656 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:11.406630 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs podName:9fdcb7aa-5f4c-4d16-a250-b272513204da nodeName:}" failed. No retries permitted until 2026-09-18 03:33:13.406619149 +0000 UTC m=+7.689380995 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs") pod "network-metrics-daemon-tf6sq" (UID: "9fdcb7aa-5f4c-4d16-a250-b272513204da") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:33:11.813742 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:11.813431 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 03:28:09 +0000 UTC" deadline="2028-05-17 10:45:34.788357758 +0000 UTC" Sep 18 03:33:11.813742 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:11.813696 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14575h12m22.974668197s" Sep 18 03:33:11.878770 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:11.878582 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:11.878770 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:11.878708 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hv4gk" podUID="34b777d9-80fc-4885-a29b-0b1f1e7d7415" Sep 18 03:33:11.879116 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:11.879097 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:11.879225 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:11.879206 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tf6sq" podUID="9fdcb7aa-5f4c-4d16-a250-b272513204da" Sep 18 03:33:11.879288 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:11.879276 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:33:11.879359 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:11.879344 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbxqp" podUID="4437fdab-4af1-45cb-b944-acc7cbff5d20" Sep 18 03:33:11.901770 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:11.901741 2568 generic.go:356] "Generic (PLEG): container finished" podID="c05fdffdb3fc054c2046976206becf4f" containerID="0b6ded787077b1e1061d82a7a035be4f3741d1b54026923778f694feea06b16c" exitCode=0 Sep 18 03:33:11.902207 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:11.901971 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-80.ec2.internal" event={"ID":"c05fdffdb3fc054c2046976206becf4f","Type":"ContainerDied","Data":"0b6ded787077b1e1061d82a7a035be4f3741d1b54026923778f694feea06b16c"} Sep 18 03:33:11.910348 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:11.910270 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" event={"ID":"80a2d237-3e90-4289-8d91-46a883cb2001","Type":"ContainerStarted","Data":"b183e18929dc9403f5a7a0d2136a81eb16dc4a25bc6f592cfdcdee21c7aa8cad"} Sep 18 03:33:11.918723 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:11.918694 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hs82j" event={"ID":"d9a7ff5b-3ca7-43f4-907d-3eff238968f1","Type":"ContainerStarted","Data":"300bd08569c3569bc1c9aaed950077f2d9592748ff10bf775b3bd2d7304851f7"} Sep 18 03:33:11.926890 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:11.926861 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-54ct2" event={"ID":"d5694211-42dd-4f1c-984b-fc2a748e8759","Type":"ContainerStarted","Data":"9bb4b73aef170bbd1c36e357bd13ea24079bb573af8b216dfa4553d97b2a9b7d"} Sep 18 03:33:11.936623 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:11.936593 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" event={"ID":"4eb89543-6d24-4d8c-aa8b-d55e3a078d21","Type":"ContainerStarted","Data":"a6f1e5723487878669a9cce891aa2d331e40c1fa2fbeabb801881f0e67e10666"} Sep 18 03:33:11.950992 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:11.950957 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ct74c" event={"ID":"f243ab41-9b0c-47d6-ab6f-7b300a516bb7","Type":"ContainerStarted","Data":"92af6ff97a142f9858ef385dbd5fb6f334ca8bd34c800d7fb478d395eed7c5c7"} Sep 18 03:33:11.963009 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:11.962977 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psksv" event={"ID":"1013fa23-5170-4d1f-8722-86bc4b8b8742","Type":"ContainerStarted","Data":"0ffbfdca2410b8f4a18f15f2636bae6248ecc5d3250d352e6314a1bdbba28b75"} Sep 18 03:33:11.969895 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:11.969865 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-87l8w" event={"ID":"fcf69b36-5d73-458d-9324-6da53a9f1c73","Type":"ContainerStarted","Data":"5b85d3aea666e4e7491d58ddeb9d9bc171d31d1b7f6bbeb68fb7cac1ebaa6fd3"} Sep 18 03:33:11.988408 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:11.988371 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-80.ec2.internal" event={"ID":"20ab69c906b5fa2b928622de52fe5786","Type":"ContainerStarted","Data":"eefcba8411eb1d95dcc99c4d69a66bf61441d5d71c0a13bde621962a294eb80a"} Sep 18 03:33:12.018993 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:12.018953 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tlvwc" event={"ID":"2fbf3341-a0d6-4123-8bc2-799fb6b913ae","Type":"ContainerStarted","Data":"a4e853987e38da6a8843d5a7a8e494db9d7f050a7daddedf5a9d7b7583d2093f"} Sep 18 03:33:12.036380 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:12.036312 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhzm9" event={"ID":"afa7f816-ccc9-406c-bd67-55e4d933d144","Type":"ContainerStarted","Data":"9cd110fc9e49f4dc20c2b978ae8beafa63d691f4839523e1f6b35d1759ac2788"} Sep 18 03:33:13.065343 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:13.065197 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-80.ec2.internal" event={"ID":"c05fdffdb3fc054c2046976206becf4f","Type":"ContainerStarted","Data":"c3a1fb0719d8b3e0b1c72d06fb13b5b3c3019e7596099e3972d143ecc852e430"} Sep 18 03:33:13.079601 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:13.079156 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-80.ec2.internal" podStartSLOduration=4.079135963 podStartE2EDuration="4.079135963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 03:33:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:33:12.007957427 +0000 UTC m=+6.290719291" watchObservedRunningTime="2026-09-18 03:33:13.079135963 +0000 UTC m=+7.361897828" Sep 18 03:33:13.420986 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:13.420892 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs\") pod \"network-metrics-daemon-tf6sq\" (UID: \"9fdcb7aa-5f4c-4d16-a250-b272513204da\") " pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:13.420986 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:13.420958 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8khqd\" (UniqueName: \"kubernetes.io/projected/4437fdab-4af1-45cb-b944-acc7cbff5d20-kube-api-access-8khqd\") pod \"network-check-target-jbxqp\" (UID: \"4437fdab-4af1-45cb-b944-acc7cbff5d20\") " pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:33:13.421199 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:13.421005 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/34b777d9-80fc-4885-a29b-0b1f1e7d7415-original-pull-secret\") pod \"global-pull-secret-syncer-hv4gk\" (UID: \"34b777d9-80fc-4885-a29b-0b1f1e7d7415\") " pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:13.421199 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:13.421172 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 03:33:13.421199 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:13.421189 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 03:33:13.421199 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:13.421201 2568 projected.go:196] Error preparing data for projected volume kube-api-access-8khqd for pod openshift-network-diagnostics/network-check-target-jbxqp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:33:13.421391 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:13.421215 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 03:33:13.421391 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:13.421254 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4437fdab-4af1-45cb-b944-acc7cbff5d20-kube-api-access-8khqd podName:4437fdab-4af1-45cb-b944-acc7cbff5d20 nodeName:}" failed. No retries permitted until 2026-09-18 03:33:17.421235191 +0000 UTC m=+11.703997037 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8khqd" (UniqueName: "kubernetes.io/projected/4437fdab-4af1-45cb-b944-acc7cbff5d20-kube-api-access-8khqd") pod "network-check-target-jbxqp" (UID: "4437fdab-4af1-45cb-b944-acc7cbff5d20") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:33:13.421391 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:13.421169 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:33:13.421391 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:13.421273 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34b777d9-80fc-4885-a29b-0b1f1e7d7415-original-pull-secret podName:34b777d9-80fc-4885-a29b-0b1f1e7d7415 nodeName:}" failed. No retries permitted until 2026-09-18 03:33:17.421263329 +0000 UTC m=+11.704025176 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/34b777d9-80fc-4885-a29b-0b1f1e7d7415-original-pull-secret") pod "global-pull-secret-syncer-hv4gk" (UID: "34b777d9-80fc-4885-a29b-0b1f1e7d7415") : object "kube-system"/"original-pull-secret" not registered Sep 18 03:33:13.421391 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:13.421299 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs podName:9fdcb7aa-5f4c-4d16-a250-b272513204da nodeName:}" failed. No retries permitted until 2026-09-18 03:33:17.421281576 +0000 UTC m=+11.704043433 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs") pod "network-metrics-daemon-tf6sq" (UID: "9fdcb7aa-5f4c-4d16-a250-b272513204da") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:33:13.879858 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:13.878975 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:33:13.879858 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:13.879103 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbxqp" podUID="4437fdab-4af1-45cb-b944-acc7cbff5d20" Sep 18 03:33:13.879858 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:13.879218 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:13.879858 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:13.879301 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hv4gk" podUID="34b777d9-80fc-4885-a29b-0b1f1e7d7415" Sep 18 03:33:13.879858 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:13.879366 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:13.879858 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:13.879442 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tf6sq" podUID="9fdcb7aa-5f4c-4d16-a250-b272513204da" Sep 18 03:33:15.876876 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:15.876844 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:33:15.877335 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:15.876988 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbxqp" podUID="4437fdab-4af1-45cb-b944-acc7cbff5d20" Sep 18 03:33:15.877335 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:15.877176 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:15.877335 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:15.877283 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hv4gk" podUID="34b777d9-80fc-4885-a29b-0b1f1e7d7415" Sep 18 03:33:15.877510 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:15.877397 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:15.877567 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:15.877510 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tf6sq" podUID="9fdcb7aa-5f4c-4d16-a250-b272513204da" Sep 18 03:33:17.455273 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:17.455196 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/34b777d9-80fc-4885-a29b-0b1f1e7d7415-original-pull-secret\") pod \"global-pull-secret-syncer-hv4gk\" (UID: \"34b777d9-80fc-4885-a29b-0b1f1e7d7415\") " pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:17.455273 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:17.455271 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs\") pod \"network-metrics-daemon-tf6sq\" (UID: \"9fdcb7aa-5f4c-4d16-a250-b272513204da\") " pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:17.455803 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:17.455300 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8khqd\" (UniqueName: \"kubernetes.io/projected/4437fdab-4af1-45cb-b944-acc7cbff5d20-kube-api-access-8khqd\") pod \"network-check-target-jbxqp\" (UID: \"4437fdab-4af1-45cb-b944-acc7cbff5d20\") " pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:33:17.455803 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:17.455321 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 03:33:17.455803 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:17.455384 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34b777d9-80fc-4885-a29b-0b1f1e7d7415-original-pull-secret podName:34b777d9-80fc-4885-a29b-0b1f1e7d7415 nodeName:}" failed. No retries permitted until 2026-09-18 03:33:25.455365606 +0000 UTC m=+19.738127454 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/34b777d9-80fc-4885-a29b-0b1f1e7d7415-original-pull-secret") pod "global-pull-secret-syncer-hv4gk" (UID: "34b777d9-80fc-4885-a29b-0b1f1e7d7415") : object "kube-system"/"original-pull-secret" not registered Sep 18 03:33:17.455803 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:17.455433 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:33:17.455803 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:17.455480 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 03:33:17.455803 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:17.455496 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 03:33:17.455803 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:17.455498 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs podName:9fdcb7aa-5f4c-4d16-a250-b272513204da nodeName:}" failed. No retries permitted until 2026-09-18 03:33:25.455480068 +0000 UTC m=+19.738241913 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs") pod "network-metrics-daemon-tf6sq" (UID: "9fdcb7aa-5f4c-4d16-a250-b272513204da") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:33:17.455803 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:17.455509 2568 projected.go:196] Error preparing data for projected volume kube-api-access-8khqd for pod openshift-network-diagnostics/network-check-target-jbxqp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:33:17.455803 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:17.455549 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4437fdab-4af1-45cb-b944-acc7cbff5d20-kube-api-access-8khqd podName:4437fdab-4af1-45cb-b944-acc7cbff5d20 nodeName:}" failed. No retries permitted until 2026-09-18 03:33:25.45553854 +0000 UTC m=+19.738300393 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8khqd" (UniqueName: "kubernetes.io/projected/4437fdab-4af1-45cb-b944-acc7cbff5d20-kube-api-access-8khqd") pod "network-check-target-jbxqp" (UID: "4437fdab-4af1-45cb-b944-acc7cbff5d20") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:33:17.878328 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:17.877611 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:33:17.878328 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:17.877717 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbxqp" podUID="4437fdab-4af1-45cb-b944-acc7cbff5d20" Sep 18 03:33:17.878328 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:17.878131 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:17.878328 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:17.878227 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hv4gk" podUID="34b777d9-80fc-4885-a29b-0b1f1e7d7415" Sep 18 03:33:17.878831 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:17.878694 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:17.878831 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:17.878798 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tf6sq" podUID="9fdcb7aa-5f4c-4d16-a250-b272513204da" Sep 18 03:33:19.876045 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:19.876010 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:19.876525 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:19.876139 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hv4gk" podUID="34b777d9-80fc-4885-a29b-0b1f1e7d7415" Sep 18 03:33:19.876525 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:19.876451 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:19.876642 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:19.876532 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tf6sq" podUID="9fdcb7aa-5f4c-4d16-a250-b272513204da" Sep 18 03:33:19.876702 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:19.876663 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:33:19.876770 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:19.876748 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbxqp" podUID="4437fdab-4af1-45cb-b944-acc7cbff5d20" Sep 18 03:33:21.877037 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:21.876992 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:21.877499 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:21.876992 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:33:21.877499 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:21.877121 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tf6sq" podUID="9fdcb7aa-5f4c-4d16-a250-b272513204da" Sep 18 03:33:21.877499 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:21.876992 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:21.877499 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:21.877184 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbxqp" podUID="4437fdab-4af1-45cb-b944-acc7cbff5d20" Sep 18 03:33:21.877499 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:21.877268 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hv4gk" podUID="34b777d9-80fc-4885-a29b-0b1f1e7d7415" Sep 18 03:33:23.876293 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:23.876261 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:23.876293 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:23.876282 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:23.876740 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:23.876379 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:33:23.876740 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:23.876399 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tf6sq" podUID="9fdcb7aa-5f4c-4d16-a250-b272513204da" Sep 18 03:33:23.876740 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:23.876482 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbxqp" podUID="4437fdab-4af1-45cb-b944-acc7cbff5d20" Sep 18 03:33:23.876740 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:23.876567 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hv4gk" podUID="34b777d9-80fc-4885-a29b-0b1f1e7d7415" Sep 18 03:33:25.508208 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:25.508171 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/34b777d9-80fc-4885-a29b-0b1f1e7d7415-original-pull-secret\") pod \"global-pull-secret-syncer-hv4gk\" (UID: \"34b777d9-80fc-4885-a29b-0b1f1e7d7415\") " pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:25.508644 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:25.508225 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs\") pod \"network-metrics-daemon-tf6sq\" (UID: \"9fdcb7aa-5f4c-4d16-a250-b272513204da\") " pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:25.508644 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:25.508251 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8khqd\" (UniqueName: \"kubernetes.io/projected/4437fdab-4af1-45cb-b944-acc7cbff5d20-kube-api-access-8khqd\") pod \"network-check-target-jbxqp\" (UID: \"4437fdab-4af1-45cb-b944-acc7cbff5d20\") " pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:33:25.508644 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:25.508335 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 03:33:25.508644 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:25.508365 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 03:33:25.508644 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:25.508380 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 03:33:25.508644 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:25.508388 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:33:25.508644 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:25.508414 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34b777d9-80fc-4885-a29b-0b1f1e7d7415-original-pull-secret podName:34b777d9-80fc-4885-a29b-0b1f1e7d7415 nodeName:}" failed. No retries permitted until 2026-09-18 03:33:41.508395763 +0000 UTC m=+35.791157627 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/34b777d9-80fc-4885-a29b-0b1f1e7d7415-original-pull-secret") pod "global-pull-secret-syncer-hv4gk" (UID: "34b777d9-80fc-4885-a29b-0b1f1e7d7415") : object "kube-system"/"original-pull-secret" not registered Sep 18 03:33:25.508644 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:25.508441 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs podName:9fdcb7aa-5f4c-4d16-a250-b272513204da nodeName:}" failed. No retries permitted until 2026-09-18 03:33:41.508423432 +0000 UTC m=+35.791185294 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs") pod "network-metrics-daemon-tf6sq" (UID: "9fdcb7aa-5f4c-4d16-a250-b272513204da") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:33:25.508644 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:25.508391 2568 projected.go:196] Error preparing data for projected volume kube-api-access-8khqd for pod openshift-network-diagnostics/network-check-target-jbxqp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:33:25.508644 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:25.508492 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4437fdab-4af1-45cb-b944-acc7cbff5d20-kube-api-access-8khqd podName:4437fdab-4af1-45cb-b944-acc7cbff5d20 nodeName:}" failed. No retries permitted until 2026-09-18 03:33:41.508482221 +0000 UTC m=+35.791244067 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8khqd" (UniqueName: "kubernetes.io/projected/4437fdab-4af1-45cb-b944-acc7cbff5d20-kube-api-access-8khqd") pod "network-check-target-jbxqp" (UID: "4437fdab-4af1-45cb-b944-acc7cbff5d20") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:33:25.876538 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:25.876451 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:33:25.876538 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:25.876455 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:25.876756 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:25.876575 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:25.876756 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:25.876574 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbxqp" podUID="4437fdab-4af1-45cb-b944-acc7cbff5d20" Sep 18 03:33:25.876756 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:25.876675 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hv4gk" podUID="34b777d9-80fc-4885-a29b-0b1f1e7d7415" Sep 18 03:33:25.876906 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:25.876766 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tf6sq" podUID="9fdcb7aa-5f4c-4d16-a250-b272513204da" Sep 18 03:33:27.879014 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:27.878986 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:27.879014 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:27.879006 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:27.879341 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:27.878986 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:33:27.879341 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:27.879148 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbxqp" podUID="4437fdab-4af1-45cb-b944-acc7cbff5d20" Sep 18 03:33:27.879341 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:27.879072 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tf6sq" podUID="9fdcb7aa-5f4c-4d16-a250-b272513204da" Sep 18 03:33:27.879341 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:27.879234 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hv4gk" podUID="34b777d9-80fc-4885-a29b-0b1f1e7d7415" Sep 18 03:33:29.098272 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.098162 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" event={"ID":"80a2d237-3e90-4289-8d91-46a883cb2001","Type":"ContainerStarted","Data":"e4a0c7ff8303284a50fedf40d085a99690bfb2abc4330d4272b9f954306652a6"} Sep 18 03:33:29.099878 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.099851 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hs82j" event={"ID":"d9a7ff5b-3ca7-43f4-907d-3eff238968f1","Type":"ContainerStarted","Data":"f842a2f39478222aadadfb80c00e4a1a645ef5019d1a48911c81e9337bcfb182"} Sep 18 03:33:29.101480 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.101453 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-54ct2" event={"ID":"d5694211-42dd-4f1c-984b-fc2a748e8759","Type":"ContainerStarted","Data":"f37effcd6ac3b79bc0ff0d72529ed18c03a7c969ed304996c9eccd80584e7d9c"} Sep 18 03:33:29.103217 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.103194 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" event={"ID":"4eb89543-6d24-4d8c-aa8b-d55e3a078d21","Type":"ContainerStarted","Data":"577e3244169c01046040e6c0e2f4fc60ba7608019c67dd5494869dfcd34c1863"} Sep 18 03:33:29.106217 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.106199 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psksv_1013fa23-5170-4d1f-8722-86bc4b8b8742/ovn-acl-logging/0.log" Sep 18 03:33:29.106608 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.106588 2568 generic.go:356] "Generic (PLEG): container finished" podID="1013fa23-5170-4d1f-8722-86bc4b8b8742" containerID="374129dfb95744846d8e0d4ba72cd2b17271538e49b97d04261ad9821a335178" exitCode=1 Sep 18 03:33:29.106686 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.106651 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psksv" event={"ID":"1013fa23-5170-4d1f-8722-86bc4b8b8742","Type":"ContainerStarted","Data":"51f03e2a13b5f951becb24826b90ebae55201088ad863bbb6d94607d6d94a9fd"} Sep 18 03:33:29.106744 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.106727 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psksv" event={"ID":"1013fa23-5170-4d1f-8722-86bc4b8b8742","Type":"ContainerStarted","Data":"01008042061c46297a874821802c2c994152789040342f2ca099916e438b91c1"} Sep 18 03:33:29.106800 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.106776 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psksv" event={"ID":"1013fa23-5170-4d1f-8722-86bc4b8b8742","Type":"ContainerStarted","Data":"1b94cceb563cd1c7ce7fe09cce6e25850ae985cc1343802b285e99f1c0bad0d1"} Sep 18 03:33:29.106841 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.106820 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psksv" event={"ID":"1013fa23-5170-4d1f-8722-86bc4b8b8742","Type":"ContainerStarted","Data":"d55ba36bdb60fa8e777669201f54218bca36b05eb4fb5e07bd4681946188b2ab"} Sep 18 03:33:29.106883 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.106868 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psksv" event={"ID":"1013fa23-5170-4d1f-8722-86bc4b8b8742","Type":"ContainerDied","Data":"374129dfb95744846d8e0d4ba72cd2b17271538e49b97d04261ad9821a335178"} Sep 18 03:33:29.106927 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.106916 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psksv" event={"ID":"1013fa23-5170-4d1f-8722-86bc4b8b8742","Type":"ContainerStarted","Data":"fdb1df0a2e59fd1d528f9668f7e192c46c16bcf24c4a37cba73e66ef304ee7b4"} Sep 18 03:33:29.108150 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.108131 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-87l8w" event={"ID":"fcf69b36-5d73-458d-9324-6da53a9f1c73","Type":"ContainerStarted","Data":"50ca22b1537434be32112c8c4282dc6f159478ca8fb04a738b8ee18522ebe1c5"} Sep 18 03:33:29.109674 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.109652 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tlvwc" event={"ID":"2fbf3341-a0d6-4123-8bc2-799fb6b913ae","Type":"ContainerStarted","Data":"c65c68ddbc375658f73518513d0d225ecc2c81449d697d006545a3f1778118d5"} Sep 18 03:33:29.111132 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.111112 2568 generic.go:356] "Generic (PLEG): container finished" podID="afa7f816-ccc9-406c-bd67-55e4d933d144" containerID="a4c2f6e872eec1f9f82b97351833ac7abb8610dd462859ddc7d787ea775af1db" exitCode=0 Sep 18 03:33:29.111228 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.111142 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhzm9" event={"ID":"afa7f816-ccc9-406c-bd67-55e4d933d144","Type":"ContainerDied","Data":"a4c2f6e872eec1f9f82b97351833ac7abb8610dd462859ddc7d787ea775af1db"} Sep 18 03:33:29.113294 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.113254 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vsdh4" podStartSLOduration=4.973233431 podStartE2EDuration="22.113242945s" totalImagesPullingTime="17.140009514s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 03:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:33:29.113236665 +0000 UTC m=+23.395998529" watchObservedRunningTime="2026-09-18 03:33:29.113242945 +0000 UTC m=+23.396004808" Sep 18 03:33:29.113403 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.113378 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-80.ec2.internal" podStartSLOduration=20.113369861 podStartE2EDuration="20.113369861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 03:33:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:33:13.079586697 +0000 UTC m=+7.362348560" watchObservedRunningTime="2026-09-18 03:33:29.113369861 +0000 UTC m=+23.396131725" Sep 18 03:33:29.126020 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.125983 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-54ct2" podStartSLOduration=3.9870909279999998 podStartE2EDuration="21.125965501s" totalImagesPullingTime="17.138874573s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 03:33:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:33:29.125620853 +0000 UTC m=+23.408382716" watchObservedRunningTime="2026-09-18 03:33:29.125965501 +0000 UTC m=+23.408727365" Sep 18 03:33:29.154130 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.154078 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-87l8w" podStartSLOduration=5.006368305 podStartE2EDuration="22.154062998s" totalImagesPullingTime="17.147694693s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 03:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:33:29.153989521 +0000 UTC m=+23.436751383" watchObservedRunningTime="2026-09-18 03:33:29.154062998 +0000 UTC m=+23.436824861" Sep 18 03:33:29.154358 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.154340 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tlvwc" podStartSLOduration=4.969435184 podStartE2EDuration="22.154334198s" totalImagesPullingTime="17.184899014s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 03:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:33:29.14008915 +0000 UTC m=+23.422851013" watchObservedRunningTime="2026-09-18 03:33:29.154334198 +0000 UTC m=+23.437096081" Sep 18 03:33:29.187761 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.187714 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hs82j" podStartSLOduration=8.054475153 podStartE2EDuration="21.187699665s" totalImagesPullingTime="13.133224512s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:33:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:33:29.187490535 +0000 UTC m=+23.470252398" watchObservedRunningTime="2026-09-18 03:33:29.187699665 +0000 UTC m=+23.470461529" Sep 18 03:33:29.703580 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.703547 2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 18 03:33:29.842339 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.842220 2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-18T03:33:29.70357166Z","UUID":"0f9e85d6-bd53-4598-95ab-90b4aea16419","Handler":null,"Name":"","Endpoint":""} Sep 18 03:33:29.843945 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.843903 2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 18 03:33:29.844060 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.843956 2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 18 03:33:29.879692 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.879668 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:33:29.879867 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.879754 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:29.879867 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:29.879766 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbxqp" podUID="4437fdab-4af1-45cb-b944-acc7cbff5d20" Sep 18 03:33:29.879867 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:29.879789 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:29.880040 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:29.879882 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tf6sq" podUID="9fdcb7aa-5f4c-4d16-a250-b272513204da" Sep 18 03:33:29.880040 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:29.879960 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hv4gk" podUID="34b777d9-80fc-4885-a29b-0b1f1e7d7415" Sep 18 03:33:30.114989 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:30.114908 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" event={"ID":"4eb89543-6d24-4d8c-aa8b-d55e3a078d21","Type":"ContainerStarted","Data":"3e60d37790054c7516941d472910b8f0c59de44f04a26603bc98a1aa85b2054e"} Sep 18 03:33:30.116211 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:30.116183 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ct74c" event={"ID":"f243ab41-9b0c-47d6-ab6f-7b300a516bb7","Type":"ContainerStarted","Data":"d4fdcafa300be5c94a9f7d62ae901d918ad4cf95cb58b311c85f2d19c28ab843"} Sep 18 03:33:30.129653 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:30.129609 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-ct74c" podStartSLOduration=4.985645227 podStartE2EDuration="22.129594003s" totalImagesPullingTime="17.143948776s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 03:33:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:33:30.129481225 +0000 UTC m=+24.412243089" watchObservedRunningTime="2026-09-18 03:33:30.129594003 +0000 UTC m=+24.412355928" Sep 18 03:33:30.783652 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:30.783613 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hs82j" Sep 18 03:33:30.784335 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:30.784308 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hs82j" Sep 18 03:33:31.121012 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:31.120975 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" event={"ID":"4eb89543-6d24-4d8c-aa8b-d55e3a078d21","Type":"ContainerStarted","Data":"c4331796d0844f8252c64d9ee852ce1f173aa706a9f95e984b0fec4ac7e4f07b"} Sep 18 03:33:31.128375 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:31.128346 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psksv_1013fa23-5170-4d1f-8722-86bc4b8b8742/ovn-acl-logging/0.log" Sep 18 03:33:31.128773 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:31.128737 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psksv" event={"ID":"1013fa23-5170-4d1f-8722-86bc4b8b8742","Type":"ContainerStarted","Data":"c24ac8e492b999094f5a47a58d29d946b878e422da339a4568fbb1f9244e0b15"} Sep 18 03:33:31.129148 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:31.129129 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hs82j" Sep 18 03:33:31.129540 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:31.129521 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hs82j" Sep 18 03:33:31.138945 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:31.138886 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d6cdv" podStartSLOduration=3.509679622 podStartE2EDuration="23.138873485s" totalImagesPullingTime="19.629193863s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 03:33:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:33:31.1382674 +0000 UTC m=+25.421029278" watchObservedRunningTime="2026-09-18 03:33:31.138873485 +0000 UTC m=+25.421635389" Sep 18 03:33:31.880016 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:31.879985 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:31.880195 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:31.879985 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:31.880195 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:31.880110 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hv4gk" podUID="34b777d9-80fc-4885-a29b-0b1f1e7d7415" Sep 18 03:33:31.880195 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:31.879992 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:33:31.880351 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:31.880196 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tf6sq" podUID="9fdcb7aa-5f4c-4d16-a250-b272513204da" Sep 18 03:33:31.880351 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:31.880244 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbxqp" podUID="4437fdab-4af1-45cb-b944-acc7cbff5d20" Sep 18 03:33:33.876603 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:33.876432 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:33:33.876981 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:33.876451 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:33.876981 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:33.876451 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:33.876981 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:33.876670 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbxqp" podUID="4437fdab-4af1-45cb-b944-acc7cbff5d20" Sep 18 03:33:33.876981 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:33.876750 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hv4gk" podUID="34b777d9-80fc-4885-a29b-0b1f1e7d7415" Sep 18 03:33:33.876981 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:33.876871 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tf6sq" podUID="9fdcb7aa-5f4c-4d16-a250-b272513204da" Sep 18 03:33:34.139204 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:34.139125 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psksv_1013fa23-5170-4d1f-8722-86bc4b8b8742/ovn-acl-logging/0.log" Sep 18 03:33:34.139619 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:34.139589 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psksv" event={"ID":"1013fa23-5170-4d1f-8722-86bc4b8b8742","Type":"ContainerStarted","Data":"821cc095e25f044277708628b1786250f87efddbe9035e0bec4901493fd3b624"} Sep 18 03:33:34.139879 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:34.139859 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:34.140050 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:34.140036 2568 scope.go:117] "RemoveContainer" containerID="374129dfb95744846d8e0d4ba72cd2b17271538e49b97d04261ad9821a335178" Sep 18 03:33:34.156553 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:34.156528 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:35.144040 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:35.143961 2568 generic.go:356] "Generic (PLEG): container finished" podID="afa7f816-ccc9-406c-bd67-55e4d933d144" containerID="e8fd6ff10056e8709afba519b296c593bf89f63f33f63b1051da9d3ef35f3755" exitCode=0 Sep 18 03:33:35.144844 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:35.144065 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhzm9" event={"ID":"afa7f816-ccc9-406c-bd67-55e4d933d144","Type":"ContainerDied","Data":"e8fd6ff10056e8709afba519b296c593bf89f63f33f63b1051da9d3ef35f3755"} Sep 18 03:33:35.148112 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:35.148083 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psksv_1013fa23-5170-4d1f-8722-86bc4b8b8742/ovn-acl-logging/0.log" Sep 18 03:33:35.148438 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:35.148417 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psksv" event={"ID":"1013fa23-5170-4d1f-8722-86bc4b8b8742","Type":"ContainerStarted","Data":"0500f46fbd184933777c6fdafe2e54844abfb804c9a1b7d5cd1a4a1336920ee9"} Sep 18 03:33:35.148527 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:35.148505 2568 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 18 03:33:35.148758 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:35.148735 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:35.163999 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:35.163969 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:35.187793 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:35.187729 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-psksv" podStartSLOduration=11.017684108 podStartE2EDuration="28.187708662s" totalImagesPullingTime="17.170024554s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 03:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:33:35.185688641 +0000 UTC m=+29.468450504" watchObservedRunningTime="2026-09-18 03:33:35.187708662 +0000 UTC m=+29.470470525" Sep 18 03:33:35.876651 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:35.876433 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:35.876858 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:35.876446 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:35.876858 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:35.876738 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hv4gk" podUID="34b777d9-80fc-4885-a29b-0b1f1e7d7415" Sep 18 03:33:35.876858 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:35.876813 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tf6sq" podUID="9fdcb7aa-5f4c-4d16-a250-b272513204da" Sep 18 03:33:35.876858 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:35.876486 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:33:35.877079 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:35.876887 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbxqp" podUID="4437fdab-4af1-45cb-b944-acc7cbff5d20" Sep 18 03:33:35.889121 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:35.889095 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hv4gk"] Sep 18 03:33:35.893025 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:35.892957 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jbxqp"] Sep 18 03:33:35.893686 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:35.893652 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tf6sq"] Sep 18 03:33:36.151068 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:36.151040 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:36.151068 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:36.151052 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:33:36.151502 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:36.151113 2568 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 18 03:33:36.151502 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:36.151050 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:36.151502 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:36.151138 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hv4gk" podUID="34b777d9-80fc-4885-a29b-0b1f1e7d7415" Sep 18 03:33:36.151502 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:36.151470 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbxqp" podUID="4437fdab-4af1-45cb-b944-acc7cbff5d20" Sep 18 03:33:36.151667 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:36.151532 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tf6sq" podUID="9fdcb7aa-5f4c-4d16-a250-b272513204da" Sep 18 03:33:36.925970 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:36.925929 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:33:37.155793 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:37.155761 2568 generic.go:356] "Generic (PLEG): container finished" podID="afa7f816-ccc9-406c-bd67-55e4d933d144" containerID="ab2e38d131371460062234ef601d36964872a53b1b3a2cb0e1883b6262141436" exitCode=0 Sep 18 03:33:37.156127 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:37.155808 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhzm9" event={"ID":"afa7f816-ccc9-406c-bd67-55e4d933d144","Type":"ContainerDied","Data":"ab2e38d131371460062234ef601d36964872a53b1b3a2cb0e1883b6262141436"} Sep 18 03:33:37.877061 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:37.877023 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:33:37.877236 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:37.877128 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:37.877236 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:37.877128 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbxqp" podUID="4437fdab-4af1-45cb-b944-acc7cbff5d20" Sep 18 03:33:37.877236 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:37.877213 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hv4gk" podUID="34b777d9-80fc-4885-a29b-0b1f1e7d7415" Sep 18 03:33:37.877354 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:37.877252 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:37.877354 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:37.877307 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tf6sq" podUID="9fdcb7aa-5f4c-4d16-a250-b272513204da" Sep 18 03:33:38.164064 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:38.164022 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhzm9" event={"ID":"afa7f816-ccc9-406c-bd67-55e4d933d144","Type":"ContainerStarted","Data":"9f45813eba6779503deb5b02404ea628e55238060d06ceb158b67169afec9bdf"} Sep 18 03:33:38.184184 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:38.184135 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-psksv" podUID="1013fa23-5170-4d1f-8722-86bc4b8b8742" containerName="ovnkube-controller" probeResult="failure" output="" Sep 18 03:33:39.169493 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:39.169456 2568 generic.go:356] "Generic (PLEG): container finished" podID="afa7f816-ccc9-406c-bd67-55e4d933d144" containerID="9f45813eba6779503deb5b02404ea628e55238060d06ceb158b67169afec9bdf" exitCode=0 Sep 18 03:33:39.169996 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:39.169529 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhzm9" event={"ID":"afa7f816-ccc9-406c-bd67-55e4d933d144","Type":"ContainerDied","Data":"9f45813eba6779503deb5b02404ea628e55238060d06ceb158b67169afec9bdf"} Sep 18 03:33:39.876336 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:39.876298 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:39.876520 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:39.876308 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:39.876520 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:39.876420 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tf6sq" podUID="9fdcb7aa-5f4c-4d16-a250-b272513204da" Sep 18 03:33:39.876520 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:39.876502 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hv4gk" podUID="34b777d9-80fc-4885-a29b-0b1f1e7d7415" Sep 18 03:33:39.876669 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:39.876308 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:33:39.876669 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:39.876587 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbxqp" podUID="4437fdab-4af1-45cb-b944-acc7cbff5d20" Sep 18 03:33:41.195322 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.195283 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-80.ec2.internal" event="NodeReady" Sep 18 03:33:41.195769 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.195441 2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 18 03:33:41.235238 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.235204 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dwxzz"] Sep 18 03:33:41.251000 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.250787 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5ccx6"] Sep 18 03:33:41.251000 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.250921 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dwxzz" Sep 18 03:33:41.253853 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.253376 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 18 03:33:41.253853 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.253431 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4fvk6\"" Sep 18 03:33:41.253853 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.253436 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 18 03:33:41.263117 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.263096 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dwxzz"] Sep 18 03:33:41.263258 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.263136 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5ccx6"] Sep 18 03:33:41.263258 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.263230 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5ccx6" Sep 18 03:33:41.265988 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.265963 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 18 03:33:41.266112 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.265991 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 18 03:33:41.266112 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.265997 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 18 03:33:41.266112 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.266009 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gb67w\"" Sep 18 03:33:41.326139 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.326110 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mz2m\" (UniqueName: \"kubernetes.io/projected/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-kube-api-access-6mz2m\") pod \"dns-default-dwxzz\" (UID: \"d5d06eb4-28cf-4325-8e91-19c6ee8b5b75\") " pod="openshift-dns/dns-default-dwxzz" Sep 18 03:33:41.326348 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.326178 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-config-volume\") pod \"dns-default-dwxzz\" (UID: \"d5d06eb4-28cf-4325-8e91-19c6ee8b5b75\") " pod="openshift-dns/dns-default-dwxzz" Sep 18 03:33:41.326348 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.326209 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-tmp-dir\") pod \"dns-default-dwxzz\" (UID: \"d5d06eb4-28cf-4325-8e91-19c6ee8b5b75\") " pod="openshift-dns/dns-default-dwxzz" Sep 18 03:33:41.326348 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.326279 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls\") pod \"dns-default-dwxzz\" (UID: \"d5d06eb4-28cf-4325-8e91-19c6ee8b5b75\") " pod="openshift-dns/dns-default-dwxzz" Sep 18 03:33:41.427462 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.427429 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mz2m\" (UniqueName: \"kubernetes.io/projected/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-kube-api-access-6mz2m\") pod \"dns-default-dwxzz\" (UID: \"d5d06eb4-28cf-4325-8e91-19c6ee8b5b75\") " pod="openshift-dns/dns-default-dwxzz" Sep 18 03:33:41.427669 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.427493 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-config-volume\") pod \"dns-default-dwxzz\" (UID: \"d5d06eb4-28cf-4325-8e91-19c6ee8b5b75\") " pod="openshift-dns/dns-default-dwxzz" Sep 18 03:33:41.427669 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.427515 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ad38ce3-1483-4747-a9b3-7ccbfbb71957-cert\") pod \"ingress-canary-5ccx6\" (UID: \"2ad38ce3-1483-4747-a9b3-7ccbfbb71957\") " pod="openshift-ingress-canary/ingress-canary-5ccx6" Sep 18 03:33:41.427669 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.427532 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpxh6\" (UniqueName: \"kubernetes.io/projected/2ad38ce3-1483-4747-a9b3-7ccbfbb71957-kube-api-access-kpxh6\") pod \"ingress-canary-5ccx6\" (UID: \"2ad38ce3-1483-4747-a9b3-7ccbfbb71957\") " pod="openshift-ingress-canary/ingress-canary-5ccx6" Sep 18 03:33:41.427669 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.427550 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-tmp-dir\") pod \"dns-default-dwxzz\" (UID: \"d5d06eb4-28cf-4325-8e91-19c6ee8b5b75\") " pod="openshift-dns/dns-default-dwxzz" Sep 18 03:33:41.427669 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.427578 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls\") pod \"dns-default-dwxzz\" (UID: \"d5d06eb4-28cf-4325-8e91-19c6ee8b5b75\") " pod="openshift-dns/dns-default-dwxzz" Sep 18 03:33:41.427888 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:41.427681 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 03:33:41.427888 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:41.427722 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls podName:d5d06eb4-28cf-4325-8e91-19c6ee8b5b75 nodeName:}" failed. No retries permitted until 2026-09-18 03:33:41.927710706 +0000 UTC m=+36.210472560 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls") pod "dns-default-dwxzz" (UID: "d5d06eb4-28cf-4325-8e91-19c6ee8b5b75") : secret "dns-default-metrics-tls" not found Sep 18 03:33:41.427996 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.427971 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-tmp-dir\") pod \"dns-default-dwxzz\" (UID: \"d5d06eb4-28cf-4325-8e91-19c6ee8b5b75\") " pod="openshift-dns/dns-default-dwxzz" Sep 18 03:33:41.428156 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.428139 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-config-volume\") pod \"dns-default-dwxzz\" (UID: \"d5d06eb4-28cf-4325-8e91-19c6ee8b5b75\") " pod="openshift-dns/dns-default-dwxzz" Sep 18 03:33:41.437725 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.437698 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mz2m\" (UniqueName: \"kubernetes.io/projected/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-kube-api-access-6mz2m\") pod \"dns-default-dwxzz\" (UID: \"d5d06eb4-28cf-4325-8e91-19c6ee8b5b75\") " pod="openshift-dns/dns-default-dwxzz" Sep 18 03:33:41.528577 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.528533 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/34b777d9-80fc-4885-a29b-0b1f1e7d7415-original-pull-secret\") pod \"global-pull-secret-syncer-hv4gk\" (UID: \"34b777d9-80fc-4885-a29b-0b1f1e7d7415\") " pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:41.528577 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.528586 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs\") pod \"network-metrics-daemon-tf6sq\" (UID: \"9fdcb7aa-5f4c-4d16-a250-b272513204da\") " pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:41.528839 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.528604 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8khqd\" (UniqueName: \"kubernetes.io/projected/4437fdab-4af1-45cb-b944-acc7cbff5d20-kube-api-access-8khqd\") pod \"network-check-target-jbxqp\" (UID: \"4437fdab-4af1-45cb-b944-acc7cbff5d20\") " pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:33:41.528839 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.528620 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ad38ce3-1483-4747-a9b3-7ccbfbb71957-cert\") pod \"ingress-canary-5ccx6\" (UID: \"2ad38ce3-1483-4747-a9b3-7ccbfbb71957\") " pod="openshift-ingress-canary/ingress-canary-5ccx6" Sep 18 03:33:41.528839 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:41.528688 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 03:33:41.528839 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:41.528707 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:33:41.528839 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:41.528736 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 03:33:41.528839 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.528743 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpxh6\" (UniqueName: \"kubernetes.io/projected/2ad38ce3-1483-4747-a9b3-7ccbfbb71957-kube-api-access-kpxh6\") pod \"ingress-canary-5ccx6\" (UID: \"2ad38ce3-1483-4747-a9b3-7ccbfbb71957\") " pod="openshift-ingress-canary/ingress-canary-5ccx6" Sep 18 03:33:41.528839 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:41.528758 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 03:33:41.528839 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:41.528773 2568 projected.go:196] Error preparing data for projected volume kube-api-access-8khqd for pod openshift-network-diagnostics/network-check-target-jbxqp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:33:41.528839 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:41.528761 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34b777d9-80fc-4885-a29b-0b1f1e7d7415-original-pull-secret podName:34b777d9-80fc-4885-a29b-0b1f1e7d7415 nodeName:}" failed. No retries permitted until 2026-09-18 03:34:13.528742295 +0000 UTC m=+67.811504150 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/34b777d9-80fc-4885-a29b-0b1f1e7d7415-original-pull-secret") pod "global-pull-secret-syncer-hv4gk" (UID: "34b777d9-80fc-4885-a29b-0b1f1e7d7415") : object "kube-system"/"original-pull-secret" not registered Sep 18 03:33:41.529184 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:41.528850 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs podName:9fdcb7aa-5f4c-4d16-a250-b272513204da nodeName:}" failed. No retries permitted until 2026-09-18 03:34:13.528827289 +0000 UTC m=+67.811589145 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs") pod "network-metrics-daemon-tf6sq" (UID: "9fdcb7aa-5f4c-4d16-a250-b272513204da") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:33:41.529184 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:41.528876 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4437fdab-4af1-45cb-b944-acc7cbff5d20-kube-api-access-8khqd podName:4437fdab-4af1-45cb-b944-acc7cbff5d20 nodeName:}" failed. No retries permitted until 2026-09-18 03:34:13.528867986 +0000 UTC m=+67.811629827 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8khqd" (UniqueName: "kubernetes.io/projected/4437fdab-4af1-45cb-b944-acc7cbff5d20-kube-api-access-8khqd") pod "network-check-target-jbxqp" (UID: "4437fdab-4af1-45cb-b944-acc7cbff5d20") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:33:41.531337 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.531315 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ad38ce3-1483-4747-a9b3-7ccbfbb71957-cert\") pod \"ingress-canary-5ccx6\" (UID: \"2ad38ce3-1483-4747-a9b3-7ccbfbb71957\") " pod="openshift-ingress-canary/ingress-canary-5ccx6" Sep 18 03:33:41.536550 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.536520 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpxh6\" (UniqueName: \"kubernetes.io/projected/2ad38ce3-1483-4747-a9b3-7ccbfbb71957-kube-api-access-kpxh6\") pod \"ingress-canary-5ccx6\" (UID: \"2ad38ce3-1483-4747-a9b3-7ccbfbb71957\") " pod="openshift-ingress-canary/ingress-canary-5ccx6" Sep 18 03:33:41.573747 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.573710 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5ccx6" Sep 18 03:33:41.876686 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.876650 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:33:41.876964 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.876650 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:33:41.876964 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.876794 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:33:41.879777 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.879755 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jtr95\"" Sep 18 03:33:41.880003 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.879990 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 18 03:33:41.880102 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.880087 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 18 03:33:41.880167 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.880092 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 18 03:33:41.880637 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.880621 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 18 03:33:41.880894 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.880882 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6p5fh\"" Sep 18 03:33:41.901626 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.901605 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5ccx6"] Sep 18 03:33:41.905229 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:33:41.905206 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ad38ce3_1483_4747_a9b3_7ccbfbb71957.slice/crio-152e6fd39106a0cf305b928b406fa7d76f17c656d33ff2f49c63cde619574b12 WatchSource:0}: Error finding container 152e6fd39106a0cf305b928b406fa7d76f17c656d33ff2f49c63cde619574b12: Status 404 returned error can't find the container with id 152e6fd39106a0cf305b928b406fa7d76f17c656d33ff2f49c63cde619574b12 Sep 18 03:33:41.932017 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:41.931918 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls\") pod \"dns-default-dwxzz\" (UID: \"d5d06eb4-28cf-4325-8e91-19c6ee8b5b75\") " pod="openshift-dns/dns-default-dwxzz" Sep 18 03:33:41.932150 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:41.932047 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 03:33:41.932150 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:41.932095 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls podName:d5d06eb4-28cf-4325-8e91-19c6ee8b5b75 nodeName:}" failed. No retries permitted until 2026-09-18 03:33:42.932082781 +0000 UTC m=+37.214844635 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls") pod "dns-default-dwxzz" (UID: "d5d06eb4-28cf-4325-8e91-19c6ee8b5b75") : secret "dns-default-metrics-tls" not found Sep 18 03:33:42.178451 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:42.178411 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5ccx6" event={"ID":"2ad38ce3-1483-4747-a9b3-7ccbfbb71957","Type":"ContainerStarted","Data":"152e6fd39106a0cf305b928b406fa7d76f17c656d33ff2f49c63cde619574b12"} Sep 18 03:33:42.938764 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:42.938725 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls\") pod \"dns-default-dwxzz\" (UID: \"d5d06eb4-28cf-4325-8e91-19c6ee8b5b75\") " pod="openshift-dns/dns-default-dwxzz" Sep 18 03:33:42.939731 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:42.938891 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 03:33:42.939731 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:42.938993 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls podName:d5d06eb4-28cf-4325-8e91-19c6ee8b5b75 nodeName:}" failed. No retries permitted until 2026-09-18 03:33:44.938968728 +0000 UTC m=+39.221730578 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls") pod "dns-default-dwxzz" (UID: "d5d06eb4-28cf-4325-8e91-19c6ee8b5b75") : secret "dns-default-metrics-tls" not found Sep 18 03:33:44.954372 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:44.954194 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls\") pod \"dns-default-dwxzz\" (UID: \"d5d06eb4-28cf-4325-8e91-19c6ee8b5b75\") " pod="openshift-dns/dns-default-dwxzz" Sep 18 03:33:44.954706 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:44.954369 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 03:33:44.954706 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:44.954442 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls podName:d5d06eb4-28cf-4325-8e91-19c6ee8b5b75 nodeName:}" failed. No retries permitted until 2026-09-18 03:33:48.954421993 +0000 UTC m=+43.237183853 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls") pod "dns-default-dwxzz" (UID: "d5d06eb4-28cf-4325-8e91-19c6ee8b5b75") : secret "dns-default-metrics-tls" not found Sep 18 03:33:45.188871 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:45.188839 2568 generic.go:356] "Generic (PLEG): container finished" podID="afa7f816-ccc9-406c-bd67-55e4d933d144" containerID="e15a439c81a5232f1aac09aed12d3e12323da26a2e1d551045c65944e8a3563c" exitCode=0 Sep 18 03:33:45.189047 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:45.188918 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhzm9" event={"ID":"afa7f816-ccc9-406c-bd67-55e4d933d144","Type":"ContainerDied","Data":"e15a439c81a5232f1aac09aed12d3e12323da26a2e1d551045c65944e8a3563c"} Sep 18 03:33:45.190431 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:45.190410 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5ccx6" event={"ID":"2ad38ce3-1483-4747-a9b3-7ccbfbb71957","Type":"ContainerStarted","Data":"7b9711479c77df39eef43f38938f2c63d66b95a6a4acafe94ece35682af59a2e"} Sep 18 03:33:46.195621 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:46.195591 2568 generic.go:356] "Generic (PLEG): container finished" podID="afa7f816-ccc9-406c-bd67-55e4d933d144" containerID="0c2dd2d15b551d7d8e0907ca36697b4a283ba30ab7f87c43e1cc393089a49c3b" exitCode=0 Sep 18 03:33:46.196046 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:46.195670 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhzm9" event={"ID":"afa7f816-ccc9-406c-bd67-55e4d933d144","Type":"ContainerDied","Data":"0c2dd2d15b551d7d8e0907ca36697b4a283ba30ab7f87c43e1cc393089a49c3b"} Sep 18 03:33:46.215007 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:46.214963 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5ccx6" podStartSLOduration=2.26273394 podStartE2EDuration="5.214924176s" totalImagesPullingTime="2.952190236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:33:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:33:45.224553343 +0000 UTC m=+39.507315209" watchObservedRunningTime="2026-09-18 03:33:46.214924176 +0000 UTC m=+40.497686049" Sep 18 03:33:47.201424 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:47.201389 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhzm9" event={"ID":"afa7f816-ccc9-406c-bd67-55e4d933d144","Type":"ContainerStarted","Data":"fa197012046314ea36b77bc9d4669ddd49bcdbfd22d282347d37b167864979a6"} Sep 18 03:33:47.223921 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:47.222593 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fhzm9" podStartSLOduration=10.235817393 podStartE2EDuration="40.222573859s" totalImagesPullingTime="29.986756466s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 03:33:07 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:33:47.219773528 +0000 UTC m=+41.502535382" watchObservedRunningTime="2026-09-18 03:33:47.222573859 +0000 UTC m=+41.505335723" Sep 18 03:33:48.983217 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:48.983169 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls\") pod \"dns-default-dwxzz\" (UID: \"d5d06eb4-28cf-4325-8e91-19c6ee8b5b75\") " pod="openshift-dns/dns-default-dwxzz" Sep 18 03:33:48.983606 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:48.983318 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 03:33:48.983606 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:48.983385 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls podName:d5d06eb4-28cf-4325-8e91-19c6ee8b5b75 nodeName:}" failed. No retries permitted until 2026-09-18 03:33:56.983368377 +0000 UTC m=+51.266130221 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls") pod "dns-default-dwxzz" (UID: "d5d06eb4-28cf-4325-8e91-19c6ee8b5b75") : secret "dns-default-metrics-tls" not found Sep 18 03:33:57.036167 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:33:57.036128 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls\") pod \"dns-default-dwxzz\" (UID: \"d5d06eb4-28cf-4325-8e91-19c6ee8b5b75\") " pod="openshift-dns/dns-default-dwxzz" Sep 18 03:33:57.036561 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:57.036245 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 03:33:57.036561 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:33:57.036297 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls podName:d5d06eb4-28cf-4325-8e91-19c6ee8b5b75 nodeName:}" failed. No retries permitted until 2026-09-18 03:34:13.036283977 +0000 UTC m=+67.319045818 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls") pod "dns-default-dwxzz" (UID: "d5d06eb4-28cf-4325-8e91-19c6ee8b5b75") : secret "dns-default-metrics-tls" not found Sep 18 03:34:08.173834 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:34:08.173805 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-psksv" Sep 18 03:34:13.133803 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:34:13.133759 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls\") pod \"dns-default-dwxzz\" (UID: \"d5d06eb4-28cf-4325-8e91-19c6ee8b5b75\") " pod="openshift-dns/dns-default-dwxzz" Sep 18 03:34:13.134248 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:34:13.133879 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 03:34:13.134248 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:34:13.133944 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls podName:d5d06eb4-28cf-4325-8e91-19c6ee8b5b75 nodeName:}" failed. No retries permitted until 2026-09-18 03:34:45.133916336 +0000 UTC m=+99.416678177 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls") pod "dns-default-dwxzz" (UID: "d5d06eb4-28cf-4325-8e91-19c6ee8b5b75") : secret "dns-default-metrics-tls" not found Sep 18 03:34:13.535806 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:34:13.535762 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/34b777d9-80fc-4885-a29b-0b1f1e7d7415-original-pull-secret\") pod \"global-pull-secret-syncer-hv4gk\" (UID: \"34b777d9-80fc-4885-a29b-0b1f1e7d7415\") " pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:34:13.536102 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:34:13.535822 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs\") pod \"network-metrics-daemon-tf6sq\" (UID: \"9fdcb7aa-5f4c-4d16-a250-b272513204da\") " pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:34:13.536102 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:34:13.535975 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8khqd\" (UniqueName: \"kubernetes.io/projected/4437fdab-4af1-45cb-b944-acc7cbff5d20-kube-api-access-8khqd\") pod \"network-check-target-jbxqp\" (UID: \"4437fdab-4af1-45cb-b944-acc7cbff5d20\") " pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:34:13.538244 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:34:13.538226 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 18 03:34:13.538297 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:34:13.538266 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 18 03:34:13.538333 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:34:13.538304 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 18 03:34:13.545981 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:34:13.545965 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 18 03:34:13.546033 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:34:13.546012 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs podName:9fdcb7aa-5f4c-4d16-a250-b272513204da nodeName:}" failed. No retries permitted until 2026-09-18 03:35:17.545998956 +0000 UTC m=+131.828760801 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs") pod "network-metrics-daemon-tf6sq" (UID: "9fdcb7aa-5f4c-4d16-a250-b272513204da") : secret "metrics-daemon-secret" not found Sep 18 03:34:13.547989 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:34:13.547971 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 18 03:34:13.548964 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:34:13.548945 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/34b777d9-80fc-4885-a29b-0b1f1e7d7415-original-pull-secret\") pod \"global-pull-secret-syncer-hv4gk\" (UID: \"34b777d9-80fc-4885-a29b-0b1f1e7d7415\") " pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:34:13.558423 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:34:13.558405 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8khqd\" (UniqueName: \"kubernetes.io/projected/4437fdab-4af1-45cb-b944-acc7cbff5d20-kube-api-access-8khqd\") pod \"network-check-target-jbxqp\" (UID: \"4437fdab-4af1-45cb-b944-acc7cbff5d20\") " pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:34:13.688709 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:34:13.688678 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hv4gk" Sep 18 03:34:13.696756 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:34:13.696727 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jtr95\"" Sep 18 03:34:13.704657 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:34:13.704633 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:34:13.956746 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:34:13.956716 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hv4gk"] Sep 18 03:34:13.960460 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:34:13.960418 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34b777d9_80fc_4885_a29b_0b1f1e7d7415.slice/crio-037ded71961563e1e64c1c25a6a7e9e789f69f6107b03b36a7142b1f6ed82198 WatchSource:0}: Error finding container 037ded71961563e1e64c1c25a6a7e9e789f69f6107b03b36a7142b1f6ed82198: Status 404 returned error can't find the container with id 037ded71961563e1e64c1c25a6a7e9e789f69f6107b03b36a7142b1f6ed82198 Sep 18 03:34:13.976819 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:34:13.976794 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jbxqp"] Sep 18 03:34:13.979501 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:34:13.979482 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4437fdab_4af1_45cb_b944_acc7cbff5d20.slice/crio-69b0ec0f8651f9ebcfd7ed42162f930fad1b59306507fa518f2eae991416f0b4 WatchSource:0}: Error finding container 69b0ec0f8651f9ebcfd7ed42162f930fad1b59306507fa518f2eae991416f0b4: Status 404 returned error can't find the container with id 69b0ec0f8651f9ebcfd7ed42162f930fad1b59306507fa518f2eae991416f0b4 Sep 18 03:34:14.265526 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:34:14.265438 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jbxqp" event={"ID":"4437fdab-4af1-45cb-b944-acc7cbff5d20","Type":"ContainerStarted","Data":"69b0ec0f8651f9ebcfd7ed42162f930fad1b59306507fa518f2eae991416f0b4"} Sep 18 03:34:14.266578 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:34:14.266558 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hv4gk" event={"ID":"34b777d9-80fc-4885-a29b-0b1f1e7d7415","Type":"ContainerStarted","Data":"037ded71961563e1e64c1c25a6a7e9e789f69f6107b03b36a7142b1f6ed82198"} Sep 18 03:34:19.282302 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:34:19.282261 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jbxqp" event={"ID":"4437fdab-4af1-45cb-b944-acc7cbff5d20","Type":"ContainerStarted","Data":"ef8cb6e53b3820072f3af7cd9eb3ae7660c31b538b614c6a7f242420e43e6e75"} Sep 18 03:34:19.282834 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:34:19.282348 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:34:19.283802 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:34:19.283776 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hv4gk" event={"ID":"34b777d9-80fc-4885-a29b-0b1f1e7d7415","Type":"ContainerStarted","Data":"1adeb953954dc10a0fbcab8f7d748feb5dc14f249fa46aab99c45e464e0e413b"} Sep 18 03:34:19.297371 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:34:19.297328 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jbxqp" podStartSLOduration=67.82581441 podStartE2EDuration="1m12.297315969s" totalImagesPullingTime="4.471501559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:34:19.296296822 +0000 UTC m=+73.579058684" watchObservedRunningTime="2026-09-18 03:34:19.297315969 +0000 UTC m=+73.580077832" Sep 18 03:34:19.310027 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:34:19.309987 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hv4gk" podStartSLOduration=66.816001625 podStartE2EDuration="1m11.309972729s" totalImagesPullingTime="4.493971104s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 03:33:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:34:19.30920604 +0000 UTC m=+73.591967903" watchObservedRunningTime="2026-09-18 03:34:19.309972729 +0000 UTC m=+73.592734570" Sep 18 03:34:45.139792 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:34:45.139758 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls\") pod \"dns-default-dwxzz\" (UID: \"d5d06eb4-28cf-4325-8e91-19c6ee8b5b75\") " pod="openshift-dns/dns-default-dwxzz" Sep 18 03:34:45.140284 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:34:45.139883 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 03:34:45.140284 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:34:45.139957 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls podName:d5d06eb4-28cf-4325-8e91-19c6ee8b5b75 nodeName:}" failed. No retries permitted until 2026-09-18 03:35:49.139923931 +0000 UTC m=+163.422685773 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls") pod "dns-default-dwxzz" (UID: "d5d06eb4-28cf-4325-8e91-19c6ee8b5b75") : secret "dns-default-metrics-tls" not found Sep 18 03:34:50.288213 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:34:50.288181 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jbxqp" Sep 18 03:35:17.550215 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:35:17.550179 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs\") pod \"network-metrics-daemon-tf6sq\" (UID: \"9fdcb7aa-5f4c-4d16-a250-b272513204da\") " pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:35:17.550614 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:35:17.550301 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 18 03:35:17.550614 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:35:17.550371 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs podName:9fdcb7aa-5f4c-4d16-a250-b272513204da nodeName:}" failed. No retries permitted until 2026-09-18 03:37:19.550356789 +0000 UTC m=+253.833118630 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs") pod "network-metrics-daemon-tf6sq" (UID: "9fdcb7aa-5f4c-4d16-a250-b272513204da") : secret "metrics-daemon-secret" not found Sep 18 03:35:44.262011 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:35:44.261967 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-dwxzz" podUID="d5d06eb4-28cf-4325-8e91-19c6ee8b5b75" Sep 18 03:35:44.481503 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:35:44.481471 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dwxzz" Sep 18 03:35:44.898770 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:35:44.898734 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-tf6sq" podUID="9fdcb7aa-5f4c-4d16-a250-b272513204da" Sep 18 03:35:49.148882 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:35:49.148841 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls\") pod \"dns-default-dwxzz\" (UID: \"d5d06eb4-28cf-4325-8e91-19c6ee8b5b75\") " pod="openshift-dns/dns-default-dwxzz" Sep 18 03:35:49.149295 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:35:49.148976 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 03:35:49.149295 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:35:49.149026 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls podName:d5d06eb4-28cf-4325-8e91-19c6ee8b5b75 nodeName:}" failed. No retries permitted until 2026-09-18 03:37:51.149014279 +0000 UTC m=+285.431776131 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls") pod "dns-default-dwxzz" (UID: "d5d06eb4-28cf-4325-8e91-19c6ee8b5b75") : secret "dns-default-metrics-tls" not found Sep 18 03:35:55.876882 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:35:55.876795 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:36:43.561492 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.561457 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-bz22t"] Sep 18 03:36:43.563247 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.563231 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-bz22t" Sep 18 03:36:43.565928 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.565899 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-qq8kb"] Sep 18 03:36:43.566165 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.566002 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Sep 18 03:36:43.566490 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.566459 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Sep 18 03:36:43.566490 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.566465 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Sep 18 03:36:43.566680 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.566576 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-xm89q\"" Sep 18 03:36:43.567746 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.567727 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-p6bc5"] Sep 18 03:36:43.567885 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.567867 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-qq8kb" Sep 18 03:36:43.569608 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.569589 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-vcm9j"] Sep 18 03:36:43.569725 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.569708 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-p6bc5" Sep 18 03:36:43.570525 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.570507 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 18 03:36:43.570619 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.570584 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-6m7cg\"" Sep 18 03:36:43.570769 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.570754 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 18 03:36:43.571680 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.571661 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-59cb5d4fd6-68pct"] Sep 18 03:36:43.571791 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.571774 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-vcm9j" Sep 18 03:36:43.572387 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.572370 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 18 03:36:43.574148 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.572836 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 18 03:36:43.574148 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.573198 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 18 03:36:43.574148 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.573218 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 18 03:36:43.574148 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.573281 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-v764r\"" Sep 18 03:36:43.574827 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.574803 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 18 03:36:43.574944 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.574832 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 18 03:36:43.576000 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.575110 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:36:43.576000 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.575571 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 18 03:36:43.576770 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.576750 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-thpm9\"" Sep 18 03:36:43.578633 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.578082 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 18 03:36:43.578633 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.578524 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 18 03:36:43.579458 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.579442 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 18 03:36:43.579629 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.579610 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 18 03:36:43.580056 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.580036 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 18 03:36:43.580146 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.580051 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 18 03:36:43.580146 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.580082 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xfxjc\"" Sep 18 03:36:43.580284 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.580179 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-bz22t"] Sep 18 03:36:43.580409 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.580395 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 18 03:36:43.581331 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.581311 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-qq8kb"] Sep 18 03:36:43.583622 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.583581 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-p6bc5"] Sep 18 03:36:43.583711 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.583648 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 18 03:36:43.586151 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.586129 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-59cb5d4fd6-68pct"] Sep 18 03:36:43.594213 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.594195 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-vcm9j"] Sep 18 03:36:43.602052 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.602022 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/afa0807e-f31c-4d12-ab3a-35e9be7066f5-snapshots\") pod \"insights-operator-6f9c644dd9-p6bc5\" (UID: \"afa0807e-f31c-4d12-ab3a-35e9be7066f5\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6bc5" Sep 18 03:36:43.602149 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.602066 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-stats-auth\") pod \"router-default-59cb5d4fd6-68pct\" (UID: \"a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95\") " pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:36:43.602149 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.602091 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khrq4\" (UniqueName: \"kubernetes.io/projected/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-kube-api-access-khrq4\") pod \"router-default-59cb5d4fd6-68pct\" (UID: \"a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95\") " pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:36:43.602259 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.602152 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/afa0807e-f31c-4d12-ab3a-35e9be7066f5-serving-cert\") pod \"insights-operator-6f9c644dd9-p6bc5\" (UID: \"afa0807e-f31c-4d12-ab3a-35e9be7066f5\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6bc5" Sep 18 03:36:43.602259 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.602199 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afa0807e-f31c-4d12-ab3a-35e9be7066f5-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-p6bc5\" (UID: \"afa0807e-f31c-4d12-ab3a-35e9be7066f5\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6bc5" Sep 18 03:36:43.602259 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.602224 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ff4a6710-b06e-4c27-b451-aed9164d0ffd-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-vcm9j\" (UID: \"ff4a6710-b06e-4c27-b451-aed9164d0ffd\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-vcm9j" Sep 18 03:36:43.602259 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.602249 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff4a6710-b06e-4c27-b451-aed9164d0ffd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-vcm9j\" (UID: \"ff4a6710-b06e-4c27-b451-aed9164d0ffd\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-vcm9j" Sep 18 03:36:43.602458 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.602276 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7cjb\" (UniqueName: \"kubernetes.io/projected/c80472e0-3955-48d0-8029-34f8b355f4f3-kube-api-access-v7cjb\") pod \"volume-data-source-validator-5c4db8bdf9-qq8kb\" (UID: \"c80472e0-3955-48d0-8029-34f8b355f4f3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-qq8kb" Sep 18 03:36:43.602458 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.602299 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/afa0807e-f31c-4d12-ab3a-35e9be7066f5-tmp\") pod \"insights-operator-6f9c644dd9-p6bc5\" (UID: \"afa0807e-f31c-4d12-ab3a-35e9be7066f5\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6bc5" Sep 18 03:36:43.602458 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.602322 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-service-ca-bundle\") pod \"router-default-59cb5d4fd6-68pct\" (UID: \"a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95\") " pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:36:43.602458 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.602349 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvl7x\" (UniqueName: \"kubernetes.io/projected/afa0807e-f31c-4d12-ab3a-35e9be7066f5-kube-api-access-rvl7x\") pod \"insights-operator-6f9c644dd9-p6bc5\" (UID: \"afa0807e-f31c-4d12-ab3a-35e9be7066f5\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6bc5" Sep 18 03:36:43.602458 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.602379 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-default-certificate\") pod \"router-default-59cb5d4fd6-68pct\" (UID: \"a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95\") " pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:36:43.602458 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.602407 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/abce51bb-9c6c-4497-b966-7e03a2935245-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-bz22t\" (UID: \"abce51bb-9c6c-4497-b966-7e03a2935245\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-bz22t" Sep 18 03:36:43.602458 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.602435 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afa0807e-f31c-4d12-ab3a-35e9be7066f5-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-p6bc5\" (UID: \"afa0807e-f31c-4d12-ab3a-35e9be7066f5\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6bc5" Sep 18 03:36:43.602727 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.602458 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc49c\" (UniqueName: \"kubernetes.io/projected/ff4a6710-b06e-4c27-b451-aed9164d0ffd-kube-api-access-sc49c\") pod \"cluster-monitoring-operator-847b8bd8d7-vcm9j\" (UID: \"ff4a6710-b06e-4c27-b451-aed9164d0ffd\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-vcm9j" Sep 18 03:36:43.602727 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.602480 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-metrics-certs\") pod \"router-default-59cb5d4fd6-68pct\" (UID: \"a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95\") " pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:36:43.602727 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.602536 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kwm8\" (UniqueName: \"kubernetes.io/projected/abce51bb-9c6c-4497-b966-7e03a2935245-kube-api-access-2kwm8\") pod \"cluster-samples-operator-685d7d68b5-bz22t\" (UID: \"abce51bb-9c6c-4497-b966-7e03a2935245\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-bz22t" Sep 18 03:36:43.654972 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.654947 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-z8cfh"] Sep 18 03:36:43.657028 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.657014 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-z8cfh" Sep 18 03:36:43.659225 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.659199 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-l9nqd\"" Sep 18 03:36:43.668847 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.668829 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-z8cfh"] Sep 18 03:36:43.702733 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.702707 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kwm8\" (UniqueName: \"kubernetes.io/projected/abce51bb-9c6c-4497-b966-7e03a2935245-kube-api-access-2kwm8\") pod \"cluster-samples-operator-685d7d68b5-bz22t\" (UID: \"abce51bb-9c6c-4497-b966-7e03a2935245\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-bz22t" Sep 18 03:36:43.702855 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.702737 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/afa0807e-f31c-4d12-ab3a-35e9be7066f5-snapshots\") pod \"insights-operator-6f9c644dd9-p6bc5\" (UID: \"afa0807e-f31c-4d12-ab3a-35e9be7066f5\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6bc5" Sep 18 03:36:43.702855 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.702761 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-stats-auth\") pod \"router-default-59cb5d4fd6-68pct\" (UID: \"a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95\") " pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:36:43.702855 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.702784 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khrq4\" (UniqueName: \"kubernetes.io/projected/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-kube-api-access-khrq4\") pod \"router-default-59cb5d4fd6-68pct\" (UID: \"a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95\") " pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:36:43.702855 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.702816 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/afa0807e-f31c-4d12-ab3a-35e9be7066f5-serving-cert\") pod \"insights-operator-6f9c644dd9-p6bc5\" (UID: \"afa0807e-f31c-4d12-ab3a-35e9be7066f5\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6bc5" Sep 18 03:36:43.703087 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.702863 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afa0807e-f31c-4d12-ab3a-35e9be7066f5-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-p6bc5\" (UID: \"afa0807e-f31c-4d12-ab3a-35e9be7066f5\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6bc5" Sep 18 03:36:43.703087 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.702888 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ff4a6710-b06e-4c27-b451-aed9164d0ffd-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-vcm9j\" (UID: \"ff4a6710-b06e-4c27-b451-aed9164d0ffd\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-vcm9j" Sep 18 03:36:43.703087 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.702914 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff4a6710-b06e-4c27-b451-aed9164d0ffd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-vcm9j\" (UID: \"ff4a6710-b06e-4c27-b451-aed9164d0ffd\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-vcm9j" Sep 18 03:36:43.703087 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.702955 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7cjb\" (UniqueName: \"kubernetes.io/projected/c80472e0-3955-48d0-8029-34f8b355f4f3-kube-api-access-v7cjb\") pod \"volume-data-source-validator-5c4db8bdf9-qq8kb\" (UID: \"c80472e0-3955-48d0-8029-34f8b355f4f3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-qq8kb" Sep 18 03:36:43.703087 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.702978 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/afa0807e-f31c-4d12-ab3a-35e9be7066f5-tmp\") pod \"insights-operator-6f9c644dd9-p6bc5\" (UID: \"afa0807e-f31c-4d12-ab3a-35e9be7066f5\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6bc5" Sep 18 03:36:43.703087 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.703000 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-service-ca-bundle\") pod \"router-default-59cb5d4fd6-68pct\" (UID: \"a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95\") " pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:36:43.703087 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.703028 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvl7x\" (UniqueName: \"kubernetes.io/projected/afa0807e-f31c-4d12-ab3a-35e9be7066f5-kube-api-access-rvl7x\") pod \"insights-operator-6f9c644dd9-p6bc5\" (UID: \"afa0807e-f31c-4d12-ab3a-35e9be7066f5\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6bc5" Sep 18 03:36:43.703087 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.703065 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-default-certificate\") pod \"router-default-59cb5d4fd6-68pct\" (UID: \"a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95\") " pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:36:43.703487 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.703096 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wspc\" (UniqueName: \"kubernetes.io/projected/33ef8426-d8f4-4f6f-bff6-0c5bdd402f86-kube-api-access-6wspc\") pod \"network-check-source-7cbc45fddc-z8cfh\" (UID: \"33ef8426-d8f4-4f6f-bff6-0c5bdd402f86\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-z8cfh" Sep 18 03:36:43.703487 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.703126 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/abce51bb-9c6c-4497-b966-7e03a2935245-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-bz22t\" (UID: \"abce51bb-9c6c-4497-b966-7e03a2935245\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-bz22t" Sep 18 03:36:43.703487 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.703156 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afa0807e-f31c-4d12-ab3a-35e9be7066f5-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-p6bc5\" (UID: \"afa0807e-f31c-4d12-ab3a-35e9be7066f5\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6bc5" Sep 18 03:36:43.703487 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.703180 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc49c\" (UniqueName: \"kubernetes.io/projected/ff4a6710-b06e-4c27-b451-aed9164d0ffd-kube-api-access-sc49c\") pod \"cluster-monitoring-operator-847b8bd8d7-vcm9j\" (UID: \"ff4a6710-b06e-4c27-b451-aed9164d0ffd\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-vcm9j" Sep 18 03:36:43.703487 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.703204 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-metrics-certs\") pod \"router-default-59cb5d4fd6-68pct\" (UID: \"a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95\") " pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:36:43.703487 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:43.703326 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 18 03:36:43.703487 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.703361 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/afa0807e-f31c-4d12-ab3a-35e9be7066f5-tmp\") pod \"insights-operator-6f9c644dd9-p6bc5\" (UID: \"afa0807e-f31c-4d12-ab3a-35e9be7066f5\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6bc5" Sep 18 03:36:43.703487 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:43.703387 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-metrics-certs podName:a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95 nodeName:}" failed. No retries permitted until 2026-09-18 03:36:44.203370609 +0000 UTC m=+218.486132461 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-metrics-certs") pod "router-default-59cb5d4fd6-68pct" (UID: "a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95") : secret "router-metrics-certs-default" not found Sep 18 03:36:43.703487 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:43.703442 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 18 03:36:43.703487 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.703442 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/afa0807e-f31c-4d12-ab3a-35e9be7066f5-snapshots\") pod \"insights-operator-6f9c644dd9-p6bc5\" (UID: \"afa0807e-f31c-4d12-ab3a-35e9be7066f5\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6bc5" Sep 18 03:36:43.703487 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:43.703463 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-service-ca-bundle podName:a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95 nodeName:}" failed. No retries permitted until 2026-09-18 03:36:44.203450689 +0000 UTC m=+218.486212533 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-service-ca-bundle") pod "router-default-59cb5d4fd6-68pct" (UID: "a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95") : configmap references non-existent config key: service-ca.crt Sep 18 03:36:43.703487 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:43.703486 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff4a6710-b06e-4c27-b451-aed9164d0ffd-cluster-monitoring-operator-tls podName:ff4a6710-b06e-4c27-b451-aed9164d0ffd nodeName:}" failed. No retries permitted until 2026-09-18 03:36:44.203474779 +0000 UTC m=+218.486236635 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ff4a6710-b06e-4c27-b451-aed9164d0ffd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-vcm9j" (UID: "ff4a6710-b06e-4c27-b451-aed9164d0ffd") : secret "cluster-monitoring-operator-tls" not found Sep 18 03:36:43.704137 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:43.703543 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 18 03:36:43.704137 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:43.703573 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abce51bb-9c6c-4497-b966-7e03a2935245-samples-operator-tls podName:abce51bb-9c6c-4497-b966-7e03a2935245 nodeName:}" failed. No retries permitted until 2026-09-18 03:36:44.203563392 +0000 UTC m=+218.486325247 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/abce51bb-9c6c-4497-b966-7e03a2935245-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-bz22t" (UID: "abce51bb-9c6c-4497-b966-7e03a2935245") : secret "samples-operator-tls" not found Sep 18 03:36:43.704137 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.703696 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afa0807e-f31c-4d12-ab3a-35e9be7066f5-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-p6bc5\" (UID: \"afa0807e-f31c-4d12-ab3a-35e9be7066f5\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6bc5" Sep 18 03:36:43.704137 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.704072 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afa0807e-f31c-4d12-ab3a-35e9be7066f5-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-p6bc5\" (UID: \"afa0807e-f31c-4d12-ab3a-35e9be7066f5\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6bc5" Sep 18 03:36:43.704394 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.704372 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ff4a6710-b06e-4c27-b451-aed9164d0ffd-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-vcm9j\" (UID: \"ff4a6710-b06e-4c27-b451-aed9164d0ffd\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-vcm9j" Sep 18 03:36:43.705415 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.705390 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/afa0807e-f31c-4d12-ab3a-35e9be7066f5-serving-cert\") pod \"insights-operator-6f9c644dd9-p6bc5\" (UID: \"afa0807e-f31c-4d12-ab3a-35e9be7066f5\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6bc5" Sep 18 03:36:43.705492 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.705481 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-stats-auth\") pod \"router-default-59cb5d4fd6-68pct\" (UID: \"a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95\") " pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:36:43.705842 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.705826 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-default-certificate\") pod \"router-default-59cb5d4fd6-68pct\" (UID: \"a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95\") " pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:36:43.712865 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.712842 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvl7x\" (UniqueName: \"kubernetes.io/projected/afa0807e-f31c-4d12-ab3a-35e9be7066f5-kube-api-access-rvl7x\") pod \"insights-operator-6f9c644dd9-p6bc5\" (UID: \"afa0807e-f31c-4d12-ab3a-35e9be7066f5\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6bc5" Sep 18 03:36:43.712964 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.712915 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khrq4\" (UniqueName: \"kubernetes.io/projected/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-kube-api-access-khrq4\") pod \"router-default-59cb5d4fd6-68pct\" (UID: \"a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95\") " pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:36:43.713434 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.713416 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kwm8\" (UniqueName: \"kubernetes.io/projected/abce51bb-9c6c-4497-b966-7e03a2935245-kube-api-access-2kwm8\") pod \"cluster-samples-operator-685d7d68b5-bz22t\" (UID: \"abce51bb-9c6c-4497-b966-7e03a2935245\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-bz22t" Sep 18 03:36:43.715394 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.715373 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7cjb\" (UniqueName: \"kubernetes.io/projected/c80472e0-3955-48d0-8029-34f8b355f4f3-kube-api-access-v7cjb\") pod \"volume-data-source-validator-5c4db8bdf9-qq8kb\" (UID: \"c80472e0-3955-48d0-8029-34f8b355f4f3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-qq8kb" Sep 18 03:36:43.715632 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.715617 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc49c\" (UniqueName: \"kubernetes.io/projected/ff4a6710-b06e-4c27-b451-aed9164d0ffd-kube-api-access-sc49c\") pod \"cluster-monitoring-operator-847b8bd8d7-vcm9j\" (UID: \"ff4a6710-b06e-4c27-b451-aed9164d0ffd\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-vcm9j" Sep 18 03:36:43.804135 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.804088 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wspc\" (UniqueName: \"kubernetes.io/projected/33ef8426-d8f4-4f6f-bff6-0c5bdd402f86-kube-api-access-6wspc\") pod \"network-check-source-7cbc45fddc-z8cfh\" (UID: \"33ef8426-d8f4-4f6f-bff6-0c5bdd402f86\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-z8cfh" Sep 18 03:36:43.811956 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.811881 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wspc\" (UniqueName: \"kubernetes.io/projected/33ef8426-d8f4-4f6f-bff6-0c5bdd402f86-kube-api-access-6wspc\") pod \"network-check-source-7cbc45fddc-z8cfh\" (UID: \"33ef8426-d8f4-4f6f-bff6-0c5bdd402f86\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-z8cfh" Sep 18 03:36:43.883925 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.883903 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-qq8kb" Sep 18 03:36:43.891318 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.891297 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-p6bc5" Sep 18 03:36:43.964962 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:43.964925 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-z8cfh" Sep 18 03:36:44.150210 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:44.150187 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-qq8kb"] Sep 18 03:36:44.152307 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:36:44.152277 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc80472e0_3955_48d0_8029_34f8b355f4f3.slice/crio-39827c9df16e58ca2672f5710156fddcd46945a6e1244b5bbd0bb3f4ac2eb323 WatchSource:0}: Error finding container 39827c9df16e58ca2672f5710156fddcd46945a6e1244b5bbd0bb3f4ac2eb323: Status 404 returned error can't find the container with id 39827c9df16e58ca2672f5710156fddcd46945a6e1244b5bbd0bb3f4ac2eb323 Sep 18 03:36:44.169026 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:44.169003 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-p6bc5"] Sep 18 03:36:44.171586 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:36:44.171563 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafa0807e_f31c_4d12_ab3a_35e9be7066f5.slice/crio-760f5b017a4a41176c15efab7b81052737508a34a44b580ddbc398b4bc932726 WatchSource:0}: Error finding container 760f5b017a4a41176c15efab7b81052737508a34a44b580ddbc398b4bc932726: Status 404 returned error can't find the container with id 760f5b017a4a41176c15efab7b81052737508a34a44b580ddbc398b4bc932726 Sep 18 03:36:44.206292 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:44.206266 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff4a6710-b06e-4c27-b451-aed9164d0ffd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-vcm9j\" (UID: \"ff4a6710-b06e-4c27-b451-aed9164d0ffd\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-vcm9j" Sep 18 03:36:44.206390 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:44.206300 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-service-ca-bundle\") pod \"router-default-59cb5d4fd6-68pct\" (UID: \"a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95\") " pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:36:44.206390 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:44.206329 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/abce51bb-9c6c-4497-b966-7e03a2935245-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-bz22t\" (UID: \"abce51bb-9c6c-4497-b966-7e03a2935245\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-bz22t" Sep 18 03:36:44.206485 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:44.206401 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 18 03:36:44.206485 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:44.206414 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 18 03:36:44.206485 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:44.206418 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-service-ca-bundle podName:a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95 nodeName:}" failed. No retries permitted until 2026-09-18 03:36:45.206403469 +0000 UTC m=+219.489165310 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-service-ca-bundle") pod "router-default-59cb5d4fd6-68pct" (UID: "a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95") : configmap references non-existent config key: service-ca.crt Sep 18 03:36:44.206485 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:44.206471 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-metrics-certs\") pod \"router-default-59cb5d4fd6-68pct\" (UID: \"a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95\") " pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:36:44.206621 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:44.206530 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 18 03:36:44.206621 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:44.206561 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff4a6710-b06e-4c27-b451-aed9164d0ffd-cluster-monitoring-operator-tls podName:ff4a6710-b06e-4c27-b451-aed9164d0ffd nodeName:}" failed. No retries permitted until 2026-09-18 03:36:45.206547568 +0000 UTC m=+219.489309411 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ff4a6710-b06e-4c27-b451-aed9164d0ffd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-vcm9j" (UID: "ff4a6710-b06e-4c27-b451-aed9164d0ffd") : secret "cluster-monitoring-operator-tls" not found Sep 18 03:36:44.206621 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:44.206585 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abce51bb-9c6c-4497-b966-7e03a2935245-samples-operator-tls podName:abce51bb-9c6c-4497-b966-7e03a2935245 nodeName:}" failed. No retries permitted until 2026-09-18 03:36:45.206575591 +0000 UTC m=+219.489337436 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/abce51bb-9c6c-4497-b966-7e03a2935245-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-bz22t" (UID: "abce51bb-9c6c-4497-b966-7e03a2935245") : secret "samples-operator-tls" not found Sep 18 03:36:44.206621 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:44.206604 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-metrics-certs podName:a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95 nodeName:}" failed. No retries permitted until 2026-09-18 03:36:45.206594693 +0000 UTC m=+219.489356535 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-metrics-certs") pod "router-default-59cb5d4fd6-68pct" (UID: "a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95") : secret "router-metrics-certs-default" not found Sep 18 03:36:44.230151 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:44.230128 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-z8cfh"] Sep 18 03:36:44.232580 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:36:44.232554 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33ef8426_d8f4_4f6f_bff6_0c5bdd402f86.slice/crio-a88215293e992defe92fa2459b33d9aa17be5a82f0305d835d0e176af10f04ea WatchSource:0}: Error finding container a88215293e992defe92fa2459b33d9aa17be5a82f0305d835d0e176af10f04ea: Status 404 returned error can't find the container with id a88215293e992defe92fa2459b33d9aa17be5a82f0305d835d0e176af10f04ea Sep 18 03:36:44.626336 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:44.626244 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-qq8kb" event={"ID":"c80472e0-3955-48d0-8029-34f8b355f4f3","Type":"ContainerStarted","Data":"39827c9df16e58ca2672f5710156fddcd46945a6e1244b5bbd0bb3f4ac2eb323"} Sep 18 03:36:44.627399 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:44.627371 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-p6bc5" event={"ID":"afa0807e-f31c-4d12-ab3a-35e9be7066f5","Type":"ContainerStarted","Data":"760f5b017a4a41176c15efab7b81052737508a34a44b580ddbc398b4bc932726"} Sep 18 03:36:44.628799 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:44.628776 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-z8cfh" event={"ID":"33ef8426-d8f4-4f6f-bff6-0c5bdd402f86","Type":"ContainerStarted","Data":"45255f4b98c6ba158569581257ba0544b3a927f97581d9d154a8cd64e9399ca8"} Sep 18 03:36:44.628899 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:44.628811 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-z8cfh" event={"ID":"33ef8426-d8f4-4f6f-bff6-0c5bdd402f86","Type":"ContainerStarted","Data":"a88215293e992defe92fa2459b33d9aa17be5a82f0305d835d0e176af10f04ea"} Sep 18 03:36:44.642565 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:44.642506 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-z8cfh" podStartSLOduration=1.6424888640000002 podStartE2EDuration="1.642488864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:36:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:36:44.641588525 +0000 UTC m=+218.924350388" watchObservedRunningTime="2026-09-18 03:36:44.642488864 +0000 UTC m=+218.925250728" Sep 18 03:36:45.215033 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:45.215000 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff4a6710-b06e-4c27-b451-aed9164d0ffd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-vcm9j\" (UID: \"ff4a6710-b06e-4c27-b451-aed9164d0ffd\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-vcm9j" Sep 18 03:36:45.215224 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:45.215045 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-service-ca-bundle\") pod \"router-default-59cb5d4fd6-68pct\" (UID: \"a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95\") " pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:36:45.215224 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:45.215084 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/abce51bb-9c6c-4497-b966-7e03a2935245-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-bz22t\" (UID: \"abce51bb-9c6c-4497-b966-7e03a2935245\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-bz22t" Sep 18 03:36:45.215224 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:45.215108 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-metrics-certs\") pod \"router-default-59cb5d4fd6-68pct\" (UID: \"a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95\") " pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:36:45.215224 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:45.215164 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 18 03:36:45.215224 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:45.215225 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 18 03:36:45.215469 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:45.215238 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff4a6710-b06e-4c27-b451-aed9164d0ffd-cluster-monitoring-operator-tls podName:ff4a6710-b06e-4c27-b451-aed9164d0ffd nodeName:}" failed. No retries permitted until 2026-09-18 03:36:47.215217754 +0000 UTC m=+221.497979600 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ff4a6710-b06e-4c27-b451-aed9164d0ffd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-vcm9j" (UID: "ff4a6710-b06e-4c27-b451-aed9164d0ffd") : secret "cluster-monitoring-operator-tls" not found Sep 18 03:36:45.215469 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:45.215256 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-service-ca-bundle podName:a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95 nodeName:}" failed. No retries permitted until 2026-09-18 03:36:47.215247414 +0000 UTC m=+221.498009260 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-service-ca-bundle") pod "router-default-59cb5d4fd6-68pct" (UID: "a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95") : configmap references non-existent config key: service-ca.crt Sep 18 03:36:45.215469 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:45.215274 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 18 03:36:45.215469 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:45.215277 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-metrics-certs podName:a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95 nodeName:}" failed. No retries permitted until 2026-09-18 03:36:47.215261989 +0000 UTC m=+221.498023844 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-metrics-certs") pod "router-default-59cb5d4fd6-68pct" (UID: "a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95") : secret "router-metrics-certs-default" not found Sep 18 03:36:45.215469 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:45.215330 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abce51bb-9c6c-4497-b966-7e03a2935245-samples-operator-tls podName:abce51bb-9c6c-4497-b966-7e03a2935245 nodeName:}" failed. No retries permitted until 2026-09-18 03:36:47.215315166 +0000 UTC m=+221.498077008 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/abce51bb-9c6c-4497-b966-7e03a2935245-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-bz22t" (UID: "abce51bb-9c6c-4497-b966-7e03a2935245") : secret "samples-operator-tls" not found Sep 18 03:36:45.633788 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:45.633710 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-qq8kb" event={"ID":"c80472e0-3955-48d0-8029-34f8b355f4f3","Type":"ContainerStarted","Data":"2e197e84a3d3bf2aaa2b35621348805792f1a744d30763499b58e2a0a9dd33f5"} Sep 18 03:36:45.647878 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:45.647824 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-qq8kb" podStartSLOduration=1.549940319 podStartE2EDuration="2.64779593s" totalImagesPullingTime="1.097855611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:36:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:36:45.646894196 +0000 UTC m=+219.929656070" watchObservedRunningTime="2026-09-18 03:36:45.64779593 +0000 UTC m=+219.930557790" Sep 18 03:36:46.638668 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:46.638569 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-p6bc5" event={"ID":"afa0807e-f31c-4d12-ab3a-35e9be7066f5","Type":"ContainerStarted","Data":"2318418607f90dbffc72ff808dd2ef5e4a6e6f6a8fd8bcd4498487424a46beb6"} Sep 18 03:36:46.654354 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:46.654299 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-6f9c644dd9-p6bc5" podStartSLOduration=1.559185899 podStartE2EDuration="3.654279072s" totalImagesPullingTime="2.095093173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:36:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:36:46.653365039 +0000 UTC m=+220.936126905" watchObservedRunningTime="2026-09-18 03:36:46.654279072 +0000 UTC m=+220.937040916" Sep 18 03:36:47.228204 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:47.228163 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff4a6710-b06e-4c27-b451-aed9164d0ffd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-vcm9j\" (UID: \"ff4a6710-b06e-4c27-b451-aed9164d0ffd\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-vcm9j" Sep 18 03:36:47.228380 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:47.228206 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-service-ca-bundle\") pod \"router-default-59cb5d4fd6-68pct\" (UID: \"a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95\") " pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:36:47.228380 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:47.228249 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/abce51bb-9c6c-4497-b966-7e03a2935245-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-bz22t\" (UID: \"abce51bb-9c6c-4497-b966-7e03a2935245\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-bz22t" Sep 18 03:36:47.228380 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:47.228277 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-metrics-certs\") pod \"router-default-59cb5d4fd6-68pct\" (UID: \"a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95\") " pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:36:47.228380 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:47.228323 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 18 03:36:47.228380 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:47.228354 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-service-ca-bundle podName:a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95 nodeName:}" failed. No retries permitted until 2026-09-18 03:36:51.228334864 +0000 UTC m=+225.511096706 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-service-ca-bundle") pod "router-default-59cb5d4fd6-68pct" (UID: "a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95") : configmap references non-existent config key: service-ca.crt Sep 18 03:36:47.228380 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:47.228382 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 18 03:36:47.228589 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:47.228403 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 18 03:36:47.228589 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:47.228382 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff4a6710-b06e-4c27-b451-aed9164d0ffd-cluster-monitoring-operator-tls podName:ff4a6710-b06e-4c27-b451-aed9164d0ffd nodeName:}" failed. No retries permitted until 2026-09-18 03:36:51.228369125 +0000 UTC m=+225.511130966 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ff4a6710-b06e-4c27-b451-aed9164d0ffd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-vcm9j" (UID: "ff4a6710-b06e-4c27-b451-aed9164d0ffd") : secret "cluster-monitoring-operator-tls" not found Sep 18 03:36:47.228589 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:47.228456 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-metrics-certs podName:a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95 nodeName:}" failed. No retries permitted until 2026-09-18 03:36:51.228444961 +0000 UTC m=+225.511206802 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-metrics-certs") pod "router-default-59cb5d4fd6-68pct" (UID: "a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95") : secret "router-metrics-certs-default" not found Sep 18 03:36:47.228589 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:47.228473 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abce51bb-9c6c-4497-b966-7e03a2935245-samples-operator-tls podName:abce51bb-9c6c-4497-b966-7e03a2935245 nodeName:}" failed. No retries permitted until 2026-09-18 03:36:51.228466179 +0000 UTC m=+225.511228020 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/abce51bb-9c6c-4497-b966-7e03a2935245-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-bz22t" (UID: "abce51bb-9c6c-4497-b966-7e03a2935245") : secret "samples-operator-tls" not found Sep 18 03:36:47.306879 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:47.306846 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-m5g7q"] Sep 18 03:36:47.308902 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:47.308883 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m5g7q" Sep 18 03:36:47.311324 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:47.311295 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 18 03:36:47.311421 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:47.311324 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 18 03:36:47.311421 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:47.311407 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tg5sj\"" Sep 18 03:36:47.323656 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:47.323629 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m5g7q"] Sep 18 03:36:47.328928 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:47.328907 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b99b8fd-1388-4e40-b69a-fcc3d901a224-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m5g7q\" (UID: \"4b99b8fd-1388-4e40-b69a-fcc3d901a224\") " pod="openshift-insights/insights-runtime-extractor-m5g7q" Sep 18 03:36:47.329040 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:47.328962 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b99b8fd-1388-4e40-b69a-fcc3d901a224-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m5g7q\" (UID: \"4b99b8fd-1388-4e40-b69a-fcc3d901a224\") " pod="openshift-insights/insights-runtime-extractor-m5g7q" Sep 18 03:36:47.329040 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:47.329024 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4b99b8fd-1388-4e40-b69a-fcc3d901a224-crio-socket\") pod \"insights-runtime-extractor-m5g7q\" (UID: \"4b99b8fd-1388-4e40-b69a-fcc3d901a224\") " pod="openshift-insights/insights-runtime-extractor-m5g7q" Sep 18 03:36:47.329153 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:47.329080 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b99b8fd-1388-4e40-b69a-fcc3d901a224-data-volume\") pod \"insights-runtime-extractor-m5g7q\" (UID: \"4b99b8fd-1388-4e40-b69a-fcc3d901a224\") " pod="openshift-insights/insights-runtime-extractor-m5g7q" Sep 18 03:36:47.329153 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:47.329116 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jnwc\" (UniqueName: \"kubernetes.io/projected/4b99b8fd-1388-4e40-b69a-fcc3d901a224-kube-api-access-8jnwc\") pod \"insights-runtime-extractor-m5g7q\" (UID: \"4b99b8fd-1388-4e40-b69a-fcc3d901a224\") " pod="openshift-insights/insights-runtime-extractor-m5g7q" Sep 18 03:36:47.430402 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:47.430354 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4b99b8fd-1388-4e40-b69a-fcc3d901a224-crio-socket\") pod \"insights-runtime-extractor-m5g7q\" (UID: \"4b99b8fd-1388-4e40-b69a-fcc3d901a224\") " pod="openshift-insights/insights-runtime-extractor-m5g7q" Sep 18 03:36:47.430402 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:47.430407 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b99b8fd-1388-4e40-b69a-fcc3d901a224-data-volume\") pod \"insights-runtime-extractor-m5g7q\" (UID: \"4b99b8fd-1388-4e40-b69a-fcc3d901a224\") " pod="openshift-insights/insights-runtime-extractor-m5g7q" Sep 18 03:36:47.430669 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:47.430480 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4b99b8fd-1388-4e40-b69a-fcc3d901a224-crio-socket\") pod \"insights-runtime-extractor-m5g7q\" (UID: \"4b99b8fd-1388-4e40-b69a-fcc3d901a224\") " pod="openshift-insights/insights-runtime-extractor-m5g7q" Sep 18 03:36:47.430669 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:47.430509 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jnwc\" (UniqueName: \"kubernetes.io/projected/4b99b8fd-1388-4e40-b69a-fcc3d901a224-kube-api-access-8jnwc\") pod \"insights-runtime-extractor-m5g7q\" (UID: \"4b99b8fd-1388-4e40-b69a-fcc3d901a224\") " pod="openshift-insights/insights-runtime-extractor-m5g7q" Sep 18 03:36:47.430669 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:47.430569 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b99b8fd-1388-4e40-b69a-fcc3d901a224-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m5g7q\" (UID: \"4b99b8fd-1388-4e40-b69a-fcc3d901a224\") " pod="openshift-insights/insights-runtime-extractor-m5g7q" Sep 18 03:36:47.430669 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:47.430596 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b99b8fd-1388-4e40-b69a-fcc3d901a224-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m5g7q\" (UID: \"4b99b8fd-1388-4e40-b69a-fcc3d901a224\") " pod="openshift-insights/insights-runtime-extractor-m5g7q" Sep 18 03:36:47.430857 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:47.430689 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 18 03:36:47.430857 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:47.430722 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 18 03:36:47.430857 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:47.430722 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b99b8fd-1388-4e40-b69a-fcc3d901a224-data-volume\") pod \"insights-runtime-extractor-m5g7q\" (UID: \"4b99b8fd-1388-4e40-b69a-fcc3d901a224\") " pod="openshift-insights/insights-runtime-extractor-m5g7q" Sep 18 03:36:47.430857 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:47.430758 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b99b8fd-1388-4e40-b69a-fcc3d901a224-kube-rbac-proxy-cm podName:4b99b8fd-1388-4e40-b69a-fcc3d901a224 nodeName:}" failed. No retries permitted until 2026-09-18 03:36:47.930741072 +0000 UTC m=+222.213502931 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4b99b8fd-1388-4e40-b69a-fcc3d901a224-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m5g7q" (UID: "4b99b8fd-1388-4e40-b69a-fcc3d901a224") : configmap "kube-rbac-proxy" not found Sep 18 03:36:47.430857 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:47.430773 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b99b8fd-1388-4e40-b69a-fcc3d901a224-insights-runtime-extractor-tls podName:4b99b8fd-1388-4e40-b69a-fcc3d901a224 nodeName:}" failed. No retries permitted until 2026-09-18 03:36:47.930766552 +0000 UTC m=+222.213528393 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4b99b8fd-1388-4e40-b69a-fcc3d901a224-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m5g7q" (UID: "4b99b8fd-1388-4e40-b69a-fcc3d901a224") : secret "insights-runtime-extractor-tls" not found Sep 18 03:36:47.438669 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:47.438648 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jnwc\" (UniqueName: \"kubernetes.io/projected/4b99b8fd-1388-4e40-b69a-fcc3d901a224-kube-api-access-8jnwc\") pod \"insights-runtime-extractor-m5g7q\" (UID: \"4b99b8fd-1388-4e40-b69a-fcc3d901a224\") " pod="openshift-insights/insights-runtime-extractor-m5g7q" Sep 18 03:36:47.933820 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:47.933790 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b99b8fd-1388-4e40-b69a-fcc3d901a224-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m5g7q\" (UID: \"4b99b8fd-1388-4e40-b69a-fcc3d901a224\") " pod="openshift-insights/insights-runtime-extractor-m5g7q" Sep 18 03:36:47.934295 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:47.933961 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 18 03:36:47.934295 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:47.933986 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b99b8fd-1388-4e40-b69a-fcc3d901a224-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m5g7q\" (UID: \"4b99b8fd-1388-4e40-b69a-fcc3d901a224\") " pod="openshift-insights/insights-runtime-extractor-m5g7q" Sep 18 03:36:47.934295 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:47.934025 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b99b8fd-1388-4e40-b69a-fcc3d901a224-insights-runtime-extractor-tls podName:4b99b8fd-1388-4e40-b69a-fcc3d901a224 nodeName:}" failed. No retries permitted until 2026-09-18 03:36:48.934006223 +0000 UTC m=+223.216768079 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4b99b8fd-1388-4e40-b69a-fcc3d901a224-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m5g7q" (UID: "4b99b8fd-1388-4e40-b69a-fcc3d901a224") : secret "insights-runtime-extractor-tls" not found Sep 18 03:36:47.934295 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:47.934061 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 18 03:36:47.934295 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:47.934110 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b99b8fd-1388-4e40-b69a-fcc3d901a224-kube-rbac-proxy-cm podName:4b99b8fd-1388-4e40-b69a-fcc3d901a224 nodeName:}" failed. No retries permitted until 2026-09-18 03:36:48.934098444 +0000 UTC m=+223.216860286 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4b99b8fd-1388-4e40-b69a-fcc3d901a224-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m5g7q" (UID: "4b99b8fd-1388-4e40-b69a-fcc3d901a224") : configmap "kube-rbac-proxy" not found Sep 18 03:36:48.941566 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:48.941525 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b99b8fd-1388-4e40-b69a-fcc3d901a224-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m5g7q\" (UID: \"4b99b8fd-1388-4e40-b69a-fcc3d901a224\") " pod="openshift-insights/insights-runtime-extractor-m5g7q" Sep 18 03:36:48.942015 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:48.941589 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b99b8fd-1388-4e40-b69a-fcc3d901a224-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m5g7q\" (UID: \"4b99b8fd-1388-4e40-b69a-fcc3d901a224\") " pod="openshift-insights/insights-runtime-extractor-m5g7q" Sep 18 03:36:48.942015 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:48.941678 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 18 03:36:48.942015 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:48.941731 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 18 03:36:48.942015 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:48.941754 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b99b8fd-1388-4e40-b69a-fcc3d901a224-kube-rbac-proxy-cm podName:4b99b8fd-1388-4e40-b69a-fcc3d901a224 nodeName:}" failed. No retries permitted until 2026-09-18 03:36:50.941731761 +0000 UTC m=+225.224493602 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4b99b8fd-1388-4e40-b69a-fcc3d901a224-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m5g7q" (UID: "4b99b8fd-1388-4e40-b69a-fcc3d901a224") : configmap "kube-rbac-proxy" not found Sep 18 03:36:48.942015 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:48.941788 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b99b8fd-1388-4e40-b69a-fcc3d901a224-insights-runtime-extractor-tls podName:4b99b8fd-1388-4e40-b69a-fcc3d901a224 nodeName:}" failed. No retries permitted until 2026-09-18 03:36:50.941771147 +0000 UTC m=+225.224532995 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4b99b8fd-1388-4e40-b69a-fcc3d901a224-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m5g7q" (UID: "4b99b8fd-1388-4e40-b69a-fcc3d901a224") : secret "insights-runtime-extractor-tls" not found Sep 18 03:36:48.993682 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:48.993654 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-5qbkt"] Sep 18 03:36:48.995679 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:48.995659 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-5qbkt" Sep 18 03:36:48.999217 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:48.999180 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Sep 18 03:36:49.000147 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:49.000132 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-mr28l\"" Sep 18 03:36:49.003703 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:49.003685 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Sep 18 03:36:49.018778 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:49.018755 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-5qbkt"] Sep 18 03:36:49.042226 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:49.042196 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4sbv\" (UniqueName: \"kubernetes.io/projected/44c39f94-93e8-46a5-aa1c-0d16cc984015-kube-api-access-r4sbv\") pod \"migrator-76794cbd6c-5qbkt\" (UID: \"44c39f94-93e8-46a5-aa1c-0d16cc984015\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-5qbkt" Sep 18 03:36:49.063249 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:49.063227 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-87l8w_fcf69b36-5d73-458d-9324-6da53a9f1c73/dns-node-resolver/0.log" Sep 18 03:36:49.142778 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:49.142753 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4sbv\" (UniqueName: \"kubernetes.io/projected/44c39f94-93e8-46a5-aa1c-0d16cc984015-kube-api-access-r4sbv\") pod \"migrator-76794cbd6c-5qbkt\" (UID: \"44c39f94-93e8-46a5-aa1c-0d16cc984015\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-5qbkt" Sep 18 03:36:49.150455 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:49.150430 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4sbv\" (UniqueName: \"kubernetes.io/projected/44c39f94-93e8-46a5-aa1c-0d16cc984015-kube-api-access-r4sbv\") pod \"migrator-76794cbd6c-5qbkt\" (UID: \"44c39f94-93e8-46a5-aa1c-0d16cc984015\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-5qbkt" Sep 18 03:36:49.305567 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:49.305483 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-5qbkt" Sep 18 03:36:49.568521 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:49.568443 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-5qbkt"] Sep 18 03:36:49.571421 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:36:49.571394 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44c39f94_93e8_46a5_aa1c_0d16cc984015.slice/crio-c2a8be1af08acea367c13150e51c8d23e75fedf215fb35b7c41d65fb9e8fee7d WatchSource:0}: Error finding container c2a8be1af08acea367c13150e51c8d23e75fedf215fb35b7c41d65fb9e8fee7d: Status 404 returned error can't find the container with id c2a8be1af08acea367c13150e51c8d23e75fedf215fb35b7c41d65fb9e8fee7d Sep 18 03:36:49.650191 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:49.650149 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-5qbkt" event={"ID":"44c39f94-93e8-46a5-aa1c-0d16cc984015","Type":"ContainerStarted","Data":"c2a8be1af08acea367c13150e51c8d23e75fedf215fb35b7c41d65fb9e8fee7d"} Sep 18 03:36:50.063847 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:50.063816 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-54ct2_d5694211-42dd-4f1c-984b-fc2a748e8759/node-ca/0.log" Sep 18 03:36:50.960038 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:50.960012 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b99b8fd-1388-4e40-b69a-fcc3d901a224-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m5g7q\" (UID: \"4b99b8fd-1388-4e40-b69a-fcc3d901a224\") " pod="openshift-insights/insights-runtime-extractor-m5g7q" Sep 18 03:36:50.960186 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:50.960055 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b99b8fd-1388-4e40-b69a-fcc3d901a224-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m5g7q\" (UID: \"4b99b8fd-1388-4e40-b69a-fcc3d901a224\") " pod="openshift-insights/insights-runtime-extractor-m5g7q" Sep 18 03:36:50.960186 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:50.960158 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 18 03:36:50.960298 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:50.960193 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 18 03:36:50.960298 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:50.960215 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b99b8fd-1388-4e40-b69a-fcc3d901a224-insights-runtime-extractor-tls podName:4b99b8fd-1388-4e40-b69a-fcc3d901a224 nodeName:}" failed. No retries permitted until 2026-09-18 03:36:54.960197063 +0000 UTC m=+229.242958907 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4b99b8fd-1388-4e40-b69a-fcc3d901a224-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m5g7q" (UID: "4b99b8fd-1388-4e40-b69a-fcc3d901a224") : secret "insights-runtime-extractor-tls" not found Sep 18 03:36:50.960298 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:50.960232 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b99b8fd-1388-4e40-b69a-fcc3d901a224-kube-rbac-proxy-cm podName:4b99b8fd-1388-4e40-b69a-fcc3d901a224 nodeName:}" failed. No retries permitted until 2026-09-18 03:36:54.96022177 +0000 UTC m=+229.242983617 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4b99b8fd-1388-4e40-b69a-fcc3d901a224-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m5g7q" (UID: "4b99b8fd-1388-4e40-b69a-fcc3d901a224") : configmap "kube-rbac-proxy" not found Sep 18 03:36:51.064069 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:51.064037 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5ccx6_2ad38ce3-1483-4747-a9b3-7ccbfbb71957/serve-healthcheck-canary/0.log" Sep 18 03:36:51.263400 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:51.263326 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff4a6710-b06e-4c27-b451-aed9164d0ffd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-vcm9j\" (UID: \"ff4a6710-b06e-4c27-b451-aed9164d0ffd\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-vcm9j" Sep 18 03:36:51.263400 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:51.263376 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/abce51bb-9c6c-4497-b966-7e03a2935245-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-bz22t\" (UID: \"abce51bb-9c6c-4497-b966-7e03a2935245\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-bz22t" Sep 18 03:36:51.263567 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:51.263427 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-metrics-certs\") pod \"router-default-59cb5d4fd6-68pct\" (UID: \"a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95\") " pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:36:51.263567 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:51.263443 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-service-ca-bundle\") pod \"router-default-59cb5d4fd6-68pct\" (UID: \"a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95\") " pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:36:51.263567 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:51.263477 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 18 03:36:51.263567 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:51.263533 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff4a6710-b06e-4c27-b451-aed9164d0ffd-cluster-monitoring-operator-tls podName:ff4a6710-b06e-4c27-b451-aed9164d0ffd nodeName:}" failed. No retries permitted until 2026-09-18 03:36:59.263518114 +0000 UTC m=+233.546279965 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ff4a6710-b06e-4c27-b451-aed9164d0ffd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-vcm9j" (UID: "ff4a6710-b06e-4c27-b451-aed9164d0ffd") : secret "cluster-monitoring-operator-tls" not found Sep 18 03:36:51.263567 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:51.263555 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 18 03:36:51.263567 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:51.263561 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-service-ca-bundle podName:a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95 nodeName:}" failed. No retries permitted until 2026-09-18 03:36:59.26354513 +0000 UTC m=+233.546306984 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-service-ca-bundle") pod "router-default-59cb5d4fd6-68pct" (UID: "a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95") : configmap references non-existent config key: service-ca.crt Sep 18 03:36:51.263760 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:51.263593 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abce51bb-9c6c-4497-b966-7e03a2935245-samples-operator-tls podName:abce51bb-9c6c-4497-b966-7e03a2935245 nodeName:}" failed. No retries permitted until 2026-09-18 03:36:59.263582166 +0000 UTC m=+233.546344011 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/abce51bb-9c6c-4497-b966-7e03a2935245-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-bz22t" (UID: "abce51bb-9c6c-4497-b966-7e03a2935245") : secret "samples-operator-tls" not found Sep 18 03:36:51.263760 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:51.263603 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 18 03:36:51.263760 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:51.263638 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-metrics-certs podName:a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95 nodeName:}" failed. No retries permitted until 2026-09-18 03:36:59.263627523 +0000 UTC m=+233.546389378 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-metrics-certs") pod "router-default-59cb5d4fd6-68pct" (UID: "a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95") : secret "router-metrics-certs-default" not found Sep 18 03:36:51.658504 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:51.658471 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-5qbkt" event={"ID":"44c39f94-93e8-46a5-aa1c-0d16cc984015","Type":"ContainerStarted","Data":"3a207de0edd37226f24bdee76614ff48411f88c975edb33b3b2afaa355390766"} Sep 18 03:36:51.658660 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:51.658530 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-5qbkt" event={"ID":"44c39f94-93e8-46a5-aa1c-0d16cc984015","Type":"ContainerStarted","Data":"f3a681a55d0a0caacb86ccc313bd11c45dd6dbf035c6ec2e1e073c020acdd2d6"} Sep 18 03:36:51.674550 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:51.674511 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-5qbkt" podStartSLOduration=2.629871892 podStartE2EDuration="3.674497486s" totalImagesPullingTime="1.044625594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:36:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:36:51.673896841 +0000 UTC m=+225.956658706" watchObservedRunningTime="2026-09-18 03:36:51.674497486 +0000 UTC m=+225.957259387" Sep 18 03:36:54.991668 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:54.991626 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b99b8fd-1388-4e40-b69a-fcc3d901a224-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m5g7q\" (UID: \"4b99b8fd-1388-4e40-b69a-fcc3d901a224\") " pod="openshift-insights/insights-runtime-extractor-m5g7q" Sep 18 03:36:54.992079 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:54.991683 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b99b8fd-1388-4e40-b69a-fcc3d901a224-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m5g7q\" (UID: \"4b99b8fd-1388-4e40-b69a-fcc3d901a224\") " pod="openshift-insights/insights-runtime-extractor-m5g7q" Sep 18 03:36:54.992079 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:54.991749 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 18 03:36:54.992079 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:54.991793 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 18 03:36:54.992079 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:54.991818 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b99b8fd-1388-4e40-b69a-fcc3d901a224-kube-rbac-proxy-cm podName:4b99b8fd-1388-4e40-b69a-fcc3d901a224 nodeName:}" failed. No retries permitted until 2026-09-18 03:37:02.991802833 +0000 UTC m=+237.274564679 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4b99b8fd-1388-4e40-b69a-fcc3d901a224-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m5g7q" (UID: "4b99b8fd-1388-4e40-b69a-fcc3d901a224") : configmap "kube-rbac-proxy" not found Sep 18 03:36:54.992079 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:36:54.991867 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b99b8fd-1388-4e40-b69a-fcc3d901a224-insights-runtime-extractor-tls podName:4b99b8fd-1388-4e40-b69a-fcc3d901a224 nodeName:}" failed. No retries permitted until 2026-09-18 03:37:02.991853751 +0000 UTC m=+237.274615604 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4b99b8fd-1388-4e40-b69a-fcc3d901a224-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m5g7q" (UID: "4b99b8fd-1388-4e40-b69a-fcc3d901a224") : secret "insights-runtime-extractor-tls" not found Sep 18 03:36:59.328074 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:59.328038 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-metrics-certs\") pod \"router-default-59cb5d4fd6-68pct\" (UID: \"a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95\") " pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:36:59.328074 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:59.328074 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-service-ca-bundle\") pod \"router-default-59cb5d4fd6-68pct\" (UID: \"a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95\") " pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:36:59.328569 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:59.328116 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff4a6710-b06e-4c27-b451-aed9164d0ffd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-vcm9j\" (UID: \"ff4a6710-b06e-4c27-b451-aed9164d0ffd\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-vcm9j" Sep 18 03:36:59.328569 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:59.328149 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/abce51bb-9c6c-4497-b966-7e03a2935245-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-bz22t\" (UID: \"abce51bb-9c6c-4497-b966-7e03a2935245\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-bz22t" Sep 18 03:36:59.328848 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:59.328823 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-service-ca-bundle\") pod \"router-default-59cb5d4fd6-68pct\" (UID: \"a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95\") " pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:36:59.330432 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:59.330410 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff4a6710-b06e-4c27-b451-aed9164d0ffd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-vcm9j\" (UID: \"ff4a6710-b06e-4c27-b451-aed9164d0ffd\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-vcm9j" Sep 18 03:36:59.330525 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:59.330496 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/abce51bb-9c6c-4497-b966-7e03a2935245-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-bz22t\" (UID: \"abce51bb-9c6c-4497-b966-7e03a2935245\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-bz22t" Sep 18 03:36:59.330564 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:59.330503 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95-metrics-certs\") pod \"router-default-59cb5d4fd6-68pct\" (UID: \"a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95\") " pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:36:59.475650 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:59.475614 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-bz22t" Sep 18 03:36:59.497652 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:59.497625 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-vcm9j" Sep 18 03:36:59.502359 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:59.502335 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:36:59.746522 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:59.746399 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-bz22t"] Sep 18 03:36:59.779025 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:59.779001 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-vcm9j"] Sep 18 03:36:59.783989 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:36:59.783959 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff4a6710_b06e_4c27_b451_aed9164d0ffd.slice/crio-31fa5ee10d46ac700d152cf21c9bdb148a4785f295fdd6ca860ea40ac41724f0 WatchSource:0}: Error finding container 31fa5ee10d46ac700d152cf21c9bdb148a4785f295fdd6ca860ea40ac41724f0: Status 404 returned error can't find the container with id 31fa5ee10d46ac700d152cf21c9bdb148a4785f295fdd6ca860ea40ac41724f0 Sep 18 03:36:59.786183 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:36:59.786087 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-59cb5d4fd6-68pct"] Sep 18 03:36:59.788499 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:36:59.788473 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda827da8d_aca7_4b1e_8cb3_88ea5c4d3e95.slice/crio-e38ccc6953d77e4404d52e03d9f383e388f59a671278247b240ac3eb0348cd03 WatchSource:0}: Error finding container e38ccc6953d77e4404d52e03d9f383e388f59a671278247b240ac3eb0348cd03: Status 404 returned error can't find the container with id e38ccc6953d77e4404d52e03d9f383e388f59a671278247b240ac3eb0348cd03 Sep 18 03:37:00.688828 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:00.688789 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-59cb5d4fd6-68pct" event={"ID":"a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95","Type":"ContainerStarted","Data":"170637a417e88890f2e871425d031600316e7d1a150524c6ee56a91e33ce8167"} Sep 18 03:37:00.689303 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:00.688851 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-59cb5d4fd6-68pct" event={"ID":"a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95","Type":"ContainerStarted","Data":"e38ccc6953d77e4404d52e03d9f383e388f59a671278247b240ac3eb0348cd03"} Sep 18 03:37:00.690152 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:00.690121 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-vcm9j" event={"ID":"ff4a6710-b06e-4c27-b451-aed9164d0ffd","Type":"ContainerStarted","Data":"31fa5ee10d46ac700d152cf21c9bdb148a4785f295fdd6ca860ea40ac41724f0"} Sep 18 03:37:00.691465 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:00.691429 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-bz22t" event={"ID":"abce51bb-9c6c-4497-b966-7e03a2935245","Type":"ContainerStarted","Data":"4a7eaa1bfc7c0a709f6ffe316dfaa67d88f17c6f8b2ccf1bd8f61f5ae399624a"} Sep 18 03:37:00.706920 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:00.706864 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-59cb5d4fd6-68pct" podStartSLOduration=17.706843125 podStartE2EDuration="17.706843125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:36:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:37:00.706188346 +0000 UTC m=+234.988950211" watchObservedRunningTime="2026-09-18 03:37:00.706843125 +0000 UTC m=+234.989604990" Sep 18 03:37:01.503399 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:01.503361 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:37:01.506110 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:01.506082 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:37:01.695777 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:01.695730 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:37:01.697169 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:01.697141 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-59cb5d4fd6-68pct" Sep 18 03:37:02.700304 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:02.700266 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-vcm9j" event={"ID":"ff4a6710-b06e-4c27-b451-aed9164d0ffd","Type":"ContainerStarted","Data":"6c2037df72fac92f6b7ee0c53bd95d468a5f7295e64ac9dab909827d07e9c665"} Sep 18 03:37:02.702260 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:02.702238 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-bz22t" event={"ID":"abce51bb-9c6c-4497-b966-7e03a2935245","Type":"ContainerStarted","Data":"69c410bf25aff5b1d2a14f7fc6b4b08ef8a95342f257bf0704a5fcdabd50afc8"} Sep 18 03:37:02.702384 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:02.702275 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-bz22t" event={"ID":"abce51bb-9c6c-4497-b966-7e03a2935245","Type":"ContainerStarted","Data":"cbde1acaaa70475dbc081ff276501fbd146dc38b2f3bca4c109396031c2a8324"} Sep 18 03:37:02.717672 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:02.717632 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-vcm9j" podStartSLOduration=17.633879465 podStartE2EDuration="19.717619338s" totalImagesPullingTime="2.083739873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:36:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:37:02.716953167 +0000 UTC m=+236.999715026" watchObservedRunningTime="2026-09-18 03:37:02.717619338 +0000 UTC m=+237.000381201" Sep 18 03:37:02.732096 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:02.732053 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-bz22t" podStartSLOduration=17.665514046 podStartE2EDuration="19.732040712s" totalImagesPullingTime="2.066526666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:36:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:37:02.731984966 +0000 UTC m=+237.014746831" watchObservedRunningTime="2026-09-18 03:37:02.732040712 +0000 UTC m=+237.014802574" Sep 18 03:37:03.057130 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:03.057049 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b99b8fd-1388-4e40-b69a-fcc3d901a224-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m5g7q\" (UID: \"4b99b8fd-1388-4e40-b69a-fcc3d901a224\") " pod="openshift-insights/insights-runtime-extractor-m5g7q" Sep 18 03:37:03.057130 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:03.057085 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b99b8fd-1388-4e40-b69a-fcc3d901a224-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m5g7q\" (UID: \"4b99b8fd-1388-4e40-b69a-fcc3d901a224\") " pod="openshift-insights/insights-runtime-extractor-m5g7q" Sep 18 03:37:03.057538 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:03.057518 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b99b8fd-1388-4e40-b69a-fcc3d901a224-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m5g7q\" (UID: \"4b99b8fd-1388-4e40-b69a-fcc3d901a224\") " pod="openshift-insights/insights-runtime-extractor-m5g7q" Sep 18 03:37:03.059327 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:03.059311 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b99b8fd-1388-4e40-b69a-fcc3d901a224-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m5g7q\" (UID: \"4b99b8fd-1388-4e40-b69a-fcc3d901a224\") " pod="openshift-insights/insights-runtime-extractor-m5g7q" Sep 18 03:37:03.217839 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:03.217808 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m5g7q" Sep 18 03:37:03.485597 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:03.485569 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m5g7q"] Sep 18 03:37:03.488665 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:37:03.488642 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b99b8fd_1388_4e40_b69a_fcc3d901a224.slice/crio-9b6e8901c87157027bccec75f59865b94ecd8d9b6caee81344607695a66b99a3 WatchSource:0}: Error finding container 9b6e8901c87157027bccec75f59865b94ecd8d9b6caee81344607695a66b99a3: Status 404 returned error can't find the container with id 9b6e8901c87157027bccec75f59865b94ecd8d9b6caee81344607695a66b99a3 Sep 18 03:37:03.706874 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:03.706843 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m5g7q" event={"ID":"4b99b8fd-1388-4e40-b69a-fcc3d901a224","Type":"ContainerStarted","Data":"2d60757b3f049d84f3147b0dfd674f4536a8cb8d43c735a2170ae5bdbab2cb0a"} Sep 18 03:37:03.707303 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:03.706902 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m5g7q" event={"ID":"4b99b8fd-1388-4e40-b69a-fcc3d901a224","Type":"ContainerStarted","Data":"9b6e8901c87157027bccec75f59865b94ecd8d9b6caee81344607695a66b99a3"} Sep 18 03:37:04.712404 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:04.712365 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m5g7q" event={"ID":"4b99b8fd-1388-4e40-b69a-fcc3d901a224","Type":"ContainerStarted","Data":"91d4e628c49d08cbde3c1701ba023add58e2b3fc44fb5fd55a7fca92ef370e99"} Sep 18 03:37:05.717219 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:05.717185 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m5g7q" event={"ID":"4b99b8fd-1388-4e40-b69a-fcc3d901a224","Type":"ContainerStarted","Data":"33f2c942f282b755a1aedecbd136ff0590d08a7ea5b64989e11fd8a230f04da7"} Sep 18 03:37:05.734048 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:05.733996 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-m5g7q" podStartSLOduration=16.760623248 podStartE2EDuration="18.733980635s" totalImagesPullingTime="1.973357387s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 03:36:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:37:05.733182785 +0000 UTC m=+240.015944639" watchObservedRunningTime="2026-09-18 03:37:05.733980635 +0000 UTC m=+240.016742499" Sep 18 03:37:09.755233 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:09.755205 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-pdpn7"] Sep 18 03:37:09.761478 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:09.761449 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-pdpn7" Sep 18 03:37:09.763699 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:09.763677 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-7jp56\"" Sep 18 03:37:09.763826 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:09.763741 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 18 03:37:09.769545 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:09.769523 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-pdpn7"] Sep 18 03:37:09.806169 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:09.806136 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/93874dc2-35da-4afd-9016-216a9c5afdc9-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-pdpn7\" (UID: \"93874dc2-35da-4afd-9016-216a9c5afdc9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-pdpn7" Sep 18 03:37:09.875158 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:09.875127 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6d95fdc8fb-hgkfh"] Sep 18 03:37:09.877195 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:09.877176 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:09.879629 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:09.879593 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 18 03:37:09.879629 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:09.879614 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jgsps\"" Sep 18 03:37:09.879808 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:09.879709 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 18 03:37:09.879958 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:09.879919 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 18 03:37:09.884671 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:09.884653 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 18 03:37:09.892381 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:09.892360 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d95fdc8fb-hgkfh"] Sep 18 03:37:09.907205 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:09.907185 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/93874dc2-35da-4afd-9016-216a9c5afdc9-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-pdpn7\" (UID: \"93874dc2-35da-4afd-9016-216a9c5afdc9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-pdpn7" Sep 18 03:37:09.909636 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:09.909615 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/93874dc2-35da-4afd-9016-216a9c5afdc9-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-pdpn7\" (UID: \"93874dc2-35da-4afd-9016-216a9c5afdc9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-pdpn7" Sep 18 03:37:10.008095 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.008017 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tjjn\" (UniqueName: \"kubernetes.io/projected/212ab307-3717-41ab-93e2-1a92ae45fecf-kube-api-access-8tjjn\") pod \"image-registry-6d95fdc8fb-hgkfh\" (UID: \"212ab307-3717-41ab-93e2-1a92ae45fecf\") " pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:10.008095 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.008045 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/212ab307-3717-41ab-93e2-1a92ae45fecf-trusted-ca\") pod \"image-registry-6d95fdc8fb-hgkfh\" (UID: \"212ab307-3717-41ab-93e2-1a92ae45fecf\") " pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:10.008095 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.008065 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/212ab307-3717-41ab-93e2-1a92ae45fecf-registry-tls\") pod \"image-registry-6d95fdc8fb-hgkfh\" (UID: \"212ab307-3717-41ab-93e2-1a92ae45fecf\") " pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:10.008095 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.008094 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/212ab307-3717-41ab-93e2-1a92ae45fecf-installation-pull-secrets\") pod \"image-registry-6d95fdc8fb-hgkfh\" (UID: \"212ab307-3717-41ab-93e2-1a92ae45fecf\") " pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:10.008315 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.008110 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/212ab307-3717-41ab-93e2-1a92ae45fecf-image-registry-private-configuration\") pod \"image-registry-6d95fdc8fb-hgkfh\" (UID: \"212ab307-3717-41ab-93e2-1a92ae45fecf\") " pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:10.008315 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.008136 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/212ab307-3717-41ab-93e2-1a92ae45fecf-registry-certificates\") pod \"image-registry-6d95fdc8fb-hgkfh\" (UID: \"212ab307-3717-41ab-93e2-1a92ae45fecf\") " pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:10.008377 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.008332 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/212ab307-3717-41ab-93e2-1a92ae45fecf-bound-sa-token\") pod \"image-registry-6d95fdc8fb-hgkfh\" (UID: \"212ab307-3717-41ab-93e2-1a92ae45fecf\") " pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:10.008427 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.008408 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/212ab307-3717-41ab-93e2-1a92ae45fecf-ca-trust-extracted\") pod \"image-registry-6d95fdc8fb-hgkfh\" (UID: \"212ab307-3717-41ab-93e2-1a92ae45fecf\") " pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:10.071720 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.071690 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-pdpn7" Sep 18 03:37:10.109180 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.109151 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/212ab307-3717-41ab-93e2-1a92ae45fecf-bound-sa-token\") pod \"image-registry-6d95fdc8fb-hgkfh\" (UID: \"212ab307-3717-41ab-93e2-1a92ae45fecf\") " pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:10.109352 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.109201 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/212ab307-3717-41ab-93e2-1a92ae45fecf-ca-trust-extracted\") pod \"image-registry-6d95fdc8fb-hgkfh\" (UID: \"212ab307-3717-41ab-93e2-1a92ae45fecf\") " pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:10.109352 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.109220 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tjjn\" (UniqueName: \"kubernetes.io/projected/212ab307-3717-41ab-93e2-1a92ae45fecf-kube-api-access-8tjjn\") pod \"image-registry-6d95fdc8fb-hgkfh\" (UID: \"212ab307-3717-41ab-93e2-1a92ae45fecf\") " pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:10.109352 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.109244 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/212ab307-3717-41ab-93e2-1a92ae45fecf-trusted-ca\") pod \"image-registry-6d95fdc8fb-hgkfh\" (UID: \"212ab307-3717-41ab-93e2-1a92ae45fecf\") " pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:10.109352 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.109260 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/212ab307-3717-41ab-93e2-1a92ae45fecf-registry-tls\") pod \"image-registry-6d95fdc8fb-hgkfh\" (UID: \"212ab307-3717-41ab-93e2-1a92ae45fecf\") " pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:10.109352 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.109302 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/212ab307-3717-41ab-93e2-1a92ae45fecf-installation-pull-secrets\") pod \"image-registry-6d95fdc8fb-hgkfh\" (UID: \"212ab307-3717-41ab-93e2-1a92ae45fecf\") " pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:10.109352 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.109329 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/212ab307-3717-41ab-93e2-1a92ae45fecf-image-registry-private-configuration\") pod \"image-registry-6d95fdc8fb-hgkfh\" (UID: \"212ab307-3717-41ab-93e2-1a92ae45fecf\") " pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:10.109658 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.109367 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/212ab307-3717-41ab-93e2-1a92ae45fecf-registry-certificates\") pod \"image-registry-6d95fdc8fb-hgkfh\" (UID: \"212ab307-3717-41ab-93e2-1a92ae45fecf\") " pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:10.109658 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.109606 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/212ab307-3717-41ab-93e2-1a92ae45fecf-ca-trust-extracted\") pod \"image-registry-6d95fdc8fb-hgkfh\" (UID: \"212ab307-3717-41ab-93e2-1a92ae45fecf\") " pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:10.110190 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.110167 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/212ab307-3717-41ab-93e2-1a92ae45fecf-registry-certificates\") pod \"image-registry-6d95fdc8fb-hgkfh\" (UID: \"212ab307-3717-41ab-93e2-1a92ae45fecf\") " pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:10.110643 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.110619 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/212ab307-3717-41ab-93e2-1a92ae45fecf-trusted-ca\") pod \"image-registry-6d95fdc8fb-hgkfh\" (UID: \"212ab307-3717-41ab-93e2-1a92ae45fecf\") " pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:10.111871 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.111822 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/212ab307-3717-41ab-93e2-1a92ae45fecf-image-registry-private-configuration\") pod \"image-registry-6d95fdc8fb-hgkfh\" (UID: \"212ab307-3717-41ab-93e2-1a92ae45fecf\") " pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:10.112022 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.111911 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/212ab307-3717-41ab-93e2-1a92ae45fecf-installation-pull-secrets\") pod \"image-registry-6d95fdc8fb-hgkfh\" (UID: \"212ab307-3717-41ab-93e2-1a92ae45fecf\") " pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:10.112236 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.112216 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/212ab307-3717-41ab-93e2-1a92ae45fecf-registry-tls\") pod \"image-registry-6d95fdc8fb-hgkfh\" (UID: \"212ab307-3717-41ab-93e2-1a92ae45fecf\") " pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:10.117273 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.117067 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tjjn\" (UniqueName: \"kubernetes.io/projected/212ab307-3717-41ab-93e2-1a92ae45fecf-kube-api-access-8tjjn\") pod \"image-registry-6d95fdc8fb-hgkfh\" (UID: \"212ab307-3717-41ab-93e2-1a92ae45fecf\") " pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:10.118342 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.117950 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/212ab307-3717-41ab-93e2-1a92ae45fecf-bound-sa-token\") pod \"image-registry-6d95fdc8fb-hgkfh\" (UID: \"212ab307-3717-41ab-93e2-1a92ae45fecf\") " pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:10.186431 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.186398 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:10.336357 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.336337 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-pdpn7"] Sep 18 03:37:10.338208 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:37:10.338186 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93874dc2_35da_4afd_9016_216a9c5afdc9.slice/crio-f669185e9e9cc38dcf791f2393ca4aa14894eb71392200d54b35deaee6f475e3 WatchSource:0}: Error finding container f669185e9e9cc38dcf791f2393ca4aa14894eb71392200d54b35deaee6f475e3: Status 404 returned error can't find the container with id f669185e9e9cc38dcf791f2393ca4aa14894eb71392200d54b35deaee6f475e3 Sep 18 03:37:10.453752 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.453725 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d95fdc8fb-hgkfh"] Sep 18 03:37:10.455997 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:37:10.455962 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod212ab307_3717_41ab_93e2_1a92ae45fecf.slice/crio-96d9e074c8d1c652f9c0f81f940e4f142490021862c727bfb70fe4fee0c49a0a WatchSource:0}: Error finding container 96d9e074c8d1c652f9c0f81f940e4f142490021862c727bfb70fe4fee0c49a0a: Status 404 returned error can't find the container with id 96d9e074c8d1c652f9c0f81f940e4f142490021862c727bfb70fe4fee0c49a0a Sep 18 03:37:10.736659 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.736627 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" event={"ID":"212ab307-3717-41ab-93e2-1a92ae45fecf","Type":"ContainerStarted","Data":"0363c17d5e4c63f37f28f6c9fa348e2b9c90774d10142231a667ef469a7a60c3"} Sep 18 03:37:10.736830 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.736674 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" event={"ID":"212ab307-3717-41ab-93e2-1a92ae45fecf","Type":"ContainerStarted","Data":"96d9e074c8d1c652f9c0f81f940e4f142490021862c727bfb70fe4fee0c49a0a"} Sep 18 03:37:10.736830 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.736740 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:10.737978 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.737954 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-pdpn7" event={"ID":"93874dc2-35da-4afd-9016-216a9c5afdc9","Type":"ContainerStarted","Data":"f669185e9e9cc38dcf791f2393ca4aa14894eb71392200d54b35deaee6f475e3"} Sep 18 03:37:10.754032 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:10.753992 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" podStartSLOduration=1.7539776150000002 podStartE2EDuration="1.753977615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:37:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:37:10.752845693 +0000 UTC m=+245.035607555" watchObservedRunningTime="2026-09-18 03:37:10.753977615 +0000 UTC m=+245.036739478" Sep 18 03:37:11.742493 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:11.742464 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-pdpn7" event={"ID":"93874dc2-35da-4afd-9016-216a9c5afdc9","Type":"ContainerStarted","Data":"9dbf009c97617dd5b0271f5c1caf50cbd1190bb06fd4d729760bd89476025b47"} Sep 18 03:37:11.742860 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:11.742775 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-pdpn7" Sep 18 03:37:11.747218 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:11.747194 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-pdpn7" Sep 18 03:37:11.762640 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:11.762600 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-pdpn7" podStartSLOduration=1.566830639 podStartE2EDuration="2.762588708s" totalImagesPullingTime="1.195758069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:37:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:37:11.762085275 +0000 UTC m=+246.044847141" watchObservedRunningTime="2026-09-18 03:37:11.762588708 +0000 UTC m=+246.045350571" Sep 18 03:37:12.415524 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:12.415492 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-v72z7"] Sep 18 03:37:12.417599 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:12.417584 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-v72z7" Sep 18 03:37:12.419823 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:12.419782 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 18 03:37:12.419823 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:12.419791 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 18 03:37:12.420072 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:12.419960 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-zqc42\"" Sep 18 03:37:12.420380 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:12.420361 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 18 03:37:12.426293 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:12.426273 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-v72z7"] Sep 18 03:37:12.530107 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:12.530067 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/088dec8d-8500-44f7-bf61-8540ac607fc2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-v72z7\" (UID: \"088dec8d-8500-44f7-bf61-8540ac607fc2\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-v72z7" Sep 18 03:37:12.530263 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:12.530119 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/088dec8d-8500-44f7-bf61-8540ac607fc2-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-v72z7\" (UID: \"088dec8d-8500-44f7-bf61-8540ac607fc2\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-v72z7" Sep 18 03:37:12.530263 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:12.530206 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/088dec8d-8500-44f7-bf61-8540ac607fc2-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-v72z7\" (UID: \"088dec8d-8500-44f7-bf61-8540ac607fc2\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-v72z7" Sep 18 03:37:12.530263 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:12.530247 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b5tn\" (UniqueName: \"kubernetes.io/projected/088dec8d-8500-44f7-bf61-8540ac607fc2-kube-api-access-2b5tn\") pod \"prometheus-operator-7bd5d56d6-v72z7\" (UID: \"088dec8d-8500-44f7-bf61-8540ac607fc2\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-v72z7" Sep 18 03:37:12.630921 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:12.630886 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/088dec8d-8500-44f7-bf61-8540ac607fc2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-v72z7\" (UID: \"088dec8d-8500-44f7-bf61-8540ac607fc2\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-v72z7" Sep 18 03:37:12.631093 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:12.630943 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/088dec8d-8500-44f7-bf61-8540ac607fc2-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-v72z7\" (UID: \"088dec8d-8500-44f7-bf61-8540ac607fc2\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-v72z7" Sep 18 03:37:12.631093 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:12.630981 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/088dec8d-8500-44f7-bf61-8540ac607fc2-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-v72z7\" (UID: \"088dec8d-8500-44f7-bf61-8540ac607fc2\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-v72z7" Sep 18 03:37:12.631093 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:12.631005 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b5tn\" (UniqueName: \"kubernetes.io/projected/088dec8d-8500-44f7-bf61-8540ac607fc2-kube-api-access-2b5tn\") pod \"prometheus-operator-7bd5d56d6-v72z7\" (UID: \"088dec8d-8500-44f7-bf61-8540ac607fc2\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-v72z7" Sep 18 03:37:12.631240 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:37:12.631090 2568 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Sep 18 03:37:12.631240 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:37:12.631159 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/088dec8d-8500-44f7-bf61-8540ac607fc2-prometheus-operator-tls podName:088dec8d-8500-44f7-bf61-8540ac607fc2 nodeName:}" failed. No retries permitted until 2026-09-18 03:37:13.131142578 +0000 UTC m=+247.413904419 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/088dec8d-8500-44f7-bf61-8540ac607fc2-prometheus-operator-tls") pod "prometheus-operator-7bd5d56d6-v72z7" (UID: "088dec8d-8500-44f7-bf61-8540ac607fc2") : secret "prometheus-operator-tls" not found Sep 18 03:37:12.631679 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:12.631660 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/088dec8d-8500-44f7-bf61-8540ac607fc2-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-v72z7\" (UID: \"088dec8d-8500-44f7-bf61-8540ac607fc2\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-v72z7" Sep 18 03:37:12.633271 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:12.633251 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/088dec8d-8500-44f7-bf61-8540ac607fc2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-v72z7\" (UID: \"088dec8d-8500-44f7-bf61-8540ac607fc2\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-v72z7" Sep 18 03:37:12.638554 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:12.638532 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b5tn\" (UniqueName: \"kubernetes.io/projected/088dec8d-8500-44f7-bf61-8540ac607fc2-kube-api-access-2b5tn\") pod \"prometheus-operator-7bd5d56d6-v72z7\" (UID: \"088dec8d-8500-44f7-bf61-8540ac607fc2\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-v72z7" Sep 18 03:37:13.134946 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:13.134907 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/088dec8d-8500-44f7-bf61-8540ac607fc2-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-v72z7\" (UID: \"088dec8d-8500-44f7-bf61-8540ac607fc2\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-v72z7" Sep 18 03:37:13.137187 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:13.137167 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/088dec8d-8500-44f7-bf61-8540ac607fc2-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-v72z7\" (UID: \"088dec8d-8500-44f7-bf61-8540ac607fc2\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-v72z7" Sep 18 03:37:13.327414 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:13.327375 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-v72z7" Sep 18 03:37:13.592670 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:13.592637 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-v72z7"] Sep 18 03:37:13.595805 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:37:13.595779 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod088dec8d_8500_44f7_bf61_8540ac607fc2.slice/crio-8979ae9ad4b394ecb3b4e8cc642a009f9167f0ece2114bbdbdc9f150c2b2b844 WatchSource:0}: Error finding container 8979ae9ad4b394ecb3b4e8cc642a009f9167f0ece2114bbdbdc9f150c2b2b844: Status 404 returned error can't find the container with id 8979ae9ad4b394ecb3b4e8cc642a009f9167f0ece2114bbdbdc9f150c2b2b844 Sep 18 03:37:13.749971 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:13.749926 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-v72z7" event={"ID":"088dec8d-8500-44f7-bf61-8540ac607fc2","Type":"ContainerStarted","Data":"8979ae9ad4b394ecb3b4e8cc642a009f9167f0ece2114bbdbdc9f150c2b2b844"} Sep 18 03:37:15.758246 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:15.758217 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-v72z7" event={"ID":"088dec8d-8500-44f7-bf61-8540ac607fc2","Type":"ContainerStarted","Data":"ceb84054ff923ce95438c478229a503eb3869f5bc9a0d54d46d126c27c3db98c"} Sep 18 03:37:15.758623 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:15.758265 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-v72z7" event={"ID":"088dec8d-8500-44f7-bf61-8540ac607fc2","Type":"ContainerStarted","Data":"75312bdb833b99280ff567c96adfc4d35e8b43e2273198ae281b4520b13ffb78"} Sep 18 03:37:15.774465 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:15.774415 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-v72z7" podStartSLOduration=2.35981342 podStartE2EDuration="3.774399531s" totalImagesPullingTime="1.414586111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:37:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:37:15.773184608 +0000 UTC m=+250.055946490" watchObservedRunningTime="2026-09-18 03:37:15.774399531 +0000 UTC m=+250.057161418" Sep 18 03:37:17.765490 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.765460 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wtl48"] Sep 18 03:37:17.768468 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.768450 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:17.771292 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.771269 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 18 03:37:17.771769 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.771752 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 18 03:37:17.771769 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.771761 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 18 03:37:17.772810 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.772794 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kzscl\"" Sep 18 03:37:17.787157 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.787137 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-55dzd"] Sep 18 03:37:17.789083 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.789070 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-55dzd" Sep 18 03:37:17.792017 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.791999 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-8plnv\"" Sep 18 03:37:17.792127 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.792022 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 18 03:37:17.792127 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.792087 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 18 03:37:17.792235 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.792136 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 18 03:37:17.810711 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.810690 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-55dzd"] Sep 18 03:37:17.872134 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.872102 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cnfd\" (UniqueName: \"kubernetes.io/projected/c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87-kube-api-access-2cnfd\") pod \"kube-state-metrics-84c885658c-55dzd\" (UID: \"c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-55dzd" Sep 18 03:37:17.872266 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.872145 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-node-exporter-textfile\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:17.872266 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.872169 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-node-exporter-tls\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:17.872266 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.872194 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-55dzd\" (UID: \"c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-55dzd" Sep 18 03:37:17.872266 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.872254 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tstp\" (UniqueName: \"kubernetes.io/projected/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-kube-api-access-9tstp\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:17.872415 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.872296 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-metrics-client-ca\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:17.872415 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.872315 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:17.872415 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.872334 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-55dzd\" (UID: \"c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-55dzd" Sep 18 03:37:17.872415 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.872375 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-node-exporter-wtmp\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:17.872415 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.872394 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-55dzd\" (UID: \"c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-55dzd" Sep 18 03:37:17.872415 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.872413 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-root\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:17.872740 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.872454 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-55dzd\" (UID: \"c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-55dzd" Sep 18 03:37:17.872740 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.872500 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-sys\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:17.872740 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.872513 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-55dzd\" (UID: \"c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-55dzd" Sep 18 03:37:17.872740 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.872530 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-node-exporter-accelerators-collector-config\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:17.973572 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.973537 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-node-exporter-tls\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:17.973572 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.973568 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-55dzd\" (UID: \"c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-55dzd" Sep 18 03:37:17.973796 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.973590 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tstp\" (UniqueName: \"kubernetes.io/projected/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-kube-api-access-9tstp\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:17.973796 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.973610 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-metrics-client-ca\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:17.973796 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.973626 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:17.973796 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.973643 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-55dzd\" (UID: \"c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-55dzd" Sep 18 03:37:17.973796 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.973662 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-node-exporter-wtmp\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:17.973796 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.973684 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-55dzd\" (UID: \"c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-55dzd" Sep 18 03:37:17.973796 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:37:17.973696 2568 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 18 03:37:17.973796 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.973709 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-root\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:17.973796 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:37:17.973764 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-node-exporter-tls podName:f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b nodeName:}" failed. No retries permitted until 2026-09-18 03:37:18.473743657 +0000 UTC m=+252.756505515 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-node-exporter-tls") pod "node-exporter-wtl48" (UID: "f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b") : secret "node-exporter-tls" not found Sep 18 03:37:17.973796 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.973782 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-root\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:17.974319 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.973806 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-55dzd\" (UID: \"c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-55dzd" Sep 18 03:37:17.974319 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.973873 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-sys\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:17.974319 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.973898 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-55dzd\" (UID: \"c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-55dzd" Sep 18 03:37:17.974319 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.973917 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-node-exporter-wtmp\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:17.974319 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.973959 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-node-exporter-accelerators-collector-config\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:17.974319 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:37:17.973992 2568 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Sep 18 03:37:17.974319 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:37:17.974061 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87-kube-state-metrics-tls podName:c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87 nodeName:}" failed. No retries permitted until 2026-09-18 03:37:18.474038304 +0000 UTC m=+252.756800147 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87-kube-state-metrics-tls") pod "kube-state-metrics-84c885658c-55dzd" (UID: "c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87") : secret "kube-state-metrics-tls" not found Sep 18 03:37:17.974319 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.973993 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cnfd\" (UniqueName: \"kubernetes.io/projected/c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87-kube-api-access-2cnfd\") pod \"kube-state-metrics-84c885658c-55dzd\" (UID: \"c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-55dzd" Sep 18 03:37:17.974319 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.974114 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-node-exporter-textfile\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:17.974319 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.974307 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-55dzd\" (UID: \"c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-55dzd" Sep 18 03:37:17.974790 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.974371 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-sys\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:17.974790 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.974376 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-metrics-client-ca\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:17.974790 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.974405 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-55dzd\" (UID: \"c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-55dzd" Sep 18 03:37:17.974790 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.974437 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-55dzd\" (UID: \"c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-55dzd" Sep 18 03:37:17.974790 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.974583 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-node-exporter-textfile\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:17.974790 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.974609 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-node-exporter-accelerators-collector-config\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:17.976450 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.976423 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:17.976575 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.976558 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-55dzd\" (UID: \"c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-55dzd" Sep 18 03:37:17.981586 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.981566 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tstp\" (UniqueName: \"kubernetes.io/projected/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-kube-api-access-9tstp\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:17.981586 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:17.981574 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cnfd\" (UniqueName: \"kubernetes.io/projected/c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87-kube-api-access-2cnfd\") pod \"kube-state-metrics-84c885658c-55dzd\" (UID: \"c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-55dzd" Sep 18 03:37:18.478423 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:18.478394 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-node-exporter-tls\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:18.478613 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:18.478435 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-55dzd\" (UID: \"c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-55dzd" Sep 18 03:37:18.481021 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:18.480990 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b-node-exporter-tls\") pod \"node-exporter-wtl48\" (UID: \"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b\") " pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:18.481115 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:18.480996 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-55dzd\" (UID: \"c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-55dzd" Sep 18 03:37:18.677921 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:18.677891 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wtl48" Sep 18 03:37:18.688825 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:37:18.688795 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0436d35_1fd8_4785_8ee0_a0a64d6c5c8b.slice/crio-190ee21bf581367cf42e87cc3f7b0bfa16167519d7fcbe52f16570735400661c WatchSource:0}: Error finding container 190ee21bf581367cf42e87cc3f7b0bfa16167519d7fcbe52f16570735400661c: Status 404 returned error can't find the container with id 190ee21bf581367cf42e87cc3f7b0bfa16167519d7fcbe52f16570735400661c Sep 18 03:37:18.697233 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:18.697211 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-55dzd" Sep 18 03:37:18.772685 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:18.772501 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wtl48" event={"ID":"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b","Type":"ContainerStarted","Data":"190ee21bf581367cf42e87cc3f7b0bfa16167519d7fcbe52f16570735400661c"} Sep 18 03:37:18.974114 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:18.974043 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-55dzd"] Sep 18 03:37:18.976157 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:37:18.976125 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1cb9ddf_10a4_4e3b_b1fd_f7bddd8e4c87.slice/crio-03510f8d7f9740180a2f299a07b44939d1d15a834a5d7536d0d503031b4ff987 WatchSource:0}: Error finding container 03510f8d7f9740180a2f299a07b44939d1d15a834a5d7536d0d503031b4ff987: Status 404 returned error can't find the container with id 03510f8d7f9740180a2f299a07b44939d1d15a834a5d7536d0d503031b4ff987 Sep 18 03:37:19.588864 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:19.588737 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs\") pod \"network-metrics-daemon-tf6sq\" (UID: \"9fdcb7aa-5f4c-4d16-a250-b272513204da\") " pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:37:19.591731 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:19.591677 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fdcb7aa-5f4c-4d16-a250-b272513204da-metrics-certs\") pod \"network-metrics-daemon-tf6sq\" (UID: \"9fdcb7aa-5f4c-4d16-a250-b272513204da\") " pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:37:19.779654 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:19.779573 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-55dzd" event={"ID":"c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87","Type":"ContainerStarted","Data":"03510f8d7f9740180a2f299a07b44939d1d15a834a5d7536d0d503031b4ff987"} Sep 18 03:37:19.880711 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:19.880635 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6p5fh\"" Sep 18 03:37:19.888859 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:19.888835 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tf6sq" Sep 18 03:37:20.525144 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:20.525071 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tf6sq"] Sep 18 03:37:20.529043 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:37:20.528998 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fdcb7aa_5f4c_4d16_a250_b272513204da.slice/crio-3cca2ec660fc91decc5d0d870ad0ec3174c2c7adfbf9284d5ccd943181e0931b WatchSource:0}: Error finding container 3cca2ec660fc91decc5d0d870ad0ec3174c2c7adfbf9284d5ccd943181e0931b: Status 404 returned error can't find the container with id 3cca2ec660fc91decc5d0d870ad0ec3174c2c7adfbf9284d5ccd943181e0931b Sep 18 03:37:20.785147 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:20.785053 2568 generic.go:356] "Generic (PLEG): container finished" podID="f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b" containerID="7cc1cabcfacf2b862f11664daa5ffd46cc6ef8248dc32499f232bb5e91876fa1" exitCode=0 Sep 18 03:37:20.785581 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:20.785144 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wtl48" event={"ID":"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b","Type":"ContainerDied","Data":"7cc1cabcfacf2b862f11664daa5ffd46cc6ef8248dc32499f232bb5e91876fa1"} Sep 18 03:37:20.787523 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:20.787485 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-55dzd" event={"ID":"c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87","Type":"ContainerStarted","Data":"21e40abbe083bc549977134bb38c2eaba8328b5b56b3b595cf2fc14178403c13"} Sep 18 03:37:20.787622 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:20.787553 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-55dzd" event={"ID":"c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87","Type":"ContainerStarted","Data":"8f6a5351744408d741f46acfc4039b37556f3364de94a35a72cace3f13100332"} Sep 18 03:37:20.787622 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:20.787587 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-55dzd" event={"ID":"c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87","Type":"ContainerStarted","Data":"eb4bd93f7be57104b159558de7790cb6a41bdc15abea7d82ca65859f5906ce1c"} Sep 18 03:37:20.788577 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:20.788558 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tf6sq" event={"ID":"9fdcb7aa-5f4c-4d16-a250-b272513204da","Type":"ContainerStarted","Data":"3cca2ec660fc91decc5d0d870ad0ec3174c2c7adfbf9284d5ccd943181e0931b"} Sep 18 03:37:20.823958 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:20.823682 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-84c885658c-55dzd" podStartSLOduration=2.536062245 podStartE2EDuration="3.823660931s" totalImagesPullingTime="1.287598686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:37:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:37:20.821549595 +0000 UTC m=+255.104311459" watchObservedRunningTime="2026-09-18 03:37:20.823660931 +0000 UTC m=+255.106422794" Sep 18 03:37:21.797918 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:21.797883 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wtl48" event={"ID":"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b","Type":"ContainerStarted","Data":"22daef1ba4f786b41d3c45d3a2fc14cac1267c2b5495eb02fee9012b079b46b8"} Sep 18 03:37:21.798331 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:21.797994 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wtl48" event={"ID":"f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b","Type":"ContainerStarted","Data":"1fc96f96560fa630452c709c6d9b5360f3b16cd3022f6adb01e1c93a00795fd2"} Sep 18 03:37:21.818781 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:21.818731 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wtl48" podStartSLOduration=3.175515143 podStartE2EDuration="4.8187155s" totalImagesPullingTime="1.573146839s" totalInitContainerRuntime="70.053518ms" isStatefulPod=true podCreationTimestamp="2026-09-18 03:37:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:37:21.81658651 +0000 UTC m=+256.099348384" watchObservedRunningTime="2026-09-18 03:37:21.8187155 +0000 UTC m=+256.101477403" Sep 18 03:37:22.540243 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:22.540212 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-qv7cp"] Sep 18 03:37:22.543862 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:22.543834 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-qv7cp" Sep 18 03:37:22.545887 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:22.545866 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 18 03:37:22.545996 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:22.545869 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-z82sw\"" Sep 18 03:37:22.550991 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:22.550971 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-qv7cp"] Sep 18 03:37:22.715136 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:22.715100 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkkrw\" (UniqueName: \"kubernetes.io/projected/130cb992-f186-4a31-9d7d-f75af513570f-kube-api-access-mkkrw\") pod \"monitoring-plugin-5984459ff8-qv7cp\" (UID: \"130cb992-f186-4a31-9d7d-f75af513570f\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-qv7cp" Sep 18 03:37:22.715320 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:22.715148 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/130cb992-f186-4a31-9d7d-f75af513570f-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-qv7cp\" (UID: \"130cb992-f186-4a31-9d7d-f75af513570f\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-qv7cp" Sep 18 03:37:22.803976 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:22.803884 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tf6sq" event={"ID":"9fdcb7aa-5f4c-4d16-a250-b272513204da","Type":"ContainerStarted","Data":"59f8228a5b93278bca351193c29ff8e2937b608d5504d54ff9d65015bdf0a241"} Sep 18 03:37:22.803976 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:22.803959 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tf6sq" event={"ID":"9fdcb7aa-5f4c-4d16-a250-b272513204da","Type":"ContainerStarted","Data":"7054be50d579d8d41aa51c9cf2ff8c1989ee38936961591d5b5e8f9e66652493"} Sep 18 03:37:22.815971 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:22.815948 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkkrw\" (UniqueName: \"kubernetes.io/projected/130cb992-f186-4a31-9d7d-f75af513570f-kube-api-access-mkkrw\") pod \"monitoring-plugin-5984459ff8-qv7cp\" (UID: \"130cb992-f186-4a31-9d7d-f75af513570f\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-qv7cp" Sep 18 03:37:22.816051 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:22.815985 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/130cb992-f186-4a31-9d7d-f75af513570f-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-qv7cp\" (UID: \"130cb992-f186-4a31-9d7d-f75af513570f\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-qv7cp" Sep 18 03:37:22.816105 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:37:22.816094 2568 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Sep 18 03:37:22.816149 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:37:22.816137 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/130cb992-f186-4a31-9d7d-f75af513570f-monitoring-plugin-cert podName:130cb992-f186-4a31-9d7d-f75af513570f nodeName:}" failed. No retries permitted until 2026-09-18 03:37:23.316124885 +0000 UTC m=+257.598886727 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/130cb992-f186-4a31-9d7d-f75af513570f-monitoring-plugin-cert") pod "monitoring-plugin-5984459ff8-qv7cp" (UID: "130cb992-f186-4a31-9d7d-f75af513570f") : secret "monitoring-plugin-cert" not found Sep 18 03:37:22.817857 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:22.817806 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-tf6sq" podStartSLOduration=253.416193779 podStartE2EDuration="4m14.817788022s" totalImagesPullingTime="1.401594243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:33:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:37:22.817350121 +0000 UTC m=+257.100111984" watchObservedRunningTime="2026-09-18 03:37:22.817788022 +0000 UTC m=+257.100549885" Sep 18 03:37:22.826512 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:22.826490 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkkrw\" (UniqueName: \"kubernetes.io/projected/130cb992-f186-4a31-9d7d-f75af513570f-kube-api-access-mkkrw\") pod \"monitoring-plugin-5984459ff8-qv7cp\" (UID: \"130cb992-f186-4a31-9d7d-f75af513570f\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-qv7cp" Sep 18 03:37:22.982993 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:22.982951 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-784749b4b9-vsm6m"] Sep 18 03:37:22.986334 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:22.986317 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" Sep 18 03:37:22.988777 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:22.988751 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 18 03:37:22.988777 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:22.988780 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 18 03:37:22.988994 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:22.988780 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 18 03:37:22.988994 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:22.988785 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 18 03:37:22.988994 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:22.988787 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-gkt6b\"" Sep 18 03:37:22.988994 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:22.988871 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 18 03:37:22.993063 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:22.993045 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 18 03:37:22.999798 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:22.999777 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-784749b4b9-vsm6m"] Sep 18 03:37:23.118288 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.118208 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3c785c79-caab-473c-9aa3-10e177f1bf0d-telemeter-client-tls\") pod \"telemeter-client-784749b4b9-vsm6m\" (UID: \"3c785c79-caab-473c-9aa3-10e177f1bf0d\") " pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" Sep 18 03:37:23.118288 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.118248 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3c785c79-caab-473c-9aa3-10e177f1bf0d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-784749b4b9-vsm6m\" (UID: \"3c785c79-caab-473c-9aa3-10e177f1bf0d\") " pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" Sep 18 03:37:23.118288 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.118267 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvbgb\" (UniqueName: \"kubernetes.io/projected/3c785c79-caab-473c-9aa3-10e177f1bf0d-kube-api-access-jvbgb\") pod \"telemeter-client-784749b4b9-vsm6m\" (UID: \"3c785c79-caab-473c-9aa3-10e177f1bf0d\") " pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" Sep 18 03:37:23.118288 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.118289 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3c785c79-caab-473c-9aa3-10e177f1bf0d-federate-client-tls\") pod \"telemeter-client-784749b4b9-vsm6m\" (UID: \"3c785c79-caab-473c-9aa3-10e177f1bf0d\") " pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" Sep 18 03:37:23.118516 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.118352 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c785c79-caab-473c-9aa3-10e177f1bf0d-serving-certs-ca-bundle\") pod \"telemeter-client-784749b4b9-vsm6m\" (UID: \"3c785c79-caab-473c-9aa3-10e177f1bf0d\") " pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" Sep 18 03:37:23.118516 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.118392 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c785c79-caab-473c-9aa3-10e177f1bf0d-metrics-client-ca\") pod \"telemeter-client-784749b4b9-vsm6m\" (UID: \"3c785c79-caab-473c-9aa3-10e177f1bf0d\") " pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" Sep 18 03:37:23.118516 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.118414 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c785c79-caab-473c-9aa3-10e177f1bf0d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-784749b4b9-vsm6m\" (UID: \"3c785c79-caab-473c-9aa3-10e177f1bf0d\") " pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" Sep 18 03:37:23.118516 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.118466 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3c785c79-caab-473c-9aa3-10e177f1bf0d-secret-telemeter-client\") pod \"telemeter-client-784749b4b9-vsm6m\" (UID: \"3c785c79-caab-473c-9aa3-10e177f1bf0d\") " pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" Sep 18 03:37:23.219913 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.219883 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3c785c79-caab-473c-9aa3-10e177f1bf0d-telemeter-client-tls\") pod \"telemeter-client-784749b4b9-vsm6m\" (UID: \"3c785c79-caab-473c-9aa3-10e177f1bf0d\") " pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" Sep 18 03:37:23.220066 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.219918 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3c785c79-caab-473c-9aa3-10e177f1bf0d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-784749b4b9-vsm6m\" (UID: \"3c785c79-caab-473c-9aa3-10e177f1bf0d\") " pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" Sep 18 03:37:23.220066 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.219962 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvbgb\" (UniqueName: \"kubernetes.io/projected/3c785c79-caab-473c-9aa3-10e177f1bf0d-kube-api-access-jvbgb\") pod \"telemeter-client-784749b4b9-vsm6m\" (UID: \"3c785c79-caab-473c-9aa3-10e177f1bf0d\") " pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" Sep 18 03:37:23.220066 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.219995 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3c785c79-caab-473c-9aa3-10e177f1bf0d-federate-client-tls\") pod \"telemeter-client-784749b4b9-vsm6m\" (UID: \"3c785c79-caab-473c-9aa3-10e177f1bf0d\") " pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" Sep 18 03:37:23.220066 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.220030 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c785c79-caab-473c-9aa3-10e177f1bf0d-serving-certs-ca-bundle\") pod \"telemeter-client-784749b4b9-vsm6m\" (UID: \"3c785c79-caab-473c-9aa3-10e177f1bf0d\") " pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" Sep 18 03:37:23.220187 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.220090 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c785c79-caab-473c-9aa3-10e177f1bf0d-metrics-client-ca\") pod \"telemeter-client-784749b4b9-vsm6m\" (UID: \"3c785c79-caab-473c-9aa3-10e177f1bf0d\") " pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" Sep 18 03:37:23.220187 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.220143 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c785c79-caab-473c-9aa3-10e177f1bf0d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-784749b4b9-vsm6m\" (UID: \"3c785c79-caab-473c-9aa3-10e177f1bf0d\") " pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" Sep 18 03:37:23.220274 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.220196 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3c785c79-caab-473c-9aa3-10e177f1bf0d-secret-telemeter-client\") pod \"telemeter-client-784749b4b9-vsm6m\" (UID: \"3c785c79-caab-473c-9aa3-10e177f1bf0d\") " pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" Sep 18 03:37:23.220850 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.220813 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c785c79-caab-473c-9aa3-10e177f1bf0d-serving-certs-ca-bundle\") pod \"telemeter-client-784749b4b9-vsm6m\" (UID: \"3c785c79-caab-473c-9aa3-10e177f1bf0d\") " pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" Sep 18 03:37:23.220850 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.220838 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c785c79-caab-473c-9aa3-10e177f1bf0d-metrics-client-ca\") pod \"telemeter-client-784749b4b9-vsm6m\" (UID: \"3c785c79-caab-473c-9aa3-10e177f1bf0d\") " pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" Sep 18 03:37:23.221086 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.221058 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c785c79-caab-473c-9aa3-10e177f1bf0d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-784749b4b9-vsm6m\" (UID: \"3c785c79-caab-473c-9aa3-10e177f1bf0d\") " pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" Sep 18 03:37:23.222694 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.222672 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3c785c79-caab-473c-9aa3-10e177f1bf0d-federate-client-tls\") pod \"telemeter-client-784749b4b9-vsm6m\" (UID: \"3c785c79-caab-473c-9aa3-10e177f1bf0d\") " pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" Sep 18 03:37:23.222771 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.222697 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3c785c79-caab-473c-9aa3-10e177f1bf0d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-784749b4b9-vsm6m\" (UID: \"3c785c79-caab-473c-9aa3-10e177f1bf0d\") " pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" Sep 18 03:37:23.222771 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.222730 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3c785c79-caab-473c-9aa3-10e177f1bf0d-telemeter-client-tls\") pod \"telemeter-client-784749b4b9-vsm6m\" (UID: \"3c785c79-caab-473c-9aa3-10e177f1bf0d\") " pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" Sep 18 03:37:23.222841 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.222777 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3c785c79-caab-473c-9aa3-10e177f1bf0d-secret-telemeter-client\") pod \"telemeter-client-784749b4b9-vsm6m\" (UID: \"3c785c79-caab-473c-9aa3-10e177f1bf0d\") " pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" Sep 18 03:37:23.227621 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.227595 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvbgb\" (UniqueName: \"kubernetes.io/projected/3c785c79-caab-473c-9aa3-10e177f1bf0d-kube-api-access-jvbgb\") pod \"telemeter-client-784749b4b9-vsm6m\" (UID: \"3c785c79-caab-473c-9aa3-10e177f1bf0d\") " pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" Sep 18 03:37:23.296984 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.296953 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" Sep 18 03:37:23.321388 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.321345 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/130cb992-f186-4a31-9d7d-f75af513570f-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-qv7cp\" (UID: \"130cb992-f186-4a31-9d7d-f75af513570f\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-qv7cp" Sep 18 03:37:23.321535 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:37:23.321515 2568 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Sep 18 03:37:23.321589 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:37:23.321578 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/130cb992-f186-4a31-9d7d-f75af513570f-monitoring-plugin-cert podName:130cb992-f186-4a31-9d7d-f75af513570f nodeName:}" failed. No retries permitted until 2026-09-18 03:37:24.321562166 +0000 UTC m=+258.604324007 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/130cb992-f186-4a31-9d7d-f75af513570f-monitoring-plugin-cert") pod "monitoring-plugin-5984459ff8-qv7cp" (UID: "130cb992-f186-4a31-9d7d-f75af513570f") : secret "monitoring-plugin-cert" not found Sep 18 03:37:23.576558 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.576529 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-784749b4b9-vsm6m"] Sep 18 03:37:23.577294 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:37:23.577270 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c785c79_caab_473c_9aa3_10e177f1bf0d.slice/crio-4e5cf2fc7990da98be244b29ec5dd0fd3071e0cbed34bbf50a085d5848f30333 WatchSource:0}: Error finding container 4e5cf2fc7990da98be244b29ec5dd0fd3071e0cbed34bbf50a085d5848f30333: Status 404 returned error can't find the container with id 4e5cf2fc7990da98be244b29ec5dd0fd3071e0cbed34bbf50a085d5848f30333 Sep 18 03:37:23.814182 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.814110 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" event={"ID":"3c785c79-caab-473c-9aa3-10e177f1bf0d","Type":"ContainerStarted","Data":"4e5cf2fc7990da98be244b29ec5dd0fd3071e0cbed34bbf50a085d5848f30333"} Sep 18 03:37:23.915506 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.915466 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 18 03:37:23.921441 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.921416 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:23.923800 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.923770 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 18 03:37:23.923914 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.923842 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 18 03:37:23.923914 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.923873 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6flsz\"" Sep 18 03:37:23.924201 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.924182 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 18 03:37:23.924679 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.924664 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 18 03:37:23.925012 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.924988 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7gbfnvf2vkk7j\"" Sep 18 03:37:23.925132 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.925117 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 18 03:37:23.925429 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.925409 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 18 03:37:23.925572 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.924721 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 18 03:37:23.926290 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.925135 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 18 03:37:23.926406 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.926387 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 18 03:37:23.926475 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.926425 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 18 03:37:23.926859 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.926825 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 18 03:37:23.926964 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.926826 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 18 03:37:23.926964 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.926957 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 18 03:37:23.927869 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.927851 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 18 03:37:23.930794 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.930772 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 18 03:37:23.935706 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:23.935685 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 18 03:37:24.027844 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.027810 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.027844 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.027843 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.028089 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.027863 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.028089 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.027897 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9ec6bf0d-7887-4c65-a3cb-1eb074138078-config-out\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.028089 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.027918 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.028089 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.028022 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.028089 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.028066 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-web-config\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.028296 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.028095 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.028296 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.028125 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.028296 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.028164 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.028296 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.028183 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.028296 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.028206 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.028296 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.028241 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.028296 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.028260 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.028529 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.028392 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9ec6bf0d-7887-4c65-a3cb-1eb074138078-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.028529 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.028410 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-config\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.028529 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.028429 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.028529 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.028452 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mr7jw\" (UniqueName: \"kubernetes.io/projected/9ec6bf0d-7887-4c65-a3cb-1eb074138078-kube-api-access-mr7jw\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.028663 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.028569 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.028663 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.028598 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.129451 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.129419 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.129451 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.129457 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-web-config\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.129679 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.129488 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.129679 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.129507 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.129679 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.129527 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.129679 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.129549 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.129679 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.129565 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.129679 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.129591 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.129679 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.129615 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.129679 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.129642 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9ec6bf0d-7887-4c65-a3cb-1eb074138078-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.129679 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.129661 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-config\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.130165 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.129682 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.130165 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.129703 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr7jw\" (UniqueName: \"kubernetes.io/projected/9ec6bf0d-7887-4c65-a3cb-1eb074138078-kube-api-access-mr7jw\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.130165 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.129747 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.130165 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.129766 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.130165 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.129786 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.130165 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.129802 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.130165 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.129816 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.130165 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.129853 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9ec6bf0d-7887-4c65-a3cb-1eb074138078-config-out\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.130165 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.129881 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.130776 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.130750 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.130925 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.130902 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.131467 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.131440 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.132004 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.131981 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.132117 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.132099 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.133261 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.133232 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.133357 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.133235 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.135343 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.135304 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.138760 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.135739 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9ec6bf0d-7887-4c65-a3cb-1eb074138078-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.138760 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.136890 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.138760 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.136909 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-config\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.138760 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.137213 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9ec6bf0d-7887-4c65-a3cb-1eb074138078-config-out\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.138760 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.137536 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.139286 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.139257 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mr7jw\" (UniqueName: \"kubernetes.io/projected/9ec6bf0d-7887-4c65-a3cb-1eb074138078-kube-api-access-mr7jw\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.139519 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.139455 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-web-config\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.139903 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.139713 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.140430 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.140381 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.140530 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.140508 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.141902 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.141870 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.143494 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.143470 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.236197 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.236087 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:24.334037 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.334003 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/130cb992-f186-4a31-9d7d-f75af513570f-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-qv7cp\" (UID: \"130cb992-f186-4a31-9d7d-f75af513570f\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-qv7cp" Sep 18 03:37:24.337144 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.337122 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/130cb992-f186-4a31-9d7d-f75af513570f-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-qv7cp\" (UID: \"130cb992-f186-4a31-9d7d-f75af513570f\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-qv7cp" Sep 18 03:37:24.353126 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.352892 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-qv7cp" Sep 18 03:37:24.536131 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.536044 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 18 03:37:24.539295 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:37:24.539265 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ec6bf0d_7887_4c65_a3cb_1eb074138078.slice/crio-fcf525a90618b4a19987cbc74c0284c84fa5f5780a0f13155197d2d6269b74be WatchSource:0}: Error finding container fcf525a90618b4a19987cbc74c0284c84fa5f5780a0f13155197d2d6269b74be: Status 404 returned error can't find the container with id fcf525a90618b4a19987cbc74c0284c84fa5f5780a0f13155197d2d6269b74be Sep 18 03:37:24.628897 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.628874 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-qv7cp"] Sep 18 03:37:24.631653 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:37:24.631625 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod130cb992_f186_4a31_9d7d_f75af513570f.slice/crio-57f1012388cfe75dd7b22cba8687d26aa519dbc1a7e735e272187d7185911fff WatchSource:0}: Error finding container 57f1012388cfe75dd7b22cba8687d26aa519dbc1a7e735e272187d7185911fff: Status 404 returned error can't find the container with id 57f1012388cfe75dd7b22cba8687d26aa519dbc1a7e735e272187d7185911fff Sep 18 03:37:24.820079 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.819973 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-qv7cp" event={"ID":"130cb992-f186-4a31-9d7d-f75af513570f","Type":"ContainerStarted","Data":"57f1012388cfe75dd7b22cba8687d26aa519dbc1a7e735e272187d7185911fff"} Sep 18 03:37:24.821602 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:24.821574 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9ec6bf0d-7887-4c65-a3cb-1eb074138078","Type":"ContainerStarted","Data":"fcf525a90618b4a19987cbc74c0284c84fa5f5780a0f13155197d2d6269b74be"} Sep 18 03:37:25.829260 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:25.829216 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" event={"ID":"3c785c79-caab-473c-9aa3-10e177f1bf0d","Type":"ContainerStarted","Data":"3ca985252a902de90a1e68e900f65158dc0de12fdfe28608b2f69b389e4858ea"} Sep 18 03:37:26.835117 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:26.835030 2568 generic.go:356] "Generic (PLEG): container finished" podID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerID="17a62549a3324fa7ff21ec2b94d11bd64a52c25f75ac7f559ccfbb6f006e634f" exitCode=0 Sep 18 03:37:26.835532 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:26.835125 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9ec6bf0d-7887-4c65-a3cb-1eb074138078","Type":"ContainerDied","Data":"17a62549a3324fa7ff21ec2b94d11bd64a52c25f75ac7f559ccfbb6f006e634f"} Sep 18 03:37:26.837446 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:26.837418 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" event={"ID":"3c785c79-caab-473c-9aa3-10e177f1bf0d","Type":"ContainerStarted","Data":"a2f5c41497fd14d7d44fd576a3d970a634787f7116c463fa3f6522d72ead8783"} Sep 18 03:37:26.837553 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:26.837472 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" event={"ID":"3c785c79-caab-473c-9aa3-10e177f1bf0d","Type":"ContainerStarted","Data":"73f82adb8e6c60668a0f4cb0c58f82b025f2a6ed318e3738bba99ae9113b2f55"} Sep 18 03:37:26.838853 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:26.838829 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-qv7cp" event={"ID":"130cb992-f186-4a31-9d7d-f75af513570f","Type":"ContainerStarted","Data":"56eb59a0f08ebc807f742f58c87cfd852b1098ee33d225a7a0a61cd4ac0a0437"} Sep 18 03:37:26.839003 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:26.838987 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-qv7cp" Sep 18 03:37:26.843840 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:26.843822 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-qv7cp" Sep 18 03:37:26.874781 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:26.874733 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5984459ff8-qv7cp" podStartSLOduration=3.040215941 podStartE2EDuration="4.874719359s" totalImagesPullingTime="1.834503418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:37:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:37:26.873329468 +0000 UTC m=+261.156091333" watchObservedRunningTime="2026-09-18 03:37:26.874719359 +0000 UTC m=+261.157481221" Sep 18 03:37:26.893172 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:26.893126 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-784749b4b9-vsm6m" podStartSLOduration=2.062196896 podStartE2EDuration="4.893112569s" totalImagesPullingTime="2.830915673s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:37:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:37:26.891298134 +0000 UTC m=+261.174060030" watchObservedRunningTime="2026-09-18 03:37:26.893112569 +0000 UTC m=+261.175874432" Sep 18 03:37:30.191100 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:30.191071 2568 patch_prober.go:28] interesting pod/image-registry-6d95fdc8fb-hgkfh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 18 03:37:30.191397 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:30.191131 2568 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" podUID="212ab307-3717-41ab-93e2-1a92ae45fecf" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 18 03:37:30.858745 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:30.858710 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9ec6bf0d-7887-4c65-a3cb-1eb074138078","Type":"ContainerStarted","Data":"b34af59704424cdbc2d30e6379d9d70c11ba05330ea47517066d7d4618ff24e5"} Sep 18 03:37:30.858994 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:30.858775 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9ec6bf0d-7887-4c65-a3cb-1eb074138078","Type":"ContainerStarted","Data":"6eb97489c468861ed41ad3fb60f14b749394fc9b91aab4147f66e4f590937da7"} Sep 18 03:37:31.749184 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:31.749138 2568 patch_prober.go:28] interesting pod/image-registry-6d95fdc8fb-hgkfh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 18 03:37:31.749576 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:31.749205 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" podUID="212ab307-3717-41ab-93e2-1a92ae45fecf" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 18 03:37:32.869576 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:32.869540 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9ec6bf0d-7887-4c65-a3cb-1eb074138078","Type":"ContainerStarted","Data":"690ece6c24e0057db1242153dec036d740b15114cdc035c2e26f2cc7d40e981e"} Sep 18 03:37:32.870028 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:32.869603 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9ec6bf0d-7887-4c65-a3cb-1eb074138078","Type":"ContainerStarted","Data":"9814146e45fa7112cb0e83b7b2eaf588f3f9f68d8be98b396348b5f4e340d0c2"} Sep 18 03:37:32.870028 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:32.869630 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9ec6bf0d-7887-4c65-a3cb-1eb074138078","Type":"ContainerStarted","Data":"4973d4403b61953a374feaee9c78ed1f7cb699e19d2342b48f6d8912230e5d7b"} Sep 18 03:37:32.870028 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:32.869651 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9ec6bf0d-7887-4c65-a3cb-1eb074138078","Type":"ContainerStarted","Data":"3910bd6695fc44eded9d6323ec0d8ce4eff503a142af39d9e37680e4f1b2521a"} Sep 18 03:37:32.903379 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:32.902665 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.609635231 podStartE2EDuration="9.902644852s" totalImagesPullingTime="7.26475285s" totalInitContainerRuntime="28.256771ms" isStatefulPod=false podCreationTimestamp="2026-09-18 03:37:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:37:32.900422314 +0000 UTC m=+267.183184187" watchObservedRunningTime="2026-09-18 03:37:32.902644852 +0000 UTC m=+267.185406719" Sep 18 03:37:34.236585 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:34.236551 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:37:40.190546 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:40.190506 2568 patch_prober.go:28] interesting pod/image-registry-6d95fdc8fb-hgkfh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 18 03:37:40.190922 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:40.190561 2568 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" podUID="212ab307-3717-41ab-93e2-1a92ae45fecf" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 18 03:37:41.746540 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:41.746508 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6d95fdc8fb-hgkfh" Sep 18 03:37:47.482993 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:37:47.482922 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-dwxzz" podUID="d5d06eb4-28cf-4325-8e91-19c6ee8b5b75" Sep 18 03:37:47.926845 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:47.926816 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dwxzz" Sep 18 03:37:51.172386 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:51.172351 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls\") pod \"dns-default-dwxzz\" (UID: \"d5d06eb4-28cf-4325-8e91-19c6ee8b5b75\") " pod="openshift-dns/dns-default-dwxzz" Sep 18 03:37:51.174633 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:51.174608 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5d06eb4-28cf-4325-8e91-19c6ee8b5b75-metrics-tls\") pod \"dns-default-dwxzz\" (UID: \"d5d06eb4-28cf-4325-8e91-19c6ee8b5b75\") " pod="openshift-dns/dns-default-dwxzz" Sep 18 03:37:51.229679 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:51.229657 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4fvk6\"" Sep 18 03:37:51.238457 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:51.238436 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dwxzz" Sep 18 03:37:51.508498 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:51.508476 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dwxzz"] Sep 18 03:37:51.510488 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:37:51.510457 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5d06eb4_28cf_4325_8e91_19c6ee8b5b75.slice/crio-84477ff1156f690ae7c8501c943675a3c49d57f42df69db615175f3685af396b WatchSource:0}: Error finding container 84477ff1156f690ae7c8501c943675a3c49d57f42df69db615175f3685af396b: Status 404 returned error can't find the container with id 84477ff1156f690ae7c8501c943675a3c49d57f42df69db615175f3685af396b Sep 18 03:37:51.944084 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:51.944027 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dwxzz" event={"ID":"d5d06eb4-28cf-4325-8e91-19c6ee8b5b75","Type":"ContainerStarted","Data":"84477ff1156f690ae7c8501c943675a3c49d57f42df69db615175f3685af396b"} Sep 18 03:37:53.954274 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:53.954248 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dwxzz" event={"ID":"d5d06eb4-28cf-4325-8e91-19c6ee8b5b75","Type":"ContainerStarted","Data":"8f2833236dfcb358d392629c5ff1a5823f66f361f6c00f309a0f960511942815"} Sep 18 03:37:53.954650 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:53.954293 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dwxzz" event={"ID":"d5d06eb4-28cf-4325-8e91-19c6ee8b5b75","Type":"ContainerStarted","Data":"10dd0f3fb320d9cdaa526fe81212ec6707759ea6f6a69709d686a5c61eee8dc6"} Sep 18 03:37:53.954650 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:53.954330 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dwxzz" Sep 18 03:37:53.971283 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:37:53.971236 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dwxzz" podStartSLOduration=251.380712709 podStartE2EDuration="4m12.971222601s" totalImagesPullingTime="1.590509892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:33:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:37:53.970134197 +0000 UTC m=+288.252896061" watchObservedRunningTime="2026-09-18 03:37:53.971222601 +0000 UTC m=+288.253984476" Sep 18 03:38:03.961645 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:03.961615 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dwxzz" Sep 18 03:38:07.787996 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:07.787969 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psksv_1013fa23-5170-4d1f-8722-86bc4b8b8742/ovn-acl-logging/0.log" Sep 18 03:38:07.793185 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:07.793161 2568 kubelet.go:1648] "Image garbage collection succeeded" Sep 18 03:38:07.794632 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:07.794615 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psksv_1013fa23-5170-4d1f-8722-86bc4b8b8742/ovn-acl-logging/0.log" Sep 18 03:38:12.027134 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:12.027099 2568 generic.go:356] "Generic (PLEG): container finished" podID="afa0807e-f31c-4d12-ab3a-35e9be7066f5" containerID="2318418607f90dbffc72ff808dd2ef5e4a6e6f6a8fd8bcd4498487424a46beb6" exitCode=0 Sep 18 03:38:12.027532 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:12.027171 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-p6bc5" event={"ID":"afa0807e-f31c-4d12-ab3a-35e9be7066f5","Type":"ContainerDied","Data":"2318418607f90dbffc72ff808dd2ef5e4a6e6f6a8fd8bcd4498487424a46beb6"} Sep 18 03:38:12.027532 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:12.027476 2568 scope.go:117] "RemoveContainer" containerID="2318418607f90dbffc72ff808dd2ef5e4a6e6f6a8fd8bcd4498487424a46beb6" Sep 18 03:38:13.033222 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:13.033190 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-p6bc5" event={"ID":"afa0807e-f31c-4d12-ab3a-35e9be7066f5","Type":"ContainerStarted","Data":"f2933238e0014b1985c677b3dcedb41231a2307996197b37b85e0a86c1dc9356"} Sep 18 03:38:24.236497 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:24.236464 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:24.257060 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:24.257037 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:25.099774 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:25.099748 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:42.277042 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:42.277001 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 18 03:38:42.277628 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:42.277407 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerName="prometheus" containerID="cri-o://6eb97489c468861ed41ad3fb60f14b749394fc9b91aab4147f66e4f590937da7" gracePeriod=600 Sep 18 03:38:42.278194 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:42.277803 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerName="kube-rbac-proxy-web" containerID="cri-o://4973d4403b61953a374feaee9c78ed1f7cb699e19d2342b48f6d8912230e5d7b" gracePeriod=600 Sep 18 03:38:42.278194 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:42.277842 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerName="config-reloader" containerID="cri-o://b34af59704424cdbc2d30e6379d9d70c11ba05330ea47517066d7d4618ff24e5" gracePeriod=600 Sep 18 03:38:42.278194 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:42.277825 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerName="kube-rbac-proxy-thanos" containerID="cri-o://690ece6c24e0057db1242153dec036d740b15114cdc035c2e26f2cc7d40e981e" gracePeriod=600 Sep 18 03:38:42.278194 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:42.277824 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerName="kube-rbac-proxy" containerID="cri-o://9814146e45fa7112cb0e83b7b2eaf588f3f9f68d8be98b396348b5f4e340d0c2" gracePeriod=600 Sep 18 03:38:42.278194 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:42.277945 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerName="thanos-sidecar" containerID="cri-o://3910bd6695fc44eded9d6323ec0d8ce4eff503a142af39d9e37680e4f1b2521a" gracePeriod=600 Sep 18 03:38:43.158083 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.158042 2568 generic.go:356] "Generic (PLEG): container finished" podID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerID="690ece6c24e0057db1242153dec036d740b15114cdc035c2e26f2cc7d40e981e" exitCode=0 Sep 18 03:38:43.158083 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.158076 2568 generic.go:356] "Generic (PLEG): container finished" podID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerID="9814146e45fa7112cb0e83b7b2eaf588f3f9f68d8be98b396348b5f4e340d0c2" exitCode=0 Sep 18 03:38:43.158083 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.158084 2568 generic.go:356] "Generic (PLEG): container finished" podID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerID="3910bd6695fc44eded9d6323ec0d8ce4eff503a142af39d9e37680e4f1b2521a" exitCode=0 Sep 18 03:38:43.158083 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.158089 2568 generic.go:356] "Generic (PLEG): container finished" podID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerID="b34af59704424cdbc2d30e6379d9d70c11ba05330ea47517066d7d4618ff24e5" exitCode=0 Sep 18 03:38:43.158328 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.158095 2568 generic.go:356] "Generic (PLEG): container finished" podID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerID="6eb97489c468861ed41ad3fb60f14b749394fc9b91aab4147f66e4f590937da7" exitCode=0 Sep 18 03:38:43.158328 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.158111 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9ec6bf0d-7887-4c65-a3cb-1eb074138078","Type":"ContainerDied","Data":"690ece6c24e0057db1242153dec036d740b15114cdc035c2e26f2cc7d40e981e"} Sep 18 03:38:43.158328 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.158174 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9ec6bf0d-7887-4c65-a3cb-1eb074138078","Type":"ContainerDied","Data":"9814146e45fa7112cb0e83b7b2eaf588f3f9f68d8be98b396348b5f4e340d0c2"} Sep 18 03:38:43.158328 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.158199 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9ec6bf0d-7887-4c65-a3cb-1eb074138078","Type":"ContainerDied","Data":"3910bd6695fc44eded9d6323ec0d8ce4eff503a142af39d9e37680e4f1b2521a"} Sep 18 03:38:43.158328 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.158222 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9ec6bf0d-7887-4c65-a3cb-1eb074138078","Type":"ContainerDied","Data":"b34af59704424cdbc2d30e6379d9d70c11ba05330ea47517066d7d4618ff24e5"} Sep 18 03:38:43.158328 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.158245 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9ec6bf0d-7887-4c65-a3cb-1eb074138078","Type":"ContainerDied","Data":"6eb97489c468861ed41ad3fb60f14b749394fc9b91aab4147f66e4f590937da7"} Sep 18 03:38:43.517904 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.517876 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:43.591833 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.591799 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-k8s-db\") pod \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " Sep 18 03:38:43.591833 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.591836 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-metrics-client-certs\") pod \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " Sep 18 03:38:43.592059 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.591862 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-kube-rbac-proxy\") pod \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " Sep 18 03:38:43.592059 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.591895 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-configmap-kubelet-serving-ca-bundle\") pod \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " Sep 18 03:38:43.592059 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.592001 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-k8s-rulefiles-0\") pod \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " Sep 18 03:38:43.592221 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.592078 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " Sep 18 03:38:43.592221 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.592109 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " Sep 18 03:38:43.592221 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.592140 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mr7jw\" (UniqueName: \"kubernetes.io/projected/9ec6bf0d-7887-4c65-a3cb-1eb074138078-kube-api-access-mr7jw\") pod \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " Sep 18 03:38:43.592221 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.592173 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-configmap-metrics-client-ca\") pod \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " Sep 18 03:38:43.592221 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.592199 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-k8s-rulefiles-2\") pod \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " Sep 18 03:38:43.592461 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.592233 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-config\") pod \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " Sep 18 03:38:43.592461 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.592263 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-configmap-serving-certs-ca-bundle\") pod \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " Sep 18 03:38:43.592461 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.592293 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-thanos-prometheus-http-client-file\") pod \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " Sep 18 03:38:43.592461 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.592291 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "9ec6bf0d-7887-4c65-a3cb-1eb074138078" (UID: "9ec6bf0d-7887-4c65-a3cb-1eb074138078"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 03:38:43.592461 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.592320 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-grpc-tls\") pod \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " Sep 18 03:38:43.592461 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.592351 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-k8s-rulefiles-1\") pod \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " Sep 18 03:38:43.592461 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.592375 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-prometheus-k8s-tls\") pod \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " Sep 18 03:38:43.592461 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.592441 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-trusted-ca-bundle\") pod \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " Sep 18 03:38:43.592822 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.592471 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9ec6bf0d-7887-4c65-a3cb-1eb074138078-config-out\") pod \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " Sep 18 03:38:43.592822 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.592494 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9ec6bf0d-7887-4c65-a3cb-1eb074138078-tls-assets\") pod \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " Sep 18 03:38:43.592822 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.592524 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-web-config\") pod \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\" (UID: \"9ec6bf0d-7887-4c65-a3cb-1eb074138078\") " Sep 18 03:38:43.592822 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.592809 2568 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:38:43.593050 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.592885 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "9ec6bf0d-7887-4c65-a3cb-1eb074138078" (UID: "9ec6bf0d-7887-4c65-a3cb-1eb074138078"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 03:38:43.594722 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.593615 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "9ec6bf0d-7887-4c65-a3cb-1eb074138078" (UID: "9ec6bf0d-7887-4c65-a3cb-1eb074138078"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 03:38:43.594722 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.593731 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "9ec6bf0d-7887-4c65-a3cb-1eb074138078" (UID: "9ec6bf0d-7887-4c65-a3cb-1eb074138078"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 03:38:43.594722 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.594309 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "9ec6bf0d-7887-4c65-a3cb-1eb074138078" (UID: "9ec6bf0d-7887-4c65-a3cb-1eb074138078"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 03:38:43.594722 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.594587 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "9ec6bf0d-7887-4c65-a3cb-1eb074138078" (UID: "9ec6bf0d-7887-4c65-a3cb-1eb074138078"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:38:43.594722 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.594632 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "9ec6bf0d-7887-4c65-a3cb-1eb074138078" (UID: "9ec6bf0d-7887-4c65-a3cb-1eb074138078"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 03:38:43.595050 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.594925 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "9ec6bf0d-7887-4c65-a3cb-1eb074138078" (UID: "9ec6bf0d-7887-4c65-a3cb-1eb074138078"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 03:38:43.595050 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.594993 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "9ec6bf0d-7887-4c65-a3cb-1eb074138078" (UID: "9ec6bf0d-7887-4c65-a3cb-1eb074138078"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 03:38:43.596043 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.596003 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "9ec6bf0d-7887-4c65-a3cb-1eb074138078" (UID: "9ec6bf0d-7887-4c65-a3cb-1eb074138078"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:38:43.597067 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.597037 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "9ec6bf0d-7887-4c65-a3cb-1eb074138078" (UID: "9ec6bf0d-7887-4c65-a3cb-1eb074138078"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:38:43.597185 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.597067 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "9ec6bf0d-7887-4c65-a3cb-1eb074138078" (UID: "9ec6bf0d-7887-4c65-a3cb-1eb074138078"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:38:43.597343 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.597323 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ec6bf0d-7887-4c65-a3cb-1eb074138078-kube-api-access-mr7jw" (OuterVolumeSpecName: "kube-api-access-mr7jw") pod "9ec6bf0d-7887-4c65-a3cb-1eb074138078" (UID: "9ec6bf0d-7887-4c65-a3cb-1eb074138078"). InnerVolumeSpecName "kube-api-access-mr7jw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 03:38:43.597483 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.597465 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ec6bf0d-7887-4c65-a3cb-1eb074138078-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9ec6bf0d-7887-4c65-a3cb-1eb074138078" (UID: "9ec6bf0d-7887-4c65-a3cb-1eb074138078"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 03:38:43.597747 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.597725 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "9ec6bf0d-7887-4c65-a3cb-1eb074138078" (UID: "9ec6bf0d-7887-4c65-a3cb-1eb074138078"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:38:43.598201 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.598169 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "9ec6bf0d-7887-4c65-a3cb-1eb074138078" (UID: "9ec6bf0d-7887-4c65-a3cb-1eb074138078"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:38:43.598680 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.598656 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ec6bf0d-7887-4c65-a3cb-1eb074138078-config-out" (OuterVolumeSpecName: "config-out") pod "9ec6bf0d-7887-4c65-a3cb-1eb074138078" (UID: "9ec6bf0d-7887-4c65-a3cb-1eb074138078"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 03:38:43.598680 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.598671 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "9ec6bf0d-7887-4c65-a3cb-1eb074138078" (UID: "9ec6bf0d-7887-4c65-a3cb-1eb074138078"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:38:43.599029 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.599004 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-config" (OuterVolumeSpecName: "config") pod "9ec6bf0d-7887-4c65-a3cb-1eb074138078" (UID: "9ec6bf0d-7887-4c65-a3cb-1eb074138078"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:38:43.608434 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.608414 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-web-config" (OuterVolumeSpecName: "web-config") pod "9ec6bf0d-7887-4c65-a3cb-1eb074138078" (UID: "9ec6bf0d-7887-4c65-a3cb-1eb074138078"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:38:43.693450 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.693401 2568 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:38:43.693450 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.693421 2568 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:38:43.693450 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.693432 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mr7jw\" (UniqueName: \"kubernetes.io/projected/9ec6bf0d-7887-4c65-a3cb-1eb074138078-kube-api-access-mr7jw\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:38:43.693450 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.693440 2568 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-configmap-metrics-client-ca\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:38:43.693450 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.693450 2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:38:43.693643 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.693458 2568 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-config\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:38:43.693643 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.693466 2568 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:38:43.693643 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.693474 2568 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-thanos-prometheus-http-client-file\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:38:43.693643 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.693482 2568 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-grpc-tls\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:38:43.693643 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.693490 2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:38:43.693643 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.693497 2568 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-prometheus-k8s-tls\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:38:43.693643 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.693506 2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-trusted-ca-bundle\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:38:43.693643 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.693514 2568 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9ec6bf0d-7887-4c65-a3cb-1eb074138078-config-out\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:38:43.693643 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.693522 2568 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9ec6bf0d-7887-4c65-a3cb-1eb074138078-tls-assets\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:38:43.693643 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.693529 2568 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-web-config\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:38:43.693643 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.693536 2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-k8s-db\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:38:43.693643 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.693544 2568 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-metrics-client-certs\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:38:43.693643 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.693552 2568 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ec6bf0d-7887-4c65-a3cb-1eb074138078-secret-kube-rbac-proxy\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:38:43.693643 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:43.693560 2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9ec6bf0d-7887-4c65-a3cb-1eb074138078-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:38:44.164532 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.164502 2568 generic.go:356] "Generic (PLEG): container finished" podID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerID="4973d4403b61953a374feaee9c78ed1f7cb699e19d2342b48f6d8912230e5d7b" exitCode=0 Sep 18 03:38:44.164678 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.164573 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.164678 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.164575 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9ec6bf0d-7887-4c65-a3cb-1eb074138078","Type":"ContainerDied","Data":"4973d4403b61953a374feaee9c78ed1f7cb699e19d2342b48f6d8912230e5d7b"} Sep 18 03:38:44.164678 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.164644 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9ec6bf0d-7887-4c65-a3cb-1eb074138078","Type":"ContainerDied","Data":"fcf525a90618b4a19987cbc74c0284c84fa5f5780a0f13155197d2d6269b74be"} Sep 18 03:38:44.164678 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.164679 2568 scope.go:117] "RemoveContainer" containerID="690ece6c24e0057db1242153dec036d740b15114cdc035c2e26f2cc7d40e981e" Sep 18 03:38:44.172348 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.172329 2568 scope.go:117] "RemoveContainer" containerID="9814146e45fa7112cb0e83b7b2eaf588f3f9f68d8be98b396348b5f4e340d0c2" Sep 18 03:38:44.178821 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.178801 2568 scope.go:117] "RemoveContainer" containerID="4973d4403b61953a374feaee9c78ed1f7cb699e19d2342b48f6d8912230e5d7b" Sep 18 03:38:44.185630 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.185613 2568 scope.go:117] "RemoveContainer" containerID="3910bd6695fc44eded9d6323ec0d8ce4eff503a142af39d9e37680e4f1b2521a" Sep 18 03:38:44.187520 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.187502 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 18 03:38:44.191082 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.191060 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 18 03:38:44.193009 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.192993 2568 scope.go:117] "RemoveContainer" containerID="b34af59704424cdbc2d30e6379d9d70c11ba05330ea47517066d7d4618ff24e5" Sep 18 03:38:44.199267 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.199251 2568 scope.go:117] "RemoveContainer" containerID="6eb97489c468861ed41ad3fb60f14b749394fc9b91aab4147f66e4f590937da7" Sep 18 03:38:44.206015 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.205996 2568 scope.go:117] "RemoveContainer" containerID="17a62549a3324fa7ff21ec2b94d11bd64a52c25f75ac7f559ccfbb6f006e634f" Sep 18 03:38:44.212461 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.212447 2568 scope.go:117] "RemoveContainer" containerID="690ece6c24e0057db1242153dec036d740b15114cdc035c2e26f2cc7d40e981e" Sep 18 03:38:44.212735 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:38:44.212714 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"690ece6c24e0057db1242153dec036d740b15114cdc035c2e26f2cc7d40e981e\": container with ID starting with 690ece6c24e0057db1242153dec036d740b15114cdc035c2e26f2cc7d40e981e not found: ID does not exist" containerID="690ece6c24e0057db1242153dec036d740b15114cdc035c2e26f2cc7d40e981e" Sep 18 03:38:44.212814 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.212740 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"690ece6c24e0057db1242153dec036d740b15114cdc035c2e26f2cc7d40e981e"} err="failed to get container status \"690ece6c24e0057db1242153dec036d740b15114cdc035c2e26f2cc7d40e981e\": rpc error: code = NotFound desc = could not find container \"690ece6c24e0057db1242153dec036d740b15114cdc035c2e26f2cc7d40e981e\": container with ID starting with 690ece6c24e0057db1242153dec036d740b15114cdc035c2e26f2cc7d40e981e not found: ID does not exist" Sep 18 03:38:44.212814 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.212767 2568 scope.go:117] "RemoveContainer" containerID="9814146e45fa7112cb0e83b7b2eaf588f3f9f68d8be98b396348b5f4e340d0c2" Sep 18 03:38:44.213042 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:38:44.213024 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9814146e45fa7112cb0e83b7b2eaf588f3f9f68d8be98b396348b5f4e340d0c2\": container with ID starting with 9814146e45fa7112cb0e83b7b2eaf588f3f9f68d8be98b396348b5f4e340d0c2 not found: ID does not exist" containerID="9814146e45fa7112cb0e83b7b2eaf588f3f9f68d8be98b396348b5f4e340d0c2" Sep 18 03:38:44.213105 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.213050 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9814146e45fa7112cb0e83b7b2eaf588f3f9f68d8be98b396348b5f4e340d0c2"} err="failed to get container status \"9814146e45fa7112cb0e83b7b2eaf588f3f9f68d8be98b396348b5f4e340d0c2\": rpc error: code = NotFound desc = could not find container \"9814146e45fa7112cb0e83b7b2eaf588f3f9f68d8be98b396348b5f4e340d0c2\": container with ID starting with 9814146e45fa7112cb0e83b7b2eaf588f3f9f68d8be98b396348b5f4e340d0c2 not found: ID does not exist" Sep 18 03:38:44.213105 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.213072 2568 scope.go:117] "RemoveContainer" containerID="4973d4403b61953a374feaee9c78ed1f7cb699e19d2342b48f6d8912230e5d7b" Sep 18 03:38:44.213306 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:38:44.213289 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4973d4403b61953a374feaee9c78ed1f7cb699e19d2342b48f6d8912230e5d7b\": container with ID starting with 4973d4403b61953a374feaee9c78ed1f7cb699e19d2342b48f6d8912230e5d7b not found: ID does not exist" containerID="4973d4403b61953a374feaee9c78ed1f7cb699e19d2342b48f6d8912230e5d7b" Sep 18 03:38:44.213341 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.213310 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4973d4403b61953a374feaee9c78ed1f7cb699e19d2342b48f6d8912230e5d7b"} err="failed to get container status \"4973d4403b61953a374feaee9c78ed1f7cb699e19d2342b48f6d8912230e5d7b\": rpc error: code = NotFound desc = could not find container \"4973d4403b61953a374feaee9c78ed1f7cb699e19d2342b48f6d8912230e5d7b\": container with ID starting with 4973d4403b61953a374feaee9c78ed1f7cb699e19d2342b48f6d8912230e5d7b not found: ID does not exist" Sep 18 03:38:44.213341 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.213323 2568 scope.go:117] "RemoveContainer" containerID="3910bd6695fc44eded9d6323ec0d8ce4eff503a142af39d9e37680e4f1b2521a" Sep 18 03:38:44.213535 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:38:44.213518 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3910bd6695fc44eded9d6323ec0d8ce4eff503a142af39d9e37680e4f1b2521a\": container with ID starting with 3910bd6695fc44eded9d6323ec0d8ce4eff503a142af39d9e37680e4f1b2521a not found: ID does not exist" containerID="3910bd6695fc44eded9d6323ec0d8ce4eff503a142af39d9e37680e4f1b2521a" Sep 18 03:38:44.213578 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.213540 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3910bd6695fc44eded9d6323ec0d8ce4eff503a142af39d9e37680e4f1b2521a"} err="failed to get container status \"3910bd6695fc44eded9d6323ec0d8ce4eff503a142af39d9e37680e4f1b2521a\": rpc error: code = NotFound desc = could not find container \"3910bd6695fc44eded9d6323ec0d8ce4eff503a142af39d9e37680e4f1b2521a\": container with ID starting with 3910bd6695fc44eded9d6323ec0d8ce4eff503a142af39d9e37680e4f1b2521a not found: ID does not exist" Sep 18 03:38:44.213578 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.213555 2568 scope.go:117] "RemoveContainer" containerID="b34af59704424cdbc2d30e6379d9d70c11ba05330ea47517066d7d4618ff24e5" Sep 18 03:38:44.213767 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:38:44.213750 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b34af59704424cdbc2d30e6379d9d70c11ba05330ea47517066d7d4618ff24e5\": container with ID starting with b34af59704424cdbc2d30e6379d9d70c11ba05330ea47517066d7d4618ff24e5 not found: ID does not exist" containerID="b34af59704424cdbc2d30e6379d9d70c11ba05330ea47517066d7d4618ff24e5" Sep 18 03:38:44.213826 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.213774 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b34af59704424cdbc2d30e6379d9d70c11ba05330ea47517066d7d4618ff24e5"} err="failed to get container status \"b34af59704424cdbc2d30e6379d9d70c11ba05330ea47517066d7d4618ff24e5\": rpc error: code = NotFound desc = could not find container \"b34af59704424cdbc2d30e6379d9d70c11ba05330ea47517066d7d4618ff24e5\": container with ID starting with b34af59704424cdbc2d30e6379d9d70c11ba05330ea47517066d7d4618ff24e5 not found: ID does not exist" Sep 18 03:38:44.213826 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.213793 2568 scope.go:117] "RemoveContainer" containerID="6eb97489c468861ed41ad3fb60f14b749394fc9b91aab4147f66e4f590937da7" Sep 18 03:38:44.214034 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:38:44.214013 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6eb97489c468861ed41ad3fb60f14b749394fc9b91aab4147f66e4f590937da7\": container with ID starting with 6eb97489c468861ed41ad3fb60f14b749394fc9b91aab4147f66e4f590937da7 not found: ID does not exist" containerID="6eb97489c468861ed41ad3fb60f14b749394fc9b91aab4147f66e4f590937da7" Sep 18 03:38:44.214110 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.214036 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eb97489c468861ed41ad3fb60f14b749394fc9b91aab4147f66e4f590937da7"} err="failed to get container status \"6eb97489c468861ed41ad3fb60f14b749394fc9b91aab4147f66e4f590937da7\": rpc error: code = NotFound desc = could not find container \"6eb97489c468861ed41ad3fb60f14b749394fc9b91aab4147f66e4f590937da7\": container with ID starting with 6eb97489c468861ed41ad3fb60f14b749394fc9b91aab4147f66e4f590937da7 not found: ID does not exist" Sep 18 03:38:44.214110 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.214048 2568 scope.go:117] "RemoveContainer" containerID="17a62549a3324fa7ff21ec2b94d11bd64a52c25f75ac7f559ccfbb6f006e634f" Sep 18 03:38:44.214278 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:38:44.214261 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17a62549a3324fa7ff21ec2b94d11bd64a52c25f75ac7f559ccfbb6f006e634f\": container with ID starting with 17a62549a3324fa7ff21ec2b94d11bd64a52c25f75ac7f559ccfbb6f006e634f not found: ID does not exist" containerID="17a62549a3324fa7ff21ec2b94d11bd64a52c25f75ac7f559ccfbb6f006e634f" Sep 18 03:38:44.214316 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.214280 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17a62549a3324fa7ff21ec2b94d11bd64a52c25f75ac7f559ccfbb6f006e634f"} err="failed to get container status \"17a62549a3324fa7ff21ec2b94d11bd64a52c25f75ac7f559ccfbb6f006e634f\": rpc error: code = NotFound desc = could not find container \"17a62549a3324fa7ff21ec2b94d11bd64a52c25f75ac7f559ccfbb6f006e634f\": container with ID starting with 17a62549a3324fa7ff21ec2b94d11bd64a52c25f75ac7f559ccfbb6f006e634f not found: ID does not exist" Sep 18 03:38:44.218798 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.218780 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 18 03:38:44.219078 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.219065 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerName="kube-rbac-proxy-web" Sep 18 03:38:44.219126 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.219080 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerName="kube-rbac-proxy-web" Sep 18 03:38:44.219126 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.219090 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerName="init-config-reloader" Sep 18 03:38:44.219126 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.219096 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerName="init-config-reloader" Sep 18 03:38:44.219126 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.219104 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerName="thanos-sidecar" Sep 18 03:38:44.219126 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.219110 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerName="thanos-sidecar" Sep 18 03:38:44.219126 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.219121 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerName="prometheus" Sep 18 03:38:44.219126 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.219126 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerName="prometheus" Sep 18 03:38:44.219319 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.219132 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerName="config-reloader" Sep 18 03:38:44.219319 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.219137 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerName="config-reloader" Sep 18 03:38:44.219319 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.219144 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerName="kube-rbac-proxy" Sep 18 03:38:44.219319 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.219148 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerName="kube-rbac-proxy" Sep 18 03:38:44.219319 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.219154 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerName="kube-rbac-proxy-thanos" Sep 18 03:38:44.219319 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.219159 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerName="kube-rbac-proxy-thanos" Sep 18 03:38:44.219319 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.219205 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerName="kube-rbac-proxy-thanos" Sep 18 03:38:44.219319 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.219212 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerName="prometheus" Sep 18 03:38:44.219319 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.219222 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerName="config-reloader" Sep 18 03:38:44.219319 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.219228 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerName="kube-rbac-proxy-web" Sep 18 03:38:44.219319 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.219233 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerName="thanos-sidecar" Sep 18 03:38:44.219319 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.219238 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" containerName="kube-rbac-proxy" Sep 18 03:38:44.224418 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.224399 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.226855 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.226834 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 18 03:38:44.226987 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.226841 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 18 03:38:44.227082 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.227062 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 18 03:38:44.227345 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.227275 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7gbfnvf2vkk7j\"" Sep 18 03:38:44.228001 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.227985 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 18 03:38:44.228001 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.227997 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 18 03:38:44.228001 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.227985 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 18 03:38:44.228235 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.228223 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 18 03:38:44.229791 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.229773 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6flsz\"" Sep 18 03:38:44.229888 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.229798 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 18 03:38:44.229888 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.229806 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 18 03:38:44.230026 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.229916 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 18 03:38:44.233461 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.233441 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 18 03:38:44.233541 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.233441 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 18 03:38:44.237641 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.237626 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 18 03:38:44.245059 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.245044 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 18 03:38:44.269763 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.269743 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 18 03:38:44.289624 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.289596 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 18 03:38:44.298812 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.298791 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.298913 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.298822 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-config-out\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.298913 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.298849 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct7gh\" (UniqueName: \"kubernetes.io/projected/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-kube-api-access-ct7gh\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.298913 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.298876 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.299147 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.298955 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.299147 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.298984 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.299147 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.299006 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.299147 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.299064 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.299147 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.299089 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.299147 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.299110 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.299147 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.299136 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.299477 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.299162 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.299477 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.299223 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.299477 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.299254 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-web-config\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.299477 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.299281 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.299477 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.299310 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-config\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.299477 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.299334 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.299477 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.299363 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.299477 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.299389 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.299477 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.299418 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.400117 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.400095 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.400217 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.400123 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-config-out\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.400217 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.400141 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct7gh\" (UniqueName: \"kubernetes.io/projected/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-kube-api-access-ct7gh\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.400217 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.400168 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.400367 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.400219 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.400367 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.400244 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.400367 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.400266 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.400367 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.400292 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.400367 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.400315 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.400367 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.400334 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.400667 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.400369 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.400667 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.400394 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.400667 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.400433 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.401115 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.401091 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.401213 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.401151 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-web-config\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.401213 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.401185 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.401324 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.401223 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-config\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.401324 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.401249 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.401324 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.401251 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.401324 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.401277 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.401324 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.401307 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.401558 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.401341 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.401665 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.401636 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.402274 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.402248 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.402999 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.402971 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-config-out\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.404050 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.403426 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.404050 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.403432 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.404050 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.403665 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.404050 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.403954 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.404050 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.403990 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.404330 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.404299 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.404537 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.404513 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.404690 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.404672 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-web-config\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.405417 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.405392 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.405670 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.405651 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.405730 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.405694 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.405824 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.405809 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-config\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.406691 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.406667 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.407256 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.407239 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.408912 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.408890 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct7gh\" (UniqueName: \"kubernetes.io/projected/d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d-kube-api-access-ct7gh\") pod \"prometheus-k8s-0\" (UID: \"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.534127 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.534060 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:38:44.812729 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:44.812703 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 18 03:38:44.814268 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:38:44.814243 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8e8f63f_c1a3_45d1_b3a5_01bbca654b6d.slice/crio-e7bb8bf545ec914a3d37db36bb7884a7c7136d268d5044348a22dcd5eb82dcc9 WatchSource:0}: Error finding container e7bb8bf545ec914a3d37db36bb7884a7c7136d268d5044348a22dcd5eb82dcc9: Status 404 returned error can't find the container with id e7bb8bf545ec914a3d37db36bb7884a7c7136d268d5044348a22dcd5eb82dcc9 Sep 18 03:38:45.170625 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:45.170599 2568 generic.go:356] "Generic (PLEG): container finished" podID="d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d" containerID="96cef3bc8d0ba4313beb3adc54333eabacf404a6d689617e2931b3818b6ed4f6" exitCode=0 Sep 18 03:38:45.170763 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:45.170630 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d","Type":"ContainerDied","Data":"96cef3bc8d0ba4313beb3adc54333eabacf404a6d689617e2931b3818b6ed4f6"} Sep 18 03:38:45.170763 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:45.170699 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d","Type":"ContainerStarted","Data":"e7bb8bf545ec914a3d37db36bb7884a7c7136d268d5044348a22dcd5eb82dcc9"} Sep 18 03:38:45.881982 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:45.881899 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ec6bf0d-7887-4c65-a3cb-1eb074138078" path="/var/lib/kubelet/pods/9ec6bf0d-7887-4c65-a3cb-1eb074138078/volumes" Sep 18 03:38:46.179795 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:46.179764 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d","Type":"ContainerStarted","Data":"7e70417f2d5f1ae3a4b718844c439d2786f28e1b680875c3f6886ae5d59566c5"} Sep 18 03:38:46.179996 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:46.179822 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d","Type":"ContainerStarted","Data":"e0d97f1e6ca484ced25da7f76c15b55d73dd1ec7755d061401ad9036ab94fbc6"} Sep 18 03:38:46.179996 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:46.179850 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d","Type":"ContainerStarted","Data":"73a7354cf7c7a1a948610b344547faee72f1a0979f06f7755f4b93f6514af641"} Sep 18 03:38:46.179996 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:46.179873 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d","Type":"ContainerStarted","Data":"e105ad1e52dc0f9238fcb7c286a477cf681256da58ea8701fabeb2fe08c59596"} Sep 18 03:38:46.179996 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:46.179895 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d","Type":"ContainerStarted","Data":"913c952267fa09cc7046150b7721d03040c71973e3ee98a4b383f1575f63fa91"} Sep 18 03:38:46.179996 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:46.179918 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d","Type":"ContainerStarted","Data":"d48280cd76141db7585db5aebcaf7a3356cb490765b38581eaf4abb4d31b9d24"} Sep 18 03:38:46.207005 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:46.206944 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.18827868 podStartE2EDuration="2.206908843s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.630163ms" isStatefulPod=false podCreationTimestamp="2026-09-18 03:38:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:38:46.204164399 +0000 UTC m=+340.486926262" watchObservedRunningTime="2026-09-18 03:38:46.206908843 +0000 UTC m=+340.489670710" Sep 18 03:38:49.534814 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:38:49.534767 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:39:44.534489 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:39:44.534452 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:39:44.550105 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:39:44.550074 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:39:45.433985 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:39:45.433957 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:40:22.291768 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:22.291688 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-wn7qq"] Sep 18 03:40:22.294815 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:22.294800 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wn7qq" Sep 18 03:40:22.298063 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:22.298047 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 18 03:40:22.298159 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:22.298046 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-98sq8\"" Sep 18 03:40:22.298159 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:22.298052 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 18 03:40:22.300261 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:22.300153 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs567\" (UniqueName: \"kubernetes.io/projected/c4bfdf00-9f36-4fbe-bd54-9c3fa5ba641d-kube-api-access-cs567\") pod \"cert-manager-cainjector-559cdd8c87-wn7qq\" (UID: \"c4bfdf00-9f36-4fbe-bd54-9c3fa5ba641d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wn7qq" Sep 18 03:40:22.300261 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:22.300202 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c4bfdf00-9f36-4fbe-bd54-9c3fa5ba641d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-wn7qq\" (UID: \"c4bfdf00-9f36-4fbe-bd54-9c3fa5ba641d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wn7qq" Sep 18 03:40:22.302085 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:22.302059 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-wn7qq"] Sep 18 03:40:22.400955 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:22.400907 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs567\" (UniqueName: \"kubernetes.io/projected/c4bfdf00-9f36-4fbe-bd54-9c3fa5ba641d-kube-api-access-cs567\") pod \"cert-manager-cainjector-559cdd8c87-wn7qq\" (UID: \"c4bfdf00-9f36-4fbe-bd54-9c3fa5ba641d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wn7qq" Sep 18 03:40:22.401123 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:22.400973 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c4bfdf00-9f36-4fbe-bd54-9c3fa5ba641d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-wn7qq\" (UID: \"c4bfdf00-9f36-4fbe-bd54-9c3fa5ba641d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wn7qq" Sep 18 03:40:22.408107 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:22.408076 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c4bfdf00-9f36-4fbe-bd54-9c3fa5ba641d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-wn7qq\" (UID: \"c4bfdf00-9f36-4fbe-bd54-9c3fa5ba641d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wn7qq" Sep 18 03:40:22.408253 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:22.408237 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs567\" (UniqueName: \"kubernetes.io/projected/c4bfdf00-9f36-4fbe-bd54-9c3fa5ba641d-kube-api-access-cs567\") pod \"cert-manager-cainjector-559cdd8c87-wn7qq\" (UID: \"c4bfdf00-9f36-4fbe-bd54-9c3fa5ba641d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wn7qq" Sep 18 03:40:22.615446 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:22.615351 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wn7qq" Sep 18 03:40:22.885286 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:22.885265 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-wn7qq"] Sep 18 03:40:22.887536 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:40:22.887512 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc4bfdf00_9f36_4fbe_bd54_9c3fa5ba641d.slice/crio-8374740e3e6cf18c35916e92155902a4961dddb491b8e8c0fde45346b518760c WatchSource:0}: Error finding container 8374740e3e6cf18c35916e92155902a4961dddb491b8e8c0fde45346b518760c: Status 404 returned error can't find the container with id 8374740e3e6cf18c35916e92155902a4961dddb491b8e8c0fde45346b518760c Sep 18 03:40:22.889950 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:22.889915 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 18 03:40:23.576715 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:23.576657 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wn7qq" event={"ID":"c4bfdf00-9f36-4fbe-bd54-9c3fa5ba641d","Type":"ContainerStarted","Data":"8374740e3e6cf18c35916e92155902a4961dddb491b8e8c0fde45346b518760c"} Sep 18 03:40:24.671510 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.671472 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf"] Sep 18 03:40:24.678212 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.678187 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf" Sep 18 03:40:24.680876 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.680851 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-pvhrl\"" Sep 18 03:40:24.681049 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.680915 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 18 03:40:24.681049 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.680850 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 18 03:40:24.681196 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.681175 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\"" Sep 18 03:40:24.681432 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.681416 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\"" Sep 18 03:40:24.681513 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.681497 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\"" Sep 18 03:40:24.686848 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.686792 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf"] Sep 18 03:40:24.721222 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.721187 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ccfa48da-d34b-4810-804d-ea7b8fa219c8-manifests\") pod \"opendatahub-operator-controller-manager-7b6444b484-ngxtf\" (UID: \"ccfa48da-d34b-4810-804d-ea7b8fa219c8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf" Sep 18 03:40:24.721396 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.721252 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ccfa48da-d34b-4810-804d-ea7b8fa219c8-metrics-cert\") pod \"opendatahub-operator-controller-manager-7b6444b484-ngxtf\" (UID: \"ccfa48da-d34b-4810-804d-ea7b8fa219c8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf" Sep 18 03:40:24.721396 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.721284 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pj6v\" (UniqueName: \"kubernetes.io/projected/ccfa48da-d34b-4810-804d-ea7b8fa219c8-kube-api-access-9pj6v\") pod \"opendatahub-operator-controller-manager-7b6444b484-ngxtf\" (UID: \"ccfa48da-d34b-4810-804d-ea7b8fa219c8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf" Sep 18 03:40:24.721396 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.721317 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ccfa48da-d34b-4810-804d-ea7b8fa219c8-tmp\") pod \"opendatahub-operator-controller-manager-7b6444b484-ngxtf\" (UID: \"ccfa48da-d34b-4810-804d-ea7b8fa219c8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf" Sep 18 03:40:24.721563 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.721425 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ccfa48da-d34b-4810-804d-ea7b8fa219c8-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b6444b484-ngxtf\" (UID: \"ccfa48da-d34b-4810-804d-ea7b8fa219c8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf" Sep 18 03:40:24.721563 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.721451 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ccfa48da-d34b-4810-804d-ea7b8fa219c8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b6444b484-ngxtf\" (UID: \"ccfa48da-d34b-4810-804d-ea7b8fa219c8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf" Sep 18 03:40:24.822395 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.822345 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ccfa48da-d34b-4810-804d-ea7b8fa219c8-tmp\") pod \"opendatahub-operator-controller-manager-7b6444b484-ngxtf\" (UID: \"ccfa48da-d34b-4810-804d-ea7b8fa219c8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf" Sep 18 03:40:24.822562 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.822453 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ccfa48da-d34b-4810-804d-ea7b8fa219c8-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b6444b484-ngxtf\" (UID: \"ccfa48da-d34b-4810-804d-ea7b8fa219c8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf" Sep 18 03:40:24.822562 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.822494 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ccfa48da-d34b-4810-804d-ea7b8fa219c8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b6444b484-ngxtf\" (UID: \"ccfa48da-d34b-4810-804d-ea7b8fa219c8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf" Sep 18 03:40:24.822562 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.822539 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ccfa48da-d34b-4810-804d-ea7b8fa219c8-manifests\") pod \"opendatahub-operator-controller-manager-7b6444b484-ngxtf\" (UID: \"ccfa48da-d34b-4810-804d-ea7b8fa219c8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf" Sep 18 03:40:24.822700 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.822584 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ccfa48da-d34b-4810-804d-ea7b8fa219c8-metrics-cert\") pod \"opendatahub-operator-controller-manager-7b6444b484-ngxtf\" (UID: \"ccfa48da-d34b-4810-804d-ea7b8fa219c8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf" Sep 18 03:40:24.822700 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.822610 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pj6v\" (UniqueName: \"kubernetes.io/projected/ccfa48da-d34b-4810-804d-ea7b8fa219c8-kube-api-access-9pj6v\") pod \"opendatahub-operator-controller-manager-7b6444b484-ngxtf\" (UID: \"ccfa48da-d34b-4810-804d-ea7b8fa219c8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf" Sep 18 03:40:24.823003 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.822974 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ccfa48da-d34b-4810-804d-ea7b8fa219c8-manifests\") pod \"opendatahub-operator-controller-manager-7b6444b484-ngxtf\" (UID: \"ccfa48da-d34b-4810-804d-ea7b8fa219c8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf" Sep 18 03:40:24.825032 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.825007 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ccfa48da-d34b-4810-804d-ea7b8fa219c8-tmp\") pod \"opendatahub-operator-controller-manager-7b6444b484-ngxtf\" (UID: \"ccfa48da-d34b-4810-804d-ea7b8fa219c8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf" Sep 18 03:40:24.825184 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.825164 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ccfa48da-d34b-4810-804d-ea7b8fa219c8-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b6444b484-ngxtf\" (UID: \"ccfa48da-d34b-4810-804d-ea7b8fa219c8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf" Sep 18 03:40:24.825432 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.825408 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ccfa48da-d34b-4810-804d-ea7b8fa219c8-metrics-cert\") pod \"opendatahub-operator-controller-manager-7b6444b484-ngxtf\" (UID: \"ccfa48da-d34b-4810-804d-ea7b8fa219c8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf" Sep 18 03:40:24.825514 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.825498 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ccfa48da-d34b-4810-804d-ea7b8fa219c8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b6444b484-ngxtf\" (UID: \"ccfa48da-d34b-4810-804d-ea7b8fa219c8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf" Sep 18 03:40:24.830680 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.830655 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pj6v\" (UniqueName: \"kubernetes.io/projected/ccfa48da-d34b-4810-804d-ea7b8fa219c8-kube-api-access-9pj6v\") pod \"opendatahub-operator-controller-manager-7b6444b484-ngxtf\" (UID: \"ccfa48da-d34b-4810-804d-ea7b8fa219c8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf" Sep 18 03:40:24.992692 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:24.992618 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf" Sep 18 03:40:25.954873 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:25.954844 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf"] Sep 18 03:40:25.956439 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:40:25.956409 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccfa48da_d34b_4810_804d_ea7b8fa219c8.slice/crio-c0ef799b95ee5478d0d325e741819a9b01447c4d5ea36ffd20f623c2682ddb96 WatchSource:0}: Error finding container c0ef799b95ee5478d0d325e741819a9b01447c4d5ea36ffd20f623c2682ddb96: Status 404 returned error can't find the container with id c0ef799b95ee5478d0d325e741819a9b01447c4d5ea36ffd20f623c2682ddb96 Sep 18 03:40:26.597310 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:26.597269 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wn7qq" event={"ID":"c4bfdf00-9f36-4fbe-bd54-9c3fa5ba641d","Type":"ContainerStarted","Data":"05457c13b4616021493c22a6a4d9971ea544adf1a5fa82312c48696e222d5126"} Sep 18 03:40:26.605792 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:26.605760 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf" event={"ID":"ccfa48da-d34b-4810-804d-ea7b8fa219c8","Type":"ContainerStarted","Data":"c0ef799b95ee5478d0d325e741819a9b01447c4d5ea36ffd20f623c2682ddb96"} Sep 18 03:40:26.622371 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:26.622313 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wn7qq" podStartSLOduration=1.7566265479999998 podStartE2EDuration="4.622292922s" totalImagesPullingTime="2.865666374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:40:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:40:26.621247575 +0000 UTC m=+440.904009458" watchObservedRunningTime="2026-09-18 03:40:26.622292922 +0000 UTC m=+440.905054787" Sep 18 03:40:29.624223 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:29.624130 2568 generic.go:356] "Generic (PLEG): container finished" podID="ccfa48da-d34b-4810-804d-ea7b8fa219c8" containerID="14e1ec3db8e0a1bf86c5cb606affe50aad39eab8cca52befe2d02b22fba80503" exitCode=0 Sep 18 03:40:29.624223 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:29.624177 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf" event={"ID":"ccfa48da-d34b-4810-804d-ea7b8fa219c8","Type":"ContainerDied","Data":"14e1ec3db8e0a1bf86c5cb606affe50aad39eab8cca52befe2d02b22fba80503"} Sep 18 03:40:30.630189 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:30.630153 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf" event={"ID":"ccfa48da-d34b-4810-804d-ea7b8fa219c8","Type":"ContainerStarted","Data":"9912e4298a49607d32344c12c3475ae015b2a35b41f41e12628766ae44fbd980"} Sep 18 03:40:30.630604 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:30.630242 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf" Sep 18 03:40:30.651307 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:30.651258 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf" podStartSLOduration=3.150927859 podStartE2EDuration="6.651244178s" totalImagesPullingTime="3.065514716s" totalInitContainerRuntime="434.801603ms" isStatefulPod=false podCreationTimestamp="2026-09-18 03:40:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:40:30.650894004 +0000 UTC m=+444.933655866" watchObservedRunningTime="2026-09-18 03:40:30.651244178 +0000 UTC m=+444.934006042" Sep 18 03:40:41.636824 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:41.636789 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7b6444b484-ngxtf" Sep 18 03:40:46.513703 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:46.513665 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7cbf85f5f5-p42nx"] Sep 18 03:40:46.611279 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:46.611252 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7cbf85f5f5-p42nx"] Sep 18 03:40:46.611435 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:46.611340 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7cbf85f5f5-p42nx" Sep 18 03:40:46.614259 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:46.614227 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 18 03:40:46.614259 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:46.614248 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mpt78\"" Sep 18 03:40:46.614422 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:46.614321 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 18 03:40:46.708564 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:46.708529 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73690a79-dfbf-420c-b858-97642b0b7491-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7cbf85f5f5-p42nx\" (UID: \"73690a79-dfbf-420c-b858-97642b0b7491\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7cbf85f5f5-p42nx" Sep 18 03:40:46.708745 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:46.708653 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zxtc\" (UniqueName: \"kubernetes.io/projected/73690a79-dfbf-420c-b858-97642b0b7491-kube-api-access-4zxtc\") pod \"kuadrant-operator-controller-manager-7cbf85f5f5-p42nx\" (UID: \"73690a79-dfbf-420c-b858-97642b0b7491\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7cbf85f5f5-p42nx" Sep 18 03:40:46.809808 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:46.809720 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zxtc\" (UniqueName: \"kubernetes.io/projected/73690a79-dfbf-420c-b858-97642b0b7491-kube-api-access-4zxtc\") pod \"kuadrant-operator-controller-manager-7cbf85f5f5-p42nx\" (UID: \"73690a79-dfbf-420c-b858-97642b0b7491\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7cbf85f5f5-p42nx" Sep 18 03:40:46.809808 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:46.809772 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73690a79-dfbf-420c-b858-97642b0b7491-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7cbf85f5f5-p42nx\" (UID: \"73690a79-dfbf-420c-b858-97642b0b7491\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7cbf85f5f5-p42nx" Sep 18 03:40:46.810186 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:46.810166 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73690a79-dfbf-420c-b858-97642b0b7491-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7cbf85f5f5-p42nx\" (UID: \"73690a79-dfbf-420c-b858-97642b0b7491\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7cbf85f5f5-p42nx" Sep 18 03:40:46.819146 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:46.819128 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zxtc\" (UniqueName: \"kubernetes.io/projected/73690a79-dfbf-420c-b858-97642b0b7491-kube-api-access-4zxtc\") pod \"kuadrant-operator-controller-manager-7cbf85f5f5-p42nx\" (UID: \"73690a79-dfbf-420c-b858-97642b0b7491\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7cbf85f5f5-p42nx" Sep 18 03:40:46.926558 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:46.926524 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7cbf85f5f5-p42nx" Sep 18 03:40:47.198371 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:47.198340 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7cbf85f5f5-p42nx"] Sep 18 03:40:47.201319 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:40:47.201285 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73690a79_dfbf_420c_b858_97642b0b7491.slice/crio-4762c960f7e0be51c3205823c2567d63c8e58600d201ec4958057229886a1e7d WatchSource:0}: Error finding container 4762c960f7e0be51c3205823c2567d63c8e58600d201ec4958057229886a1e7d: Status 404 returned error can't find the container with id 4762c960f7e0be51c3205823c2567d63c8e58600d201ec4958057229886a1e7d Sep 18 03:40:47.705992 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:47.705959 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7cbf85f5f5-p42nx" event={"ID":"73690a79-dfbf-420c-b858-97642b0b7491","Type":"ContainerStarted","Data":"4762c960f7e0be51c3205823c2567d63c8e58600d201ec4958057229886a1e7d"} Sep 18 03:40:49.019396 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:49.019361 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7c6dc6c678-4v5hd"] Sep 18 03:40:49.022206 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:49.022185 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c6dc6c678-4v5hd" Sep 18 03:40:49.024908 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:49.024756 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-qwfpf\"" Sep 18 03:40:49.024908 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:49.024838 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 18 03:40:49.025142 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:49.025114 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 18 03:40:49.035775 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:49.035739 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c6dc6c678-4v5hd"] Sep 18 03:40:49.131005 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:49.130975 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/427558c4-95a1-41e3-a4cf-ff272e522caf-tls-certs\") pod \"kube-auth-proxy-7c6dc6c678-4v5hd\" (UID: \"427558c4-95a1-41e3-a4cf-ff272e522caf\") " pod="openshift-ingress/kube-auth-proxy-7c6dc6c678-4v5hd" Sep 18 03:40:49.131178 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:49.131069 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qm6r\" (UniqueName: \"kubernetes.io/projected/427558c4-95a1-41e3-a4cf-ff272e522caf-kube-api-access-5qm6r\") pod \"kube-auth-proxy-7c6dc6c678-4v5hd\" (UID: \"427558c4-95a1-41e3-a4cf-ff272e522caf\") " pod="openshift-ingress/kube-auth-proxy-7c6dc6c678-4v5hd" Sep 18 03:40:49.131178 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:49.131152 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/427558c4-95a1-41e3-a4cf-ff272e522caf-tmp\") pod \"kube-auth-proxy-7c6dc6c678-4v5hd\" (UID: \"427558c4-95a1-41e3-a4cf-ff272e522caf\") " pod="openshift-ingress/kube-auth-proxy-7c6dc6c678-4v5hd" Sep 18 03:40:49.231948 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:49.231903 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/427558c4-95a1-41e3-a4cf-ff272e522caf-tls-certs\") pod \"kube-auth-proxy-7c6dc6c678-4v5hd\" (UID: \"427558c4-95a1-41e3-a4cf-ff272e522caf\") " pod="openshift-ingress/kube-auth-proxy-7c6dc6c678-4v5hd" Sep 18 03:40:49.232117 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:49.232007 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qm6r\" (UniqueName: \"kubernetes.io/projected/427558c4-95a1-41e3-a4cf-ff272e522caf-kube-api-access-5qm6r\") pod \"kube-auth-proxy-7c6dc6c678-4v5hd\" (UID: \"427558c4-95a1-41e3-a4cf-ff272e522caf\") " pod="openshift-ingress/kube-auth-proxy-7c6dc6c678-4v5hd" Sep 18 03:40:49.232117 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:49.232059 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/427558c4-95a1-41e3-a4cf-ff272e522caf-tmp\") pod \"kube-auth-proxy-7c6dc6c678-4v5hd\" (UID: \"427558c4-95a1-41e3-a4cf-ff272e522caf\") " pod="openshift-ingress/kube-auth-proxy-7c6dc6c678-4v5hd" Sep 18 03:40:49.232117 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:40:49.232060 2568 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Sep 18 03:40:49.232285 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:40:49.232129 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/427558c4-95a1-41e3-a4cf-ff272e522caf-tls-certs podName:427558c4-95a1-41e3-a4cf-ff272e522caf nodeName:}" failed. No retries permitted until 2026-09-18 03:40:49.73210663 +0000 UTC m=+464.014868472 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/427558c4-95a1-41e3-a4cf-ff272e522caf-tls-certs") pod "kube-auth-proxy-7c6dc6c678-4v5hd" (UID: "427558c4-95a1-41e3-a4cf-ff272e522caf") : secret "kube-auth-proxy-tls" not found Sep 18 03:40:49.234199 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:49.234179 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/427558c4-95a1-41e3-a4cf-ff272e522caf-tmp\") pod \"kube-auth-proxy-7c6dc6c678-4v5hd\" (UID: \"427558c4-95a1-41e3-a4cf-ff272e522caf\") " pod="openshift-ingress/kube-auth-proxy-7c6dc6c678-4v5hd" Sep 18 03:40:49.243296 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:49.243274 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qm6r\" (UniqueName: \"kubernetes.io/projected/427558c4-95a1-41e3-a4cf-ff272e522caf-kube-api-access-5qm6r\") pod \"kube-auth-proxy-7c6dc6c678-4v5hd\" (UID: \"427558c4-95a1-41e3-a4cf-ff272e522caf\") " pod="openshift-ingress/kube-auth-proxy-7c6dc6c678-4v5hd" Sep 18 03:40:49.736615 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:49.736579 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/427558c4-95a1-41e3-a4cf-ff272e522caf-tls-certs\") pod \"kube-auth-proxy-7c6dc6c678-4v5hd\" (UID: \"427558c4-95a1-41e3-a4cf-ff272e522caf\") " pod="openshift-ingress/kube-auth-proxy-7c6dc6c678-4v5hd" Sep 18 03:40:49.739117 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:49.739092 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/427558c4-95a1-41e3-a4cf-ff272e522caf-tls-certs\") pod \"kube-auth-proxy-7c6dc6c678-4v5hd\" (UID: \"427558c4-95a1-41e3-a4cf-ff272e522caf\") " pod="openshift-ingress/kube-auth-proxy-7c6dc6c678-4v5hd" Sep 18 03:40:49.937467 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:49.937435 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c6dc6c678-4v5hd" Sep 18 03:40:50.263641 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:50.263605 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-54765d59fd-tnkfn"] Sep 18 03:40:50.266179 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:50.266159 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-54765d59fd-tnkfn" Sep 18 03:40:50.269159 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:50.269089 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 18 03:40:50.269159 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:50.269110 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-bkpnn\"" Sep 18 03:40:50.269585 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:50.269564 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 18 03:40:50.269699 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:50.269572 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\"" Sep 18 03:40:50.283976 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:50.281816 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-54765d59fd-tnkfn"] Sep 18 03:40:50.342347 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:50.342311 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e7958393-fde7-4dee-947c-4da8a09069ce-manifests\") pod \"kserve-module-controller-manager-54765d59fd-tnkfn\" (UID: \"e7958393-fde7-4dee-947c-4da8a09069ce\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-tnkfn" Sep 18 03:40:50.342347 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:50.342351 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e7958393-fde7-4dee-947c-4da8a09069ce-trusted-ca\") pod \"kserve-module-controller-manager-54765d59fd-tnkfn\" (UID: \"e7958393-fde7-4dee-947c-4da8a09069ce\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-tnkfn" Sep 18 03:40:50.342598 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:50.342397 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnlc9\" (UniqueName: \"kubernetes.io/projected/e7958393-fde7-4dee-947c-4da8a09069ce-kube-api-access-qnlc9\") pod \"kserve-module-controller-manager-54765d59fd-tnkfn\" (UID: \"e7958393-fde7-4dee-947c-4da8a09069ce\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-tnkfn" Sep 18 03:40:50.342598 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:50.342453 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7958393-fde7-4dee-947c-4da8a09069ce-tmp\") pod \"kserve-module-controller-manager-54765d59fd-tnkfn\" (UID: \"e7958393-fde7-4dee-947c-4da8a09069ce\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-tnkfn" Sep 18 03:40:50.443651 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:50.443613 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7958393-fde7-4dee-947c-4da8a09069ce-tmp\") pod \"kserve-module-controller-manager-54765d59fd-tnkfn\" (UID: \"e7958393-fde7-4dee-947c-4da8a09069ce\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-tnkfn" Sep 18 03:40:50.443804 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:50.443691 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e7958393-fde7-4dee-947c-4da8a09069ce-manifests\") pod \"kserve-module-controller-manager-54765d59fd-tnkfn\" (UID: \"e7958393-fde7-4dee-947c-4da8a09069ce\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-tnkfn" Sep 18 03:40:50.443804 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:50.443714 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e7958393-fde7-4dee-947c-4da8a09069ce-trusted-ca\") pod \"kserve-module-controller-manager-54765d59fd-tnkfn\" (UID: \"e7958393-fde7-4dee-947c-4da8a09069ce\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-tnkfn" Sep 18 03:40:50.443804 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:50.443752 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnlc9\" (UniqueName: \"kubernetes.io/projected/e7958393-fde7-4dee-947c-4da8a09069ce-kube-api-access-qnlc9\") pod \"kserve-module-controller-manager-54765d59fd-tnkfn\" (UID: \"e7958393-fde7-4dee-947c-4da8a09069ce\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-tnkfn" Sep 18 03:40:50.444165 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:50.444104 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7958393-fde7-4dee-947c-4da8a09069ce-tmp\") pod \"kserve-module-controller-manager-54765d59fd-tnkfn\" (UID: \"e7958393-fde7-4dee-947c-4da8a09069ce\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-tnkfn" Sep 18 03:40:50.444165 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:50.444126 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e7958393-fde7-4dee-947c-4da8a09069ce-manifests\") pod \"kserve-module-controller-manager-54765d59fd-tnkfn\" (UID: \"e7958393-fde7-4dee-947c-4da8a09069ce\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-tnkfn" Sep 18 03:40:50.444614 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:50.444569 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e7958393-fde7-4dee-947c-4da8a09069ce-trusted-ca\") pod \"kserve-module-controller-manager-54765d59fd-tnkfn\" (UID: \"e7958393-fde7-4dee-947c-4da8a09069ce\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-tnkfn" Sep 18 03:40:50.459957 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:50.456300 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnlc9\" (UniqueName: \"kubernetes.io/projected/e7958393-fde7-4dee-947c-4da8a09069ce-kube-api-access-qnlc9\") pod \"kserve-module-controller-manager-54765d59fd-tnkfn\" (UID: \"e7958393-fde7-4dee-947c-4da8a09069ce\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-tnkfn" Sep 18 03:40:50.583143 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:50.581157 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-54765d59fd-tnkfn" Sep 18 03:40:51.432547 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:51.432522 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-54765d59fd-tnkfn"] Sep 18 03:40:51.434900 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:40:51.434868 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7958393_fde7_4dee_947c_4da8a09069ce.slice/crio-7105034881bdb27b8cf7879785fa48df36ec95158c346f4e4265b9ca0699efc3 WatchSource:0}: Error finding container 7105034881bdb27b8cf7879785fa48df36ec95158c346f4e4265b9ca0699efc3: Status 404 returned error can't find the container with id 7105034881bdb27b8cf7879785fa48df36ec95158c346f4e4265b9ca0699efc3 Sep 18 03:40:51.435881 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:51.435859 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c6dc6c678-4v5hd"] Sep 18 03:40:51.438969 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:40:51.438944 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod427558c4_95a1_41e3_a4cf_ff272e522caf.slice/crio-c04eb933924b8b5284ac179ce76eb1b3e6a3bf6d104f9b6387041bcae899cf83 WatchSource:0}: Error finding container c04eb933924b8b5284ac179ce76eb1b3e6a3bf6d104f9b6387041bcae899cf83: Status 404 returned error can't find the container with id c04eb933924b8b5284ac179ce76eb1b3e6a3bf6d104f9b6387041bcae899cf83 Sep 18 03:40:51.728928 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:51.728847 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-54765d59fd-tnkfn" event={"ID":"e7958393-fde7-4dee-947c-4da8a09069ce","Type":"ContainerStarted","Data":"7105034881bdb27b8cf7879785fa48df36ec95158c346f4e4265b9ca0699efc3"} Sep 18 03:40:51.730051 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:51.730019 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c6dc6c678-4v5hd" event={"ID":"427558c4-95a1-41e3-a4cf-ff272e522caf","Type":"ContainerStarted","Data":"c04eb933924b8b5284ac179ce76eb1b3e6a3bf6d104f9b6387041bcae899cf83"} Sep 18 03:40:51.731677 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:51.731653 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7cbf85f5f5-p42nx" event={"ID":"73690a79-dfbf-420c-b858-97642b0b7491","Type":"ContainerStarted","Data":"8a4493062d2e7aca815823fd08b5b83bd82b42476a24c786c24bc1ec0bad6005"} Sep 18 03:40:51.731770 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:51.731715 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7cbf85f5f5-p42nx" Sep 18 03:40:51.753646 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:51.753607 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-7cbf85f5f5-p42nx" podStartSLOduration=1.755947351 podStartE2EDuration="5.753593305s" totalImagesPullingTime="3.997645954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:40:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:40:51.752540587 +0000 UTC m=+466.035302450" watchObservedRunningTime="2026-09-18 03:40:51.753593305 +0000 UTC m=+466.036355168" Sep 18 03:40:55.760299 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:55.760262 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-54765d59fd-tnkfn" event={"ID":"e7958393-fde7-4dee-947c-4da8a09069ce","Type":"ContainerStarted","Data":"0f39936d81060eccc9381b7572555c8f9ee24498ec2e6fe370913221aeae07ab"} Sep 18 03:40:55.760710 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:55.760377 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-54765d59fd-tnkfn" Sep 18 03:40:55.761924 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:55.761900 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c6dc6c678-4v5hd" event={"ID":"427558c4-95a1-41e3-a4cf-ff272e522caf","Type":"ContainerStarted","Data":"1acb95a441f17371d6d69c9a561574d8089fb8113ba493ca7aa5bdf0894f54a8"} Sep 18 03:40:55.776140 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:55.776104 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-54765d59fd-tnkfn" podStartSLOduration=2.322665568 podStartE2EDuration="5.776092848s" totalImagesPullingTime="3.45342728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:40:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:40:55.774831165 +0000 UTC m=+470.057593022" watchObservedRunningTime="2026-09-18 03:40:55.776092848 +0000 UTC m=+470.058854726" Sep 18 03:40:55.789088 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:40:55.789051 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7c6dc6c678-4v5hd" podStartSLOduration=4.381365675 podStartE2EDuration="7.789038503s" totalImagesPullingTime="3.407672828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:40:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:40:55.78851882 +0000 UTC m=+470.071280692" watchObservedRunningTime="2026-09-18 03:40:55.789038503 +0000 UTC m=+470.071800365" Sep 18 03:41:02.740138 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:02.740106 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7cbf85f5f5-p42nx" Sep 18 03:41:03.130880 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:03.130800 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-54b6d5c989-d854p"] Sep 18 03:41:03.134565 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:03.134545 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-54b6d5c989-d854p" Sep 18 03:41:03.149516 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:03.149493 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-54b6d5c989-d854p"] Sep 18 03:41:03.266789 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:03.266752 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pf9g\" (UniqueName: \"kubernetes.io/projected/731a1bc6-8fc0-408e-9ef8-d9bab9c10783-kube-api-access-4pf9g\") pod \"kuadrant-operator-controller-manager-54b6d5c989-d854p\" (UID: \"731a1bc6-8fc0-408e-9ef8-d9bab9c10783\") " pod="kuadrant-system/kuadrant-operator-controller-manager-54b6d5c989-d854p" Sep 18 03:41:03.266974 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:03.266813 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/731a1bc6-8fc0-408e-9ef8-d9bab9c10783-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-54b6d5c989-d854p\" (UID: \"731a1bc6-8fc0-408e-9ef8-d9bab9c10783\") " pod="kuadrant-system/kuadrant-operator-controller-manager-54b6d5c989-d854p" Sep 18 03:41:03.368075 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:03.368041 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/731a1bc6-8fc0-408e-9ef8-d9bab9c10783-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-54b6d5c989-d854p\" (UID: \"731a1bc6-8fc0-408e-9ef8-d9bab9c10783\") " pod="kuadrant-system/kuadrant-operator-controller-manager-54b6d5c989-d854p" Sep 18 03:41:03.368237 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:03.368115 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pf9g\" (UniqueName: \"kubernetes.io/projected/731a1bc6-8fc0-408e-9ef8-d9bab9c10783-kube-api-access-4pf9g\") pod \"kuadrant-operator-controller-manager-54b6d5c989-d854p\" (UID: \"731a1bc6-8fc0-408e-9ef8-d9bab9c10783\") " pod="kuadrant-system/kuadrant-operator-controller-manager-54b6d5c989-d854p" Sep 18 03:41:03.368482 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:03.368459 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/731a1bc6-8fc0-408e-9ef8-d9bab9c10783-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-54b6d5c989-d854p\" (UID: \"731a1bc6-8fc0-408e-9ef8-d9bab9c10783\") " pod="kuadrant-system/kuadrant-operator-controller-manager-54b6d5c989-d854p" Sep 18 03:41:03.376252 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:03.376230 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pf9g\" (UniqueName: \"kubernetes.io/projected/731a1bc6-8fc0-408e-9ef8-d9bab9c10783-kube-api-access-4pf9g\") pod \"kuadrant-operator-controller-manager-54b6d5c989-d854p\" (UID: \"731a1bc6-8fc0-408e-9ef8-d9bab9c10783\") " pod="kuadrant-system/kuadrant-operator-controller-manager-54b6d5c989-d854p" Sep 18 03:41:03.444196 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:03.444172 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-54b6d5c989-d854p" Sep 18 03:41:03.720859 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:03.720837 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-54b6d5c989-d854p"] Sep 18 03:41:03.723119 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:41:03.723092 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod731a1bc6_8fc0_408e_9ef8_d9bab9c10783.slice/crio-cf5b4820d4c16cd272dd8b9453fac88aa0ff24b44c5da5b5d3660bb2db56b40b WatchSource:0}: Error finding container cf5b4820d4c16cd272dd8b9453fac88aa0ff24b44c5da5b5d3660bb2db56b40b: Status 404 returned error can't find the container with id cf5b4820d4c16cd272dd8b9453fac88aa0ff24b44c5da5b5d3660bb2db56b40b Sep 18 03:41:03.798795 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:03.798765 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-54b6d5c989-d854p" event={"ID":"731a1bc6-8fc0-408e-9ef8-d9bab9c10783","Type":"ContainerStarted","Data":"0c29cec8465e23242319756f4a48b4ab83b82a5ff4ef9e1e90214560cafca184"} Sep 18 03:41:03.799104 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:03.798818 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-54b6d5c989-d854p" event={"ID":"731a1bc6-8fc0-408e-9ef8-d9bab9c10783","Type":"ContainerStarted","Data":"cf5b4820d4c16cd272dd8b9453fac88aa0ff24b44c5da5b5d3660bb2db56b40b"} Sep 18 03:41:03.799104 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:03.798905 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-54b6d5c989-d854p" Sep 18 03:41:03.821443 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:03.821402 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-54b6d5c989-d854p" podStartSLOduration=0.82138927 podStartE2EDuration="821.38927ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:41:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:41:03.819412054 +0000 UTC m=+478.102173954" watchObservedRunningTime="2026-09-18 03:41:03.82138927 +0000 UTC m=+478.104151132" Sep 18 03:41:14.805335 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:14.805306 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-54b6d5c989-d854p" Sep 18 03:41:14.856686 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:14.856657 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7cbf85f5f5-p42nx"] Sep 18 03:41:14.856882 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:14.856841 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-7cbf85f5f5-p42nx" podUID="73690a79-dfbf-420c-b858-97642b0b7491" containerName="manager" containerID="cri-o://8a4493062d2e7aca815823fd08b5b83bd82b42476a24c786c24bc1ec0bad6005" gracePeriod=10 Sep 18 03:41:15.102852 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:15.102828 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7cbf85f5f5-p42nx" Sep 18 03:41:15.170141 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:15.170116 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zxtc\" (UniqueName: \"kubernetes.io/projected/73690a79-dfbf-420c-b858-97642b0b7491-kube-api-access-4zxtc\") pod \"73690a79-dfbf-420c-b858-97642b0b7491\" (UID: \"73690a79-dfbf-420c-b858-97642b0b7491\") " Sep 18 03:41:15.170266 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:15.170188 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73690a79-dfbf-420c-b858-97642b0b7491-extensions-socket-volume\") pod \"73690a79-dfbf-420c-b858-97642b0b7491\" (UID: \"73690a79-dfbf-420c-b858-97642b0b7491\") " Sep 18 03:41:15.170626 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:15.170595 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73690a79-dfbf-420c-b858-97642b0b7491-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "73690a79-dfbf-420c-b858-97642b0b7491" (UID: "73690a79-dfbf-420c-b858-97642b0b7491"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 03:41:15.172131 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:15.172109 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73690a79-dfbf-420c-b858-97642b0b7491-kube-api-access-4zxtc" (OuterVolumeSpecName: "kube-api-access-4zxtc") pod "73690a79-dfbf-420c-b858-97642b0b7491" (UID: "73690a79-dfbf-420c-b858-97642b0b7491"). InnerVolumeSpecName "kube-api-access-4zxtc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 03:41:15.271257 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:15.271229 2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73690a79-dfbf-420c-b858-97642b0b7491-extensions-socket-volume\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:41:15.271257 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:15.271259 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zxtc\" (UniqueName: \"kubernetes.io/projected/73690a79-dfbf-420c-b858-97642b0b7491-kube-api-access-4zxtc\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:41:15.851193 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:15.851164 2568 generic.go:356] "Generic (PLEG): container finished" podID="73690a79-dfbf-420c-b858-97642b0b7491" containerID="8a4493062d2e7aca815823fd08b5b83bd82b42476a24c786c24bc1ec0bad6005" exitCode=0 Sep 18 03:41:15.851579 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:15.851207 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7cbf85f5f5-p42nx" Sep 18 03:41:15.851579 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:15.851250 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7cbf85f5f5-p42nx" event={"ID":"73690a79-dfbf-420c-b858-97642b0b7491","Type":"ContainerDied","Data":"8a4493062d2e7aca815823fd08b5b83bd82b42476a24c786c24bc1ec0bad6005"} Sep 18 03:41:15.851579 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:15.851304 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7cbf85f5f5-p42nx" event={"ID":"73690a79-dfbf-420c-b858-97642b0b7491","Type":"ContainerDied","Data":"4762c960f7e0be51c3205823c2567d63c8e58600d201ec4958057229886a1e7d"} Sep 18 03:41:15.851579 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:15.851333 2568 scope.go:117] "RemoveContainer" containerID="8a4493062d2e7aca815823fd08b5b83bd82b42476a24c786c24bc1ec0bad6005" Sep 18 03:41:15.859747 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:15.859583 2568 scope.go:117] "RemoveContainer" containerID="8a4493062d2e7aca815823fd08b5b83bd82b42476a24c786c24bc1ec0bad6005" Sep 18 03:41:15.859839 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:41:15.859822 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a4493062d2e7aca815823fd08b5b83bd82b42476a24c786c24bc1ec0bad6005\": container with ID starting with 8a4493062d2e7aca815823fd08b5b83bd82b42476a24c786c24bc1ec0bad6005 not found: ID does not exist" containerID="8a4493062d2e7aca815823fd08b5b83bd82b42476a24c786c24bc1ec0bad6005" Sep 18 03:41:15.859885 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:15.859848 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a4493062d2e7aca815823fd08b5b83bd82b42476a24c786c24bc1ec0bad6005"} err="failed to get container status \"8a4493062d2e7aca815823fd08b5b83bd82b42476a24c786c24bc1ec0bad6005\": rpc error: code = NotFound desc = could not find container \"8a4493062d2e7aca815823fd08b5b83bd82b42476a24c786c24bc1ec0bad6005\": container with ID starting with 8a4493062d2e7aca815823fd08b5b83bd82b42476a24c786c24bc1ec0bad6005 not found: ID does not exist" Sep 18 03:41:15.873230 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:15.873202 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7cbf85f5f5-p42nx"] Sep 18 03:41:15.881859 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:15.881841 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7cbf85f5f5-p42nx"] Sep 18 03:41:16.767620 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:16.767590 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-54765d59fd-tnkfn" Sep 18 03:41:17.887572 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:17.887543 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73690a79-dfbf-420c-b858-97642b0b7491" path="/var/lib/kubelet/pods/73690a79-dfbf-420c-b858-97642b0b7491/volumes" Sep 18 03:41:33.581680 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:33.581637 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-dmr68"] Sep 18 03:41:33.582282 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:33.582194 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73690a79-dfbf-420c-b858-97642b0b7491" containerName="manager" Sep 18 03:41:33.582282 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:33.582214 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="73690a79-dfbf-420c-b858-97642b0b7491" containerName="manager" Sep 18 03:41:33.582392 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:33.582303 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="73690a79-dfbf-420c-b858-97642b0b7491" containerName="manager" Sep 18 03:41:33.589566 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:33.589541 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-dmr68" Sep 18 03:41:33.593407 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:33.593384 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Sep 18 03:41:33.593648 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:33.593426 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4cd2s\"" Sep 18 03:41:33.602090 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:33.602061 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-dmr68"] Sep 18 03:41:33.629732 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:33.629695 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcbbv\" (UniqueName: \"kubernetes.io/projected/be7d76da-e4b3-438f-9346-870958f90704-kube-api-access-pcbbv\") pod \"limitador-limitador-8588749977-dmr68\" (UID: \"be7d76da-e4b3-438f-9346-870958f90704\") " pod="kuadrant-system/limitador-limitador-8588749977-dmr68" Sep 18 03:41:33.629898 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:33.629812 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/be7d76da-e4b3-438f-9346-870958f90704-config-file\") pod \"limitador-limitador-8588749977-dmr68\" (UID: \"be7d76da-e4b3-438f-9346-870958f90704\") " pod="kuadrant-system/limitador-limitador-8588749977-dmr68" Sep 18 03:41:33.657193 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:33.657162 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-dmr68"] Sep 18 03:41:33.730984 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:33.730921 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcbbv\" (UniqueName: \"kubernetes.io/projected/be7d76da-e4b3-438f-9346-870958f90704-kube-api-access-pcbbv\") pod \"limitador-limitador-8588749977-dmr68\" (UID: \"be7d76da-e4b3-438f-9346-870958f90704\") " pod="kuadrant-system/limitador-limitador-8588749977-dmr68" Sep 18 03:41:33.731181 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:33.731012 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/be7d76da-e4b3-438f-9346-870958f90704-config-file\") pod \"limitador-limitador-8588749977-dmr68\" (UID: \"be7d76da-e4b3-438f-9346-870958f90704\") " pod="kuadrant-system/limitador-limitador-8588749977-dmr68" Sep 18 03:41:33.731656 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:33.731637 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/be7d76da-e4b3-438f-9346-870958f90704-config-file\") pod \"limitador-limitador-8588749977-dmr68\" (UID: \"be7d76da-e4b3-438f-9346-870958f90704\") " pod="kuadrant-system/limitador-limitador-8588749977-dmr68" Sep 18 03:41:33.738550 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:33.738521 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcbbv\" (UniqueName: \"kubernetes.io/projected/be7d76da-e4b3-438f-9346-870958f90704-kube-api-access-pcbbv\") pod \"limitador-limitador-8588749977-dmr68\" (UID: \"be7d76da-e4b3-438f-9346-870958f90704\") " pod="kuadrant-system/limitador-limitador-8588749977-dmr68" Sep 18 03:41:33.902415 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:33.902382 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-dmr68" Sep 18 03:41:34.173534 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:34.173512 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-dmr68"] Sep 18 03:41:34.175562 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:41:34.175535 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe7d76da_e4b3_438f_9346_870958f90704.slice/crio-f8cd8618423fd0454fcb54c563167da5635e1e4f61eb695a02e49fcd091c9a78 WatchSource:0}: Error finding container f8cd8618423fd0454fcb54c563167da5635e1e4f61eb695a02e49fcd091c9a78: Status 404 returned error can't find the container with id f8cd8618423fd0454fcb54c563167da5635e1e4f61eb695a02e49fcd091c9a78 Sep 18 03:41:34.339910 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:34.339877 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-cmmtl"] Sep 18 03:41:34.342832 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:34.342814 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-cmmtl" Sep 18 03:41:34.345275 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:34.345249 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-l28gj\"" Sep 18 03:41:34.350488 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:34.350465 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-cmmtl"] Sep 18 03:41:34.438706 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:34.438614 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmfqf\" (UniqueName: \"kubernetes.io/projected/b921e4f9-e4f0-4ad3-81fc-b71d6b03620a-kube-api-access-lmfqf\") pod \"authorino-7bcfd8f9cd-cmmtl\" (UID: \"b921e4f9-e4f0-4ad3-81fc-b71d6b03620a\") " pod="kuadrant-system/authorino-7bcfd8f9cd-cmmtl" Sep 18 03:41:34.539568 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:34.539539 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmfqf\" (UniqueName: \"kubernetes.io/projected/b921e4f9-e4f0-4ad3-81fc-b71d6b03620a-kube-api-access-lmfqf\") pod \"authorino-7bcfd8f9cd-cmmtl\" (UID: \"b921e4f9-e4f0-4ad3-81fc-b71d6b03620a\") " pod="kuadrant-system/authorino-7bcfd8f9cd-cmmtl" Sep 18 03:41:34.547547 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:34.547520 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmfqf\" (UniqueName: \"kubernetes.io/projected/b921e4f9-e4f0-4ad3-81fc-b71d6b03620a-kube-api-access-lmfqf\") pod \"authorino-7bcfd8f9cd-cmmtl\" (UID: \"b921e4f9-e4f0-4ad3-81fc-b71d6b03620a\") " pod="kuadrant-system/authorino-7bcfd8f9cd-cmmtl" Sep 18 03:41:34.653721 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:34.653689 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-cmmtl" Sep 18 03:41:34.922012 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:34.921986 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-cmmtl"] Sep 18 03:41:34.923606 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:41:34.923577 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb921e4f9_e4f0_4ad3_81fc_b71d6b03620a.slice/crio-0a5f794195157978e9b3ab9da06ac14b1083361971e7fbdc27bee042e5f507d9 WatchSource:0}: Error finding container 0a5f794195157978e9b3ab9da06ac14b1083361971e7fbdc27bee042e5f507d9: Status 404 returned error can't find the container with id 0a5f794195157978e9b3ab9da06ac14b1083361971e7fbdc27bee042e5f507d9 Sep 18 03:41:34.942730 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:34.942699 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-cmmtl" event={"ID":"b921e4f9-e4f0-4ad3-81fc-b71d6b03620a","Type":"ContainerStarted","Data":"0a5f794195157978e9b3ab9da06ac14b1083361971e7fbdc27bee042e5f507d9"} Sep 18 03:41:34.943974 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:34.943954 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-dmr68" event={"ID":"be7d76da-e4b3-438f-9346-870958f90704","Type":"ContainerStarted","Data":"f8cd8618423fd0454fcb54c563167da5635e1e4f61eb695a02e49fcd091c9a78"} Sep 18 03:41:35.952091 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:35.952054 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-dmr68" event={"ID":"be7d76da-e4b3-438f-9346-870958f90704","Type":"ContainerStarted","Data":"12bd5e5b6c9219e833a07a9a86f0a4a995767e79ff33800899d1174961798cef"} Sep 18 03:41:35.952543 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:35.952181 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-dmr68" Sep 18 03:41:35.966995 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:35.966903 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-dmr68" podStartSLOduration=1.809478059 podStartE2EDuration="2.9668858s" totalImagesPullingTime="1.157407741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:41:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:41:35.966070529 +0000 UTC m=+510.248832424" watchObservedRunningTime="2026-09-18 03:41:35.9668858 +0000 UTC m=+510.249647665" Sep 18 03:41:36.958598 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:36.958566 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-cmmtl" event={"ID":"b921e4f9-e4f0-4ad3-81fc-b71d6b03620a","Type":"ContainerStarted","Data":"cf0a81cbdd6bb8ca905788d7f752ff4c3026871b32b494379ea7d77fee5f8889"} Sep 18 03:41:36.974047 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:36.973986 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-cmmtl" podStartSLOduration=1.036728621 podStartE2EDuration="2.973962563s" totalImagesPullingTime="1.937233942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:41:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:41:36.972503169 +0000 UTC m=+511.255265032" watchObservedRunningTime="2026-09-18 03:41:36.973962563 +0000 UTC m=+511.256724435" Sep 18 03:41:39.658208 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:39.658172 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-cmmtl"] Sep 18 03:41:39.658561 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:39.658329 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-cmmtl" podUID="b921e4f9-e4f0-4ad3-81fc-b71d6b03620a" containerName="authorino" containerID="cri-o://cf0a81cbdd6bb8ca905788d7f752ff4c3026871b32b494379ea7d77fee5f8889" gracePeriod=30 Sep 18 03:41:39.898485 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:39.898456 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-cmmtl" Sep 18 03:41:39.977085 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:39.977008 2568 generic.go:356] "Generic (PLEG): container finished" podID="b921e4f9-e4f0-4ad3-81fc-b71d6b03620a" containerID="cf0a81cbdd6bb8ca905788d7f752ff4c3026871b32b494379ea7d77fee5f8889" exitCode=0 Sep 18 03:41:39.977085 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:39.977040 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-cmmtl" Sep 18 03:41:39.977257 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:39.977085 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-cmmtl" event={"ID":"b921e4f9-e4f0-4ad3-81fc-b71d6b03620a","Type":"ContainerDied","Data":"cf0a81cbdd6bb8ca905788d7f752ff4c3026871b32b494379ea7d77fee5f8889"} Sep 18 03:41:39.977257 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:39.977126 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-cmmtl" event={"ID":"b921e4f9-e4f0-4ad3-81fc-b71d6b03620a","Type":"ContainerDied","Data":"0a5f794195157978e9b3ab9da06ac14b1083361971e7fbdc27bee042e5f507d9"} Sep 18 03:41:39.977257 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:39.977147 2568 scope.go:117] "RemoveContainer" containerID="cf0a81cbdd6bb8ca905788d7f752ff4c3026871b32b494379ea7d77fee5f8889" Sep 18 03:41:39.985256 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:39.985233 2568 scope.go:117] "RemoveContainer" containerID="cf0a81cbdd6bb8ca905788d7f752ff4c3026871b32b494379ea7d77fee5f8889" Sep 18 03:41:39.985528 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:41:39.985510 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf0a81cbdd6bb8ca905788d7f752ff4c3026871b32b494379ea7d77fee5f8889\": container with ID starting with cf0a81cbdd6bb8ca905788d7f752ff4c3026871b32b494379ea7d77fee5f8889 not found: ID does not exist" containerID="cf0a81cbdd6bb8ca905788d7f752ff4c3026871b32b494379ea7d77fee5f8889" Sep 18 03:41:39.985596 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:39.985538 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf0a81cbdd6bb8ca905788d7f752ff4c3026871b32b494379ea7d77fee5f8889"} err="failed to get container status \"cf0a81cbdd6bb8ca905788d7f752ff4c3026871b32b494379ea7d77fee5f8889\": rpc error: code = NotFound desc = could not find container \"cf0a81cbdd6bb8ca905788d7f752ff4c3026871b32b494379ea7d77fee5f8889\": container with ID starting with cf0a81cbdd6bb8ca905788d7f752ff4c3026871b32b494379ea7d77fee5f8889 not found: ID does not exist" Sep 18 03:41:39.986874 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:39.986858 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmfqf\" (UniqueName: \"kubernetes.io/projected/b921e4f9-e4f0-4ad3-81fc-b71d6b03620a-kube-api-access-lmfqf\") pod \"b921e4f9-e4f0-4ad3-81fc-b71d6b03620a\" (UID: \"b921e4f9-e4f0-4ad3-81fc-b71d6b03620a\") " Sep 18 03:41:39.988678 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:39.988658 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b921e4f9-e4f0-4ad3-81fc-b71d6b03620a-kube-api-access-lmfqf" (OuterVolumeSpecName: "kube-api-access-lmfqf") pod "b921e4f9-e4f0-4ad3-81fc-b71d6b03620a" (UID: "b921e4f9-e4f0-4ad3-81fc-b71d6b03620a"). InnerVolumeSpecName "kube-api-access-lmfqf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 03:41:40.087927 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:40.087895 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmfqf\" (UniqueName: \"kubernetes.io/projected/b921e4f9-e4f0-4ad3-81fc-b71d6b03620a-kube-api-access-lmfqf\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:41:40.308247 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:40.308220 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-cmmtl"] Sep 18 03:41:40.324281 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:40.324257 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-cmmtl"] Sep 18 03:41:41.882817 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:41.882776 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b921e4f9-e4f0-4ad3-81fc-b71d6b03620a" path="/var/lib/kubelet/pods/b921e4f9-e4f0-4ad3-81fc-b71d6b03620a/volumes" Sep 18 03:41:42.937612 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:42.937579 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8"] Sep 18 03:41:42.938010 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:42.937995 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b921e4f9-e4f0-4ad3-81fc-b71d6b03620a" containerName="authorino" Sep 18 03:41:42.938010 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:42.938009 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b921e4f9-e4f0-4ad3-81fc-b71d6b03620a" containerName="authorino" Sep 18 03:41:42.938098 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:42.938081 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b921e4f9-e4f0-4ad3-81fc-b71d6b03620a" containerName="authorino" Sep 18 03:41:42.942983 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:42.942963 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8" Sep 18 03:41:42.945864 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:42.945844 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Sep 18 03:41:42.945984 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:42.945845 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bzpdc\"" Sep 18 03:41:42.946149 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:42.946136 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 18 03:41:42.946552 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:42.946537 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 18 03:41:42.946816 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:42.946802 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 18 03:41:42.952489 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:42.952468 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8"] Sep 18 03:41:43.011514 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:43.011481 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f-metrics-certs\") pod \"maas-api-7d7dbfc498-dpvb8\" (UID: \"fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f\") " pod="odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8" Sep 18 03:41:43.011705 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:43.011528 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f-openshift-service-ca\") pod \"maas-api-7d7dbfc498-dpvb8\" (UID: \"fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f\") " pod="odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8" Sep 18 03:41:43.011705 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:43.011550 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xnhc\" (UniqueName: \"kubernetes.io/projected/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f-kube-api-access-7xnhc\") pod \"maas-api-7d7dbfc498-dpvb8\" (UID: \"fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f\") " pod="odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8" Sep 18 03:41:43.011705 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:43.011580 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f-maas-api-tls\") pod \"maas-api-7d7dbfc498-dpvb8\" (UID: \"fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f\") " pod="odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8" Sep 18 03:41:43.112426 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:43.112399 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f-metrics-certs\") pod \"maas-api-7d7dbfc498-dpvb8\" (UID: \"fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f\") " pod="odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8" Sep 18 03:41:43.112592 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:43.112435 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f-openshift-service-ca\") pod \"maas-api-7d7dbfc498-dpvb8\" (UID: \"fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f\") " pod="odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8" Sep 18 03:41:43.112592 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:43.112452 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xnhc\" (UniqueName: \"kubernetes.io/projected/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f-kube-api-access-7xnhc\") pod \"maas-api-7d7dbfc498-dpvb8\" (UID: \"fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f\") " pod="odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8" Sep 18 03:41:43.112592 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:41:43.112543 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found Sep 18 03:41:43.112779 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:41:43.112601 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f-metrics-certs podName:fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f nodeName:}" failed. No retries permitted until 2026-09-18 03:41:43.612587173 +0000 UTC m=+517.895349014 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f-metrics-certs") pod "maas-api-7d7dbfc498-dpvb8" (UID: "fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f") : secret "maas-api-metrics-tls" not found Sep 18 03:41:43.112779 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:43.112593 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f-maas-api-tls\") pod \"maas-api-7d7dbfc498-dpvb8\" (UID: \"fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f\") " pod="odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8" Sep 18 03:41:43.113174 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:43.113153 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f-openshift-service-ca\") pod \"maas-api-7d7dbfc498-dpvb8\" (UID: \"fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f\") " pod="odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8" Sep 18 03:41:43.115028 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:43.115010 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f-maas-api-tls\") pod \"maas-api-7d7dbfc498-dpvb8\" (UID: \"fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f\") " pod="odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8" Sep 18 03:41:43.121915 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:43.121898 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xnhc\" (UniqueName: \"kubernetes.io/projected/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f-kube-api-access-7xnhc\") pod \"maas-api-7d7dbfc498-dpvb8\" (UID: \"fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f\") " pod="odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8" Sep 18 03:41:43.617208 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:43.617165 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f-metrics-certs\") pod \"maas-api-7d7dbfc498-dpvb8\" (UID: \"fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f\") " pod="odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8" Sep 18 03:41:43.619650 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:43.619623 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f-metrics-certs\") pod \"maas-api-7d7dbfc498-dpvb8\" (UID: \"fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f\") " pod="odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8" Sep 18 03:41:43.854296 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:43.854262 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8" Sep 18 03:41:43.952341 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:43.952309 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-xk5gl"] Sep 18 03:41:43.955400 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:43.955381 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-xk5gl" Sep 18 03:41:43.959079 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:43.959058 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 18 03:41:43.959179 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:43.959063 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-hwtpl\"" Sep 18 03:41:43.969717 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:43.969695 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-xk5gl"] Sep 18 03:41:44.021014 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:44.020980 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c378e07-18c0-4165-81ef-6ab860b4aa7f-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-xk5gl\" (UID: \"4c378e07-18c0-4165-81ef-6ab860b4aa7f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-xk5gl" Sep 18 03:41:44.021168 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:44.021123 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt6vc\" (UniqueName: \"kubernetes.io/projected/4c378e07-18c0-4165-81ef-6ab860b4aa7f-kube-api-access-pt6vc\") pod \"payload-pre-processing-59fbc47bf8-xk5gl\" (UID: \"4c378e07-18c0-4165-81ef-6ab860b4aa7f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-xk5gl" Sep 18 03:41:44.108452 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:44.108399 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-dmr68"] Sep 18 03:41:44.109731 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:44.109625 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-dmr68" Sep 18 03:41:44.122342 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:44.122315 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt6vc\" (UniqueName: \"kubernetes.io/projected/4c378e07-18c0-4165-81ef-6ab860b4aa7f-kube-api-access-pt6vc\") pod \"payload-pre-processing-59fbc47bf8-xk5gl\" (UID: \"4c378e07-18c0-4165-81ef-6ab860b4aa7f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-xk5gl" Sep 18 03:41:44.122491 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:44.122381 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c378e07-18c0-4165-81ef-6ab860b4aa7f-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-xk5gl\" (UID: \"4c378e07-18c0-4165-81ef-6ab860b4aa7f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-xk5gl" Sep 18 03:41:44.123091 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:44.123066 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c378e07-18c0-4165-81ef-6ab860b4aa7f-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-xk5gl\" (UID: \"4c378e07-18c0-4165-81ef-6ab860b4aa7f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-xk5gl" Sep 18 03:41:44.130729 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:44.130704 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt6vc\" (UniqueName: \"kubernetes.io/projected/4c378e07-18c0-4165-81ef-6ab860b4aa7f-kube-api-access-pt6vc\") pod \"payload-pre-processing-59fbc47bf8-xk5gl\" (UID: \"4c378e07-18c0-4165-81ef-6ab860b4aa7f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-xk5gl" Sep 18 03:41:44.133622 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:44.133599 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8"] Sep 18 03:41:44.135898 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:41:44.135878 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd581b98_b4f7_4c6c_9c1d_71f7afd6ec1f.slice/crio-b0dd2a7be6fa8cc563df4455bdb3947fe5ef54d9fb79057452c4774763621518 WatchSource:0}: Error finding container b0dd2a7be6fa8cc563df4455bdb3947fe5ef54d9fb79057452c4774763621518: Status 404 returned error can't find the container with id b0dd2a7be6fa8cc563df4455bdb3947fe5ef54d9fb79057452c4774763621518 Sep 18 03:41:44.182330 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:44.182274 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-dmr68"] Sep 18 03:41:44.265390 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:44.265371 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-xk5gl" Sep 18 03:41:44.546002 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:44.544975 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-xk5gl"] Sep 18 03:41:44.559374 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:41:44.559341 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c378e07_18c0_4165_81ef_6ab860b4aa7f.slice/crio-6ceb1bd29c5b88ae73437cef2d6d60db89b7fcff56e54d2ac9248adbd52da710 WatchSource:0}: Error finding container 6ceb1bd29c5b88ae73437cef2d6d60db89b7fcff56e54d2ac9248adbd52da710: Status 404 returned error can't find the container with id 6ceb1bd29c5b88ae73437cef2d6d60db89b7fcff56e54d2ac9248adbd52da710 Sep 18 03:41:44.981177 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:44.981089 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-7645b9c6f5-9mwtv"] Sep 18 03:41:44.983742 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:44.983719 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-7645b9c6f5-9mwtv" Sep 18 03:41:44.986142 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:44.986121 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Sep 18 03:41:44.986260 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:44.986231 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-7d5kq\"" Sep 18 03:41:44.997607 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:44.997577 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-7645b9c6f5-9mwtv"] Sep 18 03:41:45.005714 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:45.005655 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8" event={"ID":"fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f","Type":"ContainerStarted","Data":"b0dd2a7be6fa8cc563df4455bdb3947fe5ef54d9fb79057452c4774763621518"} Sep 18 03:41:45.007573 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:45.007549 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-xk5gl" event={"ID":"4c378e07-18c0-4165-81ef-6ab860b4aa7f","Type":"ContainerStarted","Data":"6ceb1bd29c5b88ae73437cef2d6d60db89b7fcff56e54d2ac9248adbd52da710"} Sep 18 03:41:45.054096 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:45.054062 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-77df98bf98-qhq74"] Sep 18 03:41:45.056857 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:45.056835 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-77df98bf98-qhq74" Sep 18 03:41:45.060001 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:45.059977 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Sep 18 03:41:45.060106 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:45.060042 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-t6gx8\"" Sep 18 03:41:45.090200 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:45.090173 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-77df98bf98-qhq74"] Sep 18 03:41:45.132521 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:45.132494 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/40c85b4f-853b-43b4-9dc7-d0f1196b8e09-cert\") pod \"kserve-controller-manager-7645b9c6f5-9mwtv\" (UID: \"40c85b4f-853b-43b4-9dc7-d0f1196b8e09\") " pod="opendatahub/kserve-controller-manager-7645b9c6f5-9mwtv" Sep 18 03:41:45.132675 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:45.132555 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b65c1dd-90a7-4030-8d8e-4860bd7313bf-tls-certs\") pod \"model-serving-api-77df98bf98-qhq74\" (UID: \"7b65c1dd-90a7-4030-8d8e-4860bd7313bf\") " pod="opendatahub/model-serving-api-77df98bf98-qhq74" Sep 18 03:41:45.132754 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:45.132687 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmspv\" (UniqueName: \"kubernetes.io/projected/7b65c1dd-90a7-4030-8d8e-4860bd7313bf-kube-api-access-wmspv\") pod \"model-serving-api-77df98bf98-qhq74\" (UID: \"7b65c1dd-90a7-4030-8d8e-4860bd7313bf\") " pod="opendatahub/model-serving-api-77df98bf98-qhq74" Sep 18 03:41:45.132754 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:45.132719 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gn6n\" (UniqueName: \"kubernetes.io/projected/40c85b4f-853b-43b4-9dc7-d0f1196b8e09-kube-api-access-8gn6n\") pod \"kserve-controller-manager-7645b9c6f5-9mwtv\" (UID: \"40c85b4f-853b-43b4-9dc7-d0f1196b8e09\") " pod="opendatahub/kserve-controller-manager-7645b9c6f5-9mwtv" Sep 18 03:41:45.233452 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:45.233374 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmspv\" (UniqueName: \"kubernetes.io/projected/7b65c1dd-90a7-4030-8d8e-4860bd7313bf-kube-api-access-wmspv\") pod \"model-serving-api-77df98bf98-qhq74\" (UID: \"7b65c1dd-90a7-4030-8d8e-4860bd7313bf\") " pod="opendatahub/model-serving-api-77df98bf98-qhq74" Sep 18 03:41:45.233452 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:45.233423 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gn6n\" (UniqueName: \"kubernetes.io/projected/40c85b4f-853b-43b4-9dc7-d0f1196b8e09-kube-api-access-8gn6n\") pod \"kserve-controller-manager-7645b9c6f5-9mwtv\" (UID: \"40c85b4f-853b-43b4-9dc7-d0f1196b8e09\") " pod="opendatahub/kserve-controller-manager-7645b9c6f5-9mwtv" Sep 18 03:41:45.233666 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:45.233510 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/40c85b4f-853b-43b4-9dc7-d0f1196b8e09-cert\") pod \"kserve-controller-manager-7645b9c6f5-9mwtv\" (UID: \"40c85b4f-853b-43b4-9dc7-d0f1196b8e09\") " pod="opendatahub/kserve-controller-manager-7645b9c6f5-9mwtv" Sep 18 03:41:45.233666 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:45.233549 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b65c1dd-90a7-4030-8d8e-4860bd7313bf-tls-certs\") pod \"model-serving-api-77df98bf98-qhq74\" (UID: \"7b65c1dd-90a7-4030-8d8e-4860bd7313bf\") " pod="opendatahub/model-serving-api-77df98bf98-qhq74" Sep 18 03:41:45.233778 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:41:45.233706 2568 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Sep 18 03:41:45.233778 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:41:45.233772 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b65c1dd-90a7-4030-8d8e-4860bd7313bf-tls-certs podName:7b65c1dd-90a7-4030-8d8e-4860bd7313bf nodeName:}" failed. No retries permitted until 2026-09-18 03:41:45.733745399 +0000 UTC m=+520.016507254 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/7b65c1dd-90a7-4030-8d8e-4860bd7313bf-tls-certs") pod "model-serving-api-77df98bf98-qhq74" (UID: "7b65c1dd-90a7-4030-8d8e-4860bd7313bf") : secret "model-serving-api-tls" not found Sep 18 03:41:45.233914 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:41:45.233896 2568 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Sep 18 03:41:45.234021 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:41:45.234005 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40c85b4f-853b-43b4-9dc7-d0f1196b8e09-cert podName:40c85b4f-853b-43b4-9dc7-d0f1196b8e09 nodeName:}" failed. No retries permitted until 2026-09-18 03:41:45.733984559 +0000 UTC m=+520.016746413 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/40c85b4f-853b-43b4-9dc7-d0f1196b8e09-cert") pod "kserve-controller-manager-7645b9c6f5-9mwtv" (UID: "40c85b4f-853b-43b4-9dc7-d0f1196b8e09") : secret "kserve-webhook-server-cert" not found Sep 18 03:41:45.262115 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:45.262086 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gn6n\" (UniqueName: \"kubernetes.io/projected/40c85b4f-853b-43b4-9dc7-d0f1196b8e09-kube-api-access-8gn6n\") pod \"kserve-controller-manager-7645b9c6f5-9mwtv\" (UID: \"40c85b4f-853b-43b4-9dc7-d0f1196b8e09\") " pod="opendatahub/kserve-controller-manager-7645b9c6f5-9mwtv" Sep 18 03:41:45.269426 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:45.269399 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmspv\" (UniqueName: \"kubernetes.io/projected/7b65c1dd-90a7-4030-8d8e-4860bd7313bf-kube-api-access-wmspv\") pod \"model-serving-api-77df98bf98-qhq74\" (UID: \"7b65c1dd-90a7-4030-8d8e-4860bd7313bf\") " pod="opendatahub/model-serving-api-77df98bf98-qhq74" Sep 18 03:41:45.738312 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:45.738277 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/40c85b4f-853b-43b4-9dc7-d0f1196b8e09-cert\") pod \"kserve-controller-manager-7645b9c6f5-9mwtv\" (UID: \"40c85b4f-853b-43b4-9dc7-d0f1196b8e09\") " pod="opendatahub/kserve-controller-manager-7645b9c6f5-9mwtv" Sep 18 03:41:45.738496 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:45.738335 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b65c1dd-90a7-4030-8d8e-4860bd7313bf-tls-certs\") pod \"model-serving-api-77df98bf98-qhq74\" (UID: \"7b65c1dd-90a7-4030-8d8e-4860bd7313bf\") " pod="opendatahub/model-serving-api-77df98bf98-qhq74" Sep 18 03:41:45.741450 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:45.741411 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b65c1dd-90a7-4030-8d8e-4860bd7313bf-tls-certs\") pod \"model-serving-api-77df98bf98-qhq74\" (UID: \"7b65c1dd-90a7-4030-8d8e-4860bd7313bf\") " pod="opendatahub/model-serving-api-77df98bf98-qhq74" Sep 18 03:41:45.741587 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:45.741410 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/40c85b4f-853b-43b4-9dc7-d0f1196b8e09-cert\") pod \"kserve-controller-manager-7645b9c6f5-9mwtv\" (UID: \"40c85b4f-853b-43b4-9dc7-d0f1196b8e09\") " pod="opendatahub/kserve-controller-manager-7645b9c6f5-9mwtv" Sep 18 03:41:45.900271 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:45.900236 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-7645b9c6f5-9mwtv" Sep 18 03:41:45.969381 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:45.969349 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-77df98bf98-qhq74" Sep 18 03:41:46.013471 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:46.013388 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-dmr68" podUID="be7d76da-e4b3-438f-9346-870958f90704" containerName="limitador" containerID="cri-o://12bd5e5b6c9219e833a07a9a86f0a4a995767e79ff33800899d1174961798cef" gracePeriod=30 Sep 18 03:41:46.537428 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:46.537402 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-77df98bf98-qhq74"] Sep 18 03:41:46.539205 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:41:46.539181 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b65c1dd_90a7_4030_8d8e_4860bd7313bf.slice/crio-911a2294a711f228ebadf1ee31ed48ebdc3ad2d894990cebc207228cd7de1f2c WatchSource:0}: Error finding container 911a2294a711f228ebadf1ee31ed48ebdc3ad2d894990cebc207228cd7de1f2c: Status 404 returned error can't find the container with id 911a2294a711f228ebadf1ee31ed48ebdc3ad2d894990cebc207228cd7de1f2c Sep 18 03:41:46.546118 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:46.546015 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-7645b9c6f5-9mwtv"] Sep 18 03:41:46.548774 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:41:46.548752 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40c85b4f_853b_43b4_9dc7_d0f1196b8e09.slice/crio-7609f410e03b8e1c2035ecfdbc687c46c3caeca87071c75dfe8eb3b7d2a39edc WatchSource:0}: Error finding container 7609f410e03b8e1c2035ecfdbc687c46c3caeca87071c75dfe8eb3b7d2a39edc: Status 404 returned error can't find the container with id 7609f410e03b8e1c2035ecfdbc687c46c3caeca87071c75dfe8eb3b7d2a39edc Sep 18 03:41:46.569063 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:46.569043 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-dmr68" Sep 18 03:41:46.645790 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:46.645762 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcbbv\" (UniqueName: \"kubernetes.io/projected/be7d76da-e4b3-438f-9346-870958f90704-kube-api-access-pcbbv\") pod \"be7d76da-e4b3-438f-9346-870958f90704\" (UID: \"be7d76da-e4b3-438f-9346-870958f90704\") " Sep 18 03:41:46.645974 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:46.645795 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/be7d76da-e4b3-438f-9346-870958f90704-config-file\") pod \"be7d76da-e4b3-438f-9346-870958f90704\" (UID: \"be7d76da-e4b3-438f-9346-870958f90704\") " Sep 18 03:41:46.646210 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:46.646187 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be7d76da-e4b3-438f-9346-870958f90704-config-file" (OuterVolumeSpecName: "config-file") pod "be7d76da-e4b3-438f-9346-870958f90704" (UID: "be7d76da-e4b3-438f-9346-870958f90704"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 03:41:46.647875 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:46.647851 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be7d76da-e4b3-438f-9346-870958f90704-kube-api-access-pcbbv" (OuterVolumeSpecName: "kube-api-access-pcbbv") pod "be7d76da-e4b3-438f-9346-870958f90704" (UID: "be7d76da-e4b3-438f-9346-870958f90704"). InnerVolumeSpecName "kube-api-access-pcbbv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 03:41:46.747431 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:46.747396 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcbbv\" (UniqueName: \"kubernetes.io/projected/be7d76da-e4b3-438f-9346-870958f90704-kube-api-access-pcbbv\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:41:46.747431 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:46.747428 2568 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/be7d76da-e4b3-438f-9346-870958f90704-config-file\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:41:47.020420 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:47.020382 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-xk5gl" event={"ID":"4c378e07-18c0-4165-81ef-6ab860b4aa7f","Type":"ContainerStarted","Data":"7d2b5a0ae6258d2eecde7c536089234fd4df8081aff4b14a926e239a812e3ff2"} Sep 18 03:41:47.020893 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:47.020588 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-xk5gl" Sep 18 03:41:47.021758 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:47.021734 2568 generic.go:356] "Generic (PLEG): container finished" podID="be7d76da-e4b3-438f-9346-870958f90704" containerID="12bd5e5b6c9219e833a07a9a86f0a4a995767e79ff33800899d1174961798cef" exitCode=0 Sep 18 03:41:47.021875 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:47.021773 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-dmr68" Sep 18 03:41:47.021875 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:47.021806 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-dmr68" event={"ID":"be7d76da-e4b3-438f-9346-870958f90704","Type":"ContainerDied","Data":"12bd5e5b6c9219e833a07a9a86f0a4a995767e79ff33800899d1174961798cef"} Sep 18 03:41:47.021875 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:47.021849 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-dmr68" event={"ID":"be7d76da-e4b3-438f-9346-870958f90704","Type":"ContainerDied","Data":"f8cd8618423fd0454fcb54c563167da5635e1e4f61eb695a02e49fcd091c9a78"} Sep 18 03:41:47.022060 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:47.021877 2568 scope.go:117] "RemoveContainer" containerID="12bd5e5b6c9219e833a07a9a86f0a4a995767e79ff33800899d1174961798cef" Sep 18 03:41:47.023225 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:47.023148 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-77df98bf98-qhq74" event={"ID":"7b65c1dd-90a7-4030-8d8e-4860bd7313bf","Type":"ContainerStarted","Data":"911a2294a711f228ebadf1ee31ed48ebdc3ad2d894990cebc207228cd7de1f2c"} Sep 18 03:41:47.024632 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:47.024610 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-7645b9c6f5-9mwtv" event={"ID":"40c85b4f-853b-43b4-9dc7-d0f1196b8e09","Type":"ContainerStarted","Data":"7609f410e03b8e1c2035ecfdbc687c46c3caeca87071c75dfe8eb3b7d2a39edc"} Sep 18 03:41:47.026356 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:47.026332 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8" event={"ID":"fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f","Type":"ContainerStarted","Data":"f4a32a0db083e3076ad340a5c31fe5d02369bb86da5ed2e7322ea742f8607778"} Sep 18 03:41:47.026472 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:47.026456 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8" Sep 18 03:41:47.031458 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:47.031443 2568 scope.go:117] "RemoveContainer" containerID="12bd5e5b6c9219e833a07a9a86f0a4a995767e79ff33800899d1174961798cef" Sep 18 03:41:47.031711 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:41:47.031694 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12bd5e5b6c9219e833a07a9a86f0a4a995767e79ff33800899d1174961798cef\": container with ID starting with 12bd5e5b6c9219e833a07a9a86f0a4a995767e79ff33800899d1174961798cef not found: ID does not exist" containerID="12bd5e5b6c9219e833a07a9a86f0a4a995767e79ff33800899d1174961798cef" Sep 18 03:41:47.031753 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:47.031719 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12bd5e5b6c9219e833a07a9a86f0a4a995767e79ff33800899d1174961798cef"} err="failed to get container status \"12bd5e5b6c9219e833a07a9a86f0a4a995767e79ff33800899d1174961798cef\": rpc error: code = NotFound desc = could not find container \"12bd5e5b6c9219e833a07a9a86f0a4a995767e79ff33800899d1174961798cef\": container with ID starting with 12bd5e5b6c9219e833a07a9a86f0a4a995767e79ff33800899d1174961798cef not found: ID does not exist" Sep 18 03:41:47.039059 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:47.039020 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-xk5gl" podStartSLOduration=2.331996316 podStartE2EDuration="4.039009318s" totalImagesPullingTime="1.707013002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:41:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:41:47.037313648 +0000 UTC m=+521.320075512" watchObservedRunningTime="2026-09-18 03:41:47.039009318 +0000 UTC m=+521.321771181" Sep 18 03:41:47.052698 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:47.052670 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-dmr68"] Sep 18 03:41:47.057903 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:47.057879 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-dmr68"] Sep 18 03:41:47.074499 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:47.074460 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8" podStartSLOduration=2.943680862 podStartE2EDuration="5.074446505s" totalImagesPullingTime="2.130765643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:41:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:41:47.072740489 +0000 UTC m=+521.355502355" watchObservedRunningTime="2026-09-18 03:41:47.074446505 +0000 UTC m=+521.357208367" Sep 18 03:41:47.884709 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:47.884672 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be7d76da-e4b3-438f-9346-870958f90704" path="/var/lib/kubelet/pods/be7d76da-e4b3-438f-9346-870958f90704/volumes" Sep 18 03:41:49.040680 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:49.040649 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-77df98bf98-qhq74" event={"ID":"7b65c1dd-90a7-4030-8d8e-4860bd7313bf","Type":"ContainerStarted","Data":"8e02e5a60f04eea1a1a0b9d522b9feed06ed123c864b4a8e4005cf2860eeb793"} Sep 18 03:41:49.041117 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:49.040759 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-77df98bf98-qhq74" Sep 18 03:41:49.042329 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:49.042301 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-7645b9c6f5-9mwtv" event={"ID":"40c85b4f-853b-43b4-9dc7-d0f1196b8e09","Type":"ContainerStarted","Data":"d4357afc6b5ea361a6e060d90d41e1cc56fe5ce86c664b872eca2010ab1acd03"} Sep 18 03:41:49.042513 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:49.042498 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-7645b9c6f5-9mwtv" Sep 18 03:41:49.058580 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:49.058530 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-77df98bf98-qhq74" podStartSLOduration=2.497078289 podStartE2EDuration="4.058517143s" totalImagesPullingTime="1.561438854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:41:49.056455385 +0000 UTC m=+523.339217250" watchObservedRunningTime="2026-09-18 03:41:49.058517143 +0000 UTC m=+523.341278997" Sep 18 03:41:49.074301 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:49.074257 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-7645b9c6f5-9mwtv" podStartSLOduration=2.7287950260000002 podStartE2EDuration="5.07424047s" totalImagesPullingTime="2.345445444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:41:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:41:49.072106236 +0000 UTC m=+523.354868100" watchObservedRunningTime="2026-09-18 03:41:49.07424047 +0000 UTC m=+523.357002333" Sep 18 03:41:53.039131 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.039057 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8" Sep 18 03:41:53.564422 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.564387 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-vm4hc"] Sep 18 03:41:53.564773 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.564759 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be7d76da-e4b3-438f-9346-870958f90704" containerName="limitador" Sep 18 03:41:53.564817 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.564774 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="be7d76da-e4b3-438f-9346-870958f90704" containerName="limitador" Sep 18 03:41:53.564856 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.564841 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="be7d76da-e4b3-438f-9346-870958f90704" containerName="limitador" Sep 18 03:41:53.567976 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.567956 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-vm4hc" Sep 18 03:41:53.570267 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.570247 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-l28gj\"" Sep 18 03:41:53.575673 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.575649 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-vm4hc"] Sep 18 03:41:53.624199 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.624171 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-vm4hc"] Sep 18 03:41:53.624419 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:41:53.624400 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-sxrzl], unattached volumes=[], failed to process volumes=[kube-api-access-sxrzl]: context canceled" pod="kuadrant-system/authorino-557f876f69-vm4hc" podUID="efc8343c-5a1a-4606-a708-4349f5c817de" Sep 18 03:41:53.650049 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.650023 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-f5bx8"] Sep 18 03:41:53.653358 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.653340 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-f5bx8" Sep 18 03:41:53.655532 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.655515 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Sep 18 03:41:53.660370 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.660348 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-f5bx8"] Sep 18 03:41:53.691361 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.691335 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-8f86fbdb5-6z6xq"] Sep 18 03:41:53.694701 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.694682 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8f86fbdb5-6z6xq" Sep 18 03:41:53.705665 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.705625 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8f86fbdb5-6z6xq"] Sep 18 03:41:53.714816 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.714794 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxrzl\" (UniqueName: \"kubernetes.io/projected/efc8343c-5a1a-4606-a708-4349f5c817de-kube-api-access-sxrzl\") pod \"authorino-557f876f69-vm4hc\" (UID: \"efc8343c-5a1a-4606-a708-4349f5c817de\") " pod="kuadrant-system/authorino-557f876f69-vm4hc" Sep 18 03:41:53.810048 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.810018 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-f5bx8"] Sep 18 03:41:53.810295 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:41:53.810276 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-49bhc tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-f5bx8" podUID="5eac1c77-4862-44ff-b0f5-64e067d5f888" Sep 18 03:41:53.815493 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.815440 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5eac1c77-4862-44ff-b0f5-64e067d5f888-tls-cert\") pod \"authorino-6d4dfc649f-f5bx8\" (UID: \"5eac1c77-4862-44ff-b0f5-64e067d5f888\") " pod="kuadrant-system/authorino-6d4dfc649f-f5bx8" Sep 18 03:41:53.815574 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.815496 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxrzl\" (UniqueName: \"kubernetes.io/projected/efc8343c-5a1a-4606-a708-4349f5c817de-kube-api-access-sxrzl\") pod \"authorino-557f876f69-vm4hc\" (UID: \"efc8343c-5a1a-4606-a708-4349f5c817de\") " pod="kuadrant-system/authorino-557f876f69-vm4hc" Sep 18 03:41:53.815574 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.815522 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63753bbc-18cb-4bce-a30c-4fe9bfe7ab94-metrics-certs\") pod \"maas-api-8f86fbdb5-6z6xq\" (UID: \"63753bbc-18cb-4bce-a30c-4fe9bfe7ab94\") " pod="odh-ai-gateway-infra/maas-api-8f86fbdb5-6z6xq" Sep 18 03:41:53.815644 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.815585 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63753bbc-18cb-4bce-a30c-4fe9bfe7ab94-maas-api-tls\") pod \"maas-api-8f86fbdb5-6z6xq\" (UID: \"63753bbc-18cb-4bce-a30c-4fe9bfe7ab94\") " pod="odh-ai-gateway-infra/maas-api-8f86fbdb5-6z6xq" Sep 18 03:41:53.815644 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.815607 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63753bbc-18cb-4bce-a30c-4fe9bfe7ab94-openshift-service-ca\") pod \"maas-api-8f86fbdb5-6z6xq\" (UID: \"63753bbc-18cb-4bce-a30c-4fe9bfe7ab94\") " pod="odh-ai-gateway-infra/maas-api-8f86fbdb5-6z6xq" Sep 18 03:41:53.815644 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.815629 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7rcb\" (UniqueName: \"kubernetes.io/projected/63753bbc-18cb-4bce-a30c-4fe9bfe7ab94-kube-api-access-f7rcb\") pod \"maas-api-8f86fbdb5-6z6xq\" (UID: \"63753bbc-18cb-4bce-a30c-4fe9bfe7ab94\") " pod="odh-ai-gateway-infra/maas-api-8f86fbdb5-6z6xq" Sep 18 03:41:53.815757 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.815662 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49bhc\" (UniqueName: \"kubernetes.io/projected/5eac1c77-4862-44ff-b0f5-64e067d5f888-kube-api-access-49bhc\") pod \"authorino-6d4dfc649f-f5bx8\" (UID: \"5eac1c77-4862-44ff-b0f5-64e067d5f888\") " pod="kuadrant-system/authorino-6d4dfc649f-f5bx8" Sep 18 03:41:53.827873 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.827837 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxrzl\" (UniqueName: \"kubernetes.io/projected/efc8343c-5a1a-4606-a708-4349f5c817de-kube-api-access-sxrzl\") pod \"authorino-557f876f69-vm4hc\" (UID: \"efc8343c-5a1a-4606-a708-4349f5c817de\") " pod="kuadrant-system/authorino-557f876f69-vm4hc" Sep 18 03:41:53.834184 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.834161 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75fd986f6b-nkbhs"] Sep 18 03:41:53.838157 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.838139 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75fd986f6b-nkbhs" Sep 18 03:41:53.848114 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.848092 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75fd986f6b-nkbhs"] Sep 18 03:41:53.916236 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.916212 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5eac1c77-4862-44ff-b0f5-64e067d5f888-tls-cert\") pod \"authorino-6d4dfc649f-f5bx8\" (UID: \"5eac1c77-4862-44ff-b0f5-64e067d5f888\") " pod="kuadrant-system/authorino-6d4dfc649f-f5bx8" Sep 18 03:41:53.916342 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.916268 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63753bbc-18cb-4bce-a30c-4fe9bfe7ab94-metrics-certs\") pod \"maas-api-8f86fbdb5-6z6xq\" (UID: \"63753bbc-18cb-4bce-a30c-4fe9bfe7ab94\") " pod="odh-ai-gateway-infra/maas-api-8f86fbdb5-6z6xq" Sep 18 03:41:53.916342 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.916308 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63753bbc-18cb-4bce-a30c-4fe9bfe7ab94-maas-api-tls\") pod \"maas-api-8f86fbdb5-6z6xq\" (UID: \"63753bbc-18cb-4bce-a30c-4fe9bfe7ab94\") " pod="odh-ai-gateway-infra/maas-api-8f86fbdb5-6z6xq" Sep 18 03:41:53.916342 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.916323 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63753bbc-18cb-4bce-a30c-4fe9bfe7ab94-openshift-service-ca\") pod \"maas-api-8f86fbdb5-6z6xq\" (UID: \"63753bbc-18cb-4bce-a30c-4fe9bfe7ab94\") " pod="odh-ai-gateway-infra/maas-api-8f86fbdb5-6z6xq" Sep 18 03:41:53.916462 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.916341 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7rcb\" (UniqueName: \"kubernetes.io/projected/63753bbc-18cb-4bce-a30c-4fe9bfe7ab94-kube-api-access-f7rcb\") pod \"maas-api-8f86fbdb5-6z6xq\" (UID: \"63753bbc-18cb-4bce-a30c-4fe9bfe7ab94\") " pod="odh-ai-gateway-infra/maas-api-8f86fbdb5-6z6xq" Sep 18 03:41:53.916462 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.916377 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49bhc\" (UniqueName: \"kubernetes.io/projected/5eac1c77-4862-44ff-b0f5-64e067d5f888-kube-api-access-49bhc\") pod \"authorino-6d4dfc649f-f5bx8\" (UID: \"5eac1c77-4862-44ff-b0f5-64e067d5f888\") " pod="kuadrant-system/authorino-6d4dfc649f-f5bx8" Sep 18 03:41:53.917063 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.917043 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63753bbc-18cb-4bce-a30c-4fe9bfe7ab94-openshift-service-ca\") pod \"maas-api-8f86fbdb5-6z6xq\" (UID: \"63753bbc-18cb-4bce-a30c-4fe9bfe7ab94\") " pod="odh-ai-gateway-infra/maas-api-8f86fbdb5-6z6xq" Sep 18 03:41:53.918758 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.918739 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63753bbc-18cb-4bce-a30c-4fe9bfe7ab94-maas-api-tls\") pod \"maas-api-8f86fbdb5-6z6xq\" (UID: \"63753bbc-18cb-4bce-a30c-4fe9bfe7ab94\") " pod="odh-ai-gateway-infra/maas-api-8f86fbdb5-6z6xq" Sep 18 03:41:53.918856 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.918761 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63753bbc-18cb-4bce-a30c-4fe9bfe7ab94-metrics-certs\") pod \"maas-api-8f86fbdb5-6z6xq\" (UID: \"63753bbc-18cb-4bce-a30c-4fe9bfe7ab94\") " pod="odh-ai-gateway-infra/maas-api-8f86fbdb5-6z6xq" Sep 18 03:41:53.918856 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.918774 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5eac1c77-4862-44ff-b0f5-64e067d5f888-tls-cert\") pod \"authorino-6d4dfc649f-f5bx8\" (UID: \"5eac1c77-4862-44ff-b0f5-64e067d5f888\") " pod="kuadrant-system/authorino-6d4dfc649f-f5bx8" Sep 18 03:41:53.923373 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.923355 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7rcb\" (UniqueName: \"kubernetes.io/projected/63753bbc-18cb-4bce-a30c-4fe9bfe7ab94-kube-api-access-f7rcb\") pod \"maas-api-8f86fbdb5-6z6xq\" (UID: \"63753bbc-18cb-4bce-a30c-4fe9bfe7ab94\") " pod="odh-ai-gateway-infra/maas-api-8f86fbdb5-6z6xq" Sep 18 03:41:53.923461 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:53.923417 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49bhc\" (UniqueName: \"kubernetes.io/projected/5eac1c77-4862-44ff-b0f5-64e067d5f888-kube-api-access-49bhc\") pod \"authorino-6d4dfc649f-f5bx8\" (UID: \"5eac1c77-4862-44ff-b0f5-64e067d5f888\") " pod="kuadrant-system/authorino-6d4dfc649f-f5bx8" Sep 18 03:41:54.006190 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:54.006155 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8f86fbdb5-6z6xq" Sep 18 03:41:54.017631 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:54.017603 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b5sn\" (UniqueName: \"kubernetes.io/projected/5779c92a-ab5d-43d5-9eb8-864ae4facd0e-kube-api-access-6b5sn\") pod \"authorino-75fd986f6b-nkbhs\" (UID: \"5779c92a-ab5d-43d5-9eb8-864ae4facd0e\") " pod="kuadrant-system/authorino-75fd986f6b-nkbhs" Sep 18 03:41:54.017753 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:54.017653 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5779c92a-ab5d-43d5-9eb8-864ae4facd0e-tls-cert\") pod \"authorino-75fd986f6b-nkbhs\" (UID: \"5779c92a-ab5d-43d5-9eb8-864ae4facd0e\") " pod="kuadrant-system/authorino-75fd986f6b-nkbhs" Sep 18 03:41:54.070401 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:54.070376 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-f5bx8" Sep 18 03:41:54.070773 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:54.070414 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-vm4hc" Sep 18 03:41:54.084398 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:54.083894 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-vm4hc" Sep 18 03:41:54.090842 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:54.090821 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-f5bx8" Sep 18 03:41:54.118159 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:54.118133 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5779c92a-ab5d-43d5-9eb8-864ae4facd0e-tls-cert\") pod \"authorino-75fd986f6b-nkbhs\" (UID: \"5779c92a-ab5d-43d5-9eb8-864ae4facd0e\") " pod="kuadrant-system/authorino-75fd986f6b-nkbhs" Sep 18 03:41:54.118337 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:54.118229 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b5sn\" (UniqueName: \"kubernetes.io/projected/5779c92a-ab5d-43d5-9eb8-864ae4facd0e-kube-api-access-6b5sn\") pod \"authorino-75fd986f6b-nkbhs\" (UID: \"5779c92a-ab5d-43d5-9eb8-864ae4facd0e\") " pod="kuadrant-system/authorino-75fd986f6b-nkbhs" Sep 18 03:41:54.120585 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:54.120564 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5779c92a-ab5d-43d5-9eb8-864ae4facd0e-tls-cert\") pod \"authorino-75fd986f6b-nkbhs\" (UID: \"5779c92a-ab5d-43d5-9eb8-864ae4facd0e\") " pod="kuadrant-system/authorino-75fd986f6b-nkbhs" Sep 18 03:41:54.125212 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:54.125194 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b5sn\" (UniqueName: \"kubernetes.io/projected/5779c92a-ab5d-43d5-9eb8-864ae4facd0e-kube-api-access-6b5sn\") pod \"authorino-75fd986f6b-nkbhs\" (UID: \"5779c92a-ab5d-43d5-9eb8-864ae4facd0e\") " pod="kuadrant-system/authorino-75fd986f6b-nkbhs" Sep 18 03:41:54.147786 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:54.147766 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75fd986f6b-nkbhs" Sep 18 03:41:54.219610 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:54.219570 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxrzl\" (UniqueName: \"kubernetes.io/projected/efc8343c-5a1a-4606-a708-4349f5c817de-kube-api-access-sxrzl\") pod \"efc8343c-5a1a-4606-a708-4349f5c817de\" (UID: \"efc8343c-5a1a-4606-a708-4349f5c817de\") " Sep 18 03:41:54.219715 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:54.219685 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5eac1c77-4862-44ff-b0f5-64e067d5f888-tls-cert\") pod \"5eac1c77-4862-44ff-b0f5-64e067d5f888\" (UID: \"5eac1c77-4862-44ff-b0f5-64e067d5f888\") " Sep 18 03:41:54.219715 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:54.219709 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49bhc\" (UniqueName: \"kubernetes.io/projected/5eac1c77-4862-44ff-b0f5-64e067d5f888-kube-api-access-49bhc\") pod \"5eac1c77-4862-44ff-b0f5-64e067d5f888\" (UID: \"5eac1c77-4862-44ff-b0f5-64e067d5f888\") " Sep 18 03:41:54.221961 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:54.221891 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efc8343c-5a1a-4606-a708-4349f5c817de-kube-api-access-sxrzl" (OuterVolumeSpecName: "kube-api-access-sxrzl") pod "efc8343c-5a1a-4606-a708-4349f5c817de" (UID: "efc8343c-5a1a-4606-a708-4349f5c817de"). InnerVolumeSpecName "kube-api-access-sxrzl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 03:41:54.222220 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:54.222186 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5eac1c77-4862-44ff-b0f5-64e067d5f888-kube-api-access-49bhc" (OuterVolumeSpecName: "kube-api-access-49bhc") pod "5eac1c77-4862-44ff-b0f5-64e067d5f888" (UID: "5eac1c77-4862-44ff-b0f5-64e067d5f888"). InnerVolumeSpecName "kube-api-access-49bhc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 03:41:54.222304 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:54.222256 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5eac1c77-4862-44ff-b0f5-64e067d5f888-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5eac1c77-4862-44ff-b0f5-64e067d5f888" (UID: "5eac1c77-4862-44ff-b0f5-64e067d5f888"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:41:54.286130 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:54.286108 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8f86fbdb5-6z6xq"] Sep 18 03:41:54.288568 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:41:54.288539 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63753bbc_18cb_4bce_a30c_4fe9bfe7ab94.slice/crio-60899b2192c2fecb1a8c406a94443e895bf39f82a4de41b3853b771206e1ad9f WatchSource:0}: Error finding container 60899b2192c2fecb1a8c406a94443e895bf39f82a4de41b3853b771206e1ad9f: Status 404 returned error can't find the container with id 60899b2192c2fecb1a8c406a94443e895bf39f82a4de41b3853b771206e1ad9f Sep 18 03:41:54.320988 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:54.320894 2568 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5eac1c77-4862-44ff-b0f5-64e067d5f888-tls-cert\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:41:54.320988 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:54.320921 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49bhc\" (UniqueName: \"kubernetes.io/projected/5eac1c77-4862-44ff-b0f5-64e067d5f888-kube-api-access-49bhc\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:41:54.320988 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:54.320957 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxrzl\" (UniqueName: \"kubernetes.io/projected/efc8343c-5a1a-4606-a708-4349f5c817de-kube-api-access-sxrzl\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:41:54.415087 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:54.414909 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75fd986f6b-nkbhs"] Sep 18 03:41:54.417164 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:41:54.417140 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5779c92a_ab5d_43d5_9eb8_864ae4facd0e.slice/crio-219fc3cfbad13daba89aadbb00df2091e9e799190469f3d929ef20d6af2296b3 WatchSource:0}: Error finding container 219fc3cfbad13daba89aadbb00df2091e9e799190469f3d929ef20d6af2296b3: Status 404 returned error can't find the container with id 219fc3cfbad13daba89aadbb00df2091e9e799190469f3d929ef20d6af2296b3 Sep 18 03:41:55.078004 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:55.077961 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75fd986f6b-nkbhs" event={"ID":"5779c92a-ab5d-43d5-9eb8-864ae4facd0e","Type":"ContainerStarted","Data":"219fc3cfbad13daba89aadbb00df2091e9e799190469f3d929ef20d6af2296b3"} Sep 18 03:41:55.079781 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:55.079747 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8f86fbdb5-6z6xq" event={"ID":"63753bbc-18cb-4bce-a30c-4fe9bfe7ab94","Type":"ContainerStarted","Data":"02290f8287a93a4fcc3770b00ea4345b7dc20c72430d89904cc99367cea0420a"} Sep 18 03:41:55.079927 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:55.079806 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8f86fbdb5-6z6xq" event={"ID":"63753bbc-18cb-4bce-a30c-4fe9bfe7ab94","Type":"ContainerStarted","Data":"60899b2192c2fecb1a8c406a94443e895bf39f82a4de41b3853b771206e1ad9f"} Sep 18 03:41:55.079927 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:55.079836 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-8f86fbdb5-6z6xq" Sep 18 03:41:55.079927 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:55.079835 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-vm4hc" Sep 18 03:41:55.080088 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:55.080026 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-f5bx8" Sep 18 03:41:55.096004 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:55.095960 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-8f86fbdb5-6z6xq" podStartSLOduration=1.6284123240000001 podStartE2EDuration="2.095925599s" totalImagesPullingTime="467.513275ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:41:55.094953711 +0000 UTC m=+529.377715571" watchObservedRunningTime="2026-09-18 03:41:55.095925599 +0000 UTC m=+529.378687465" Sep 18 03:41:55.131649 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:55.131620 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-f5bx8"] Sep 18 03:41:55.133488 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:55.133464 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-f5bx8"] Sep 18 03:41:55.155194 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:55.155171 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-vm4hc"] Sep 18 03:41:55.158528 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:55.158507 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-vm4hc"] Sep 18 03:41:55.883836 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:55.883803 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5eac1c77-4862-44ff-b0f5-64e067d5f888" path="/var/lib/kubelet/pods/5eac1c77-4862-44ff-b0f5-64e067d5f888/volumes" Sep 18 03:41:55.884104 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:55.884079 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efc8343c-5a1a-4606-a708-4349f5c817de" path="/var/lib/kubelet/pods/efc8343c-5a1a-4606-a708-4349f5c817de/volumes" Sep 18 03:41:56.087784 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:56.087746 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75fd986f6b-nkbhs" event={"ID":"5779c92a-ab5d-43d5-9eb8-864ae4facd0e","Type":"ContainerStarted","Data":"1bf22cf3d7c24d6b402cdce5b9f0d160ba61b12be2dbbee5aece5aab73999d4b"} Sep 18 03:41:56.107943 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:56.107884 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75fd986f6b-nkbhs" podStartSLOduration=2.449325707 podStartE2EDuration="3.107868359s" totalImagesPullingTime="658.542652ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:41:56.106775877 +0000 UTC m=+530.389537740" watchObservedRunningTime="2026-09-18 03:41:56.107868359 +0000 UTC m=+530.390630221" Sep 18 03:41:58.035357 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:41:58.035321 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-xk5gl" Sep 18 03:42:00.050842 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:00.050814 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-77df98bf98-qhq74" Sep 18 03:42:01.093015 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:01.092976 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-8f86fbdb5-6z6xq" Sep 18 03:42:01.135162 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:01.135130 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8"] Sep 18 03:42:01.135849 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:01.135395 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8" podUID="fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f" containerName="maas-api" containerID="cri-o://f4a32a0db083e3076ad340a5c31fe5d02369bb86da5ed2e7322ea742f8607778" gracePeriod=30 Sep 18 03:42:01.376080 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:01.376053 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8" Sep 18 03:42:01.382143 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:01.382125 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f-maas-api-tls\") pod \"fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f\" (UID: \"fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f\") " Sep 18 03:42:01.382192 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:01.382153 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f-metrics-certs\") pod \"fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f\" (UID: \"fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f\") " Sep 18 03:42:01.382192 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:01.382170 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xnhc\" (UniqueName: \"kubernetes.io/projected/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f-kube-api-access-7xnhc\") pod \"fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f\" (UID: \"fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f\") " Sep 18 03:42:01.382282 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:01.382237 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f-openshift-service-ca\") pod \"fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f\" (UID: \"fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f\") " Sep 18 03:42:01.382604 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:01.382577 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f" (UID: "fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 03:42:01.384312 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:01.384285 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f" (UID: "fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:42:01.384405 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:01.384311 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f" (UID: "fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:42:01.384405 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:01.384342 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f-kube-api-access-7xnhc" (OuterVolumeSpecName: "kube-api-access-7xnhc") pod "fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f" (UID: "fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f"). InnerVolumeSpecName "kube-api-access-7xnhc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 03:42:01.483366 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:01.483323 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f-openshift-service-ca\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:42:01.483366 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:01.483361 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f-maas-api-tls\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:42:01.483366 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:01.483372 2568 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f-metrics-certs\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:42:01.483597 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:01.483380 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xnhc\" (UniqueName: \"kubernetes.io/projected/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f-kube-api-access-7xnhc\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:42:02.119690 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:02.119657 2568 generic.go:356] "Generic (PLEG): container finished" podID="fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f" containerID="f4a32a0db083e3076ad340a5c31fe5d02369bb86da5ed2e7322ea742f8607778" exitCode=0 Sep 18 03:42:02.120168 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:02.119700 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8" Sep 18 03:42:02.120168 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:02.119737 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8" event={"ID":"fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f","Type":"ContainerDied","Data":"f4a32a0db083e3076ad340a5c31fe5d02369bb86da5ed2e7322ea742f8607778"} Sep 18 03:42:02.120168 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:02.119798 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8" event={"ID":"fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f","Type":"ContainerDied","Data":"b0dd2a7be6fa8cc563df4455bdb3947fe5ef54d9fb79057452c4774763621518"} Sep 18 03:42:02.120168 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:02.119832 2568 scope.go:117] "RemoveContainer" containerID="f4a32a0db083e3076ad340a5c31fe5d02369bb86da5ed2e7322ea742f8607778" Sep 18 03:42:02.128897 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:02.128877 2568 scope.go:117] "RemoveContainer" containerID="f4a32a0db083e3076ad340a5c31fe5d02369bb86da5ed2e7322ea742f8607778" Sep 18 03:42:02.129251 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:42:02.129232 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4a32a0db083e3076ad340a5c31fe5d02369bb86da5ed2e7322ea742f8607778\": container with ID starting with f4a32a0db083e3076ad340a5c31fe5d02369bb86da5ed2e7322ea742f8607778 not found: ID does not exist" containerID="f4a32a0db083e3076ad340a5c31fe5d02369bb86da5ed2e7322ea742f8607778" Sep 18 03:42:02.129320 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:02.129262 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4a32a0db083e3076ad340a5c31fe5d02369bb86da5ed2e7322ea742f8607778"} err="failed to get container status \"f4a32a0db083e3076ad340a5c31fe5d02369bb86da5ed2e7322ea742f8607778\": rpc error: code = NotFound desc = could not find container \"f4a32a0db083e3076ad340a5c31fe5d02369bb86da5ed2e7322ea742f8607778\": container with ID starting with f4a32a0db083e3076ad340a5c31fe5d02369bb86da5ed2e7322ea742f8607778 not found: ID does not exist" Sep 18 03:42:02.135270 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:02.135247 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8"] Sep 18 03:42:02.138761 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:02.138738 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d7dbfc498-dpvb8"] Sep 18 03:42:03.886285 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:03.886254 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f" path="/var/lib/kubelet/pods/fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f/volumes" Sep 18 03:42:20.053072 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:42:20.053040 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-7645b9c6f5-9mwtv" Sep 18 03:43:07.441045 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.441007 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs"] Sep 18 03:43:07.441448 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.441393 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f" containerName="maas-api" Sep 18 03:43:07.441448 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.441406 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f" containerName="maas-api" Sep 18 03:43:07.441532 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.441480 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd581b98-b4f7-4c6c-9c1d-71f7afd6ec1f" containerName="maas-api" Sep 18 03:43:07.445860 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.445839 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs" Sep 18 03:43:07.449798 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.449766 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 18 03:43:07.450117 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.450099 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Sep 18 03:43:07.450452 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.450221 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 18 03:43:07.450452 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.450351 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-hhmv2\"" Sep 18 03:43:07.456888 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.456860 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs"] Sep 18 03:43:07.552873 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.552840 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05fab7ac-a166-4e6f-956d-ccf0582a0063-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs\" (UID: \"05fab7ac-a166-4e6f-956d-ccf0582a0063\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs" Sep 18 03:43:07.553064 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.552928 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05fab7ac-a166-4e6f-956d-ccf0582a0063-home\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs\" (UID: \"05fab7ac-a166-4e6f-956d-ccf0582a0063\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs" Sep 18 03:43:07.553064 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.553001 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k2hr\" (UniqueName: \"kubernetes.io/projected/05fab7ac-a166-4e6f-956d-ccf0582a0063-kube-api-access-4k2hr\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs\" (UID: \"05fab7ac-a166-4e6f-956d-ccf0582a0063\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs" Sep 18 03:43:07.553064 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.553038 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05fab7ac-a166-4e6f-956d-ccf0582a0063-dshm\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs\" (UID: \"05fab7ac-a166-4e6f-956d-ccf0582a0063\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs" Sep 18 03:43:07.553173 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.553144 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05fab7ac-a166-4e6f-956d-ccf0582a0063-model-cache\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs\" (UID: \"05fab7ac-a166-4e6f-956d-ccf0582a0063\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs" Sep 18 03:43:07.553205 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.553170 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05fab7ac-a166-4e6f-956d-ccf0582a0063-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs\" (UID: \"05fab7ac-a166-4e6f-956d-ccf0582a0063\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs" Sep 18 03:43:07.654209 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.654165 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05fab7ac-a166-4e6f-956d-ccf0582a0063-dshm\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs\" (UID: \"05fab7ac-a166-4e6f-956d-ccf0582a0063\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs" Sep 18 03:43:07.654381 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.654238 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05fab7ac-a166-4e6f-956d-ccf0582a0063-model-cache\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs\" (UID: \"05fab7ac-a166-4e6f-956d-ccf0582a0063\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs" Sep 18 03:43:07.654381 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.654258 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05fab7ac-a166-4e6f-956d-ccf0582a0063-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs\" (UID: \"05fab7ac-a166-4e6f-956d-ccf0582a0063\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs" Sep 18 03:43:07.654381 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.654278 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05fab7ac-a166-4e6f-956d-ccf0582a0063-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs\" (UID: \"05fab7ac-a166-4e6f-956d-ccf0582a0063\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs" Sep 18 03:43:07.654381 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.654312 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05fab7ac-a166-4e6f-956d-ccf0582a0063-home\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs\" (UID: \"05fab7ac-a166-4e6f-956d-ccf0582a0063\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs" Sep 18 03:43:07.654381 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.654329 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k2hr\" (UniqueName: \"kubernetes.io/projected/05fab7ac-a166-4e6f-956d-ccf0582a0063-kube-api-access-4k2hr\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs\" (UID: \"05fab7ac-a166-4e6f-956d-ccf0582a0063\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs" Sep 18 03:43:07.654704 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.654677 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05fab7ac-a166-4e6f-956d-ccf0582a0063-model-cache\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs\" (UID: \"05fab7ac-a166-4e6f-956d-ccf0582a0063\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs" Sep 18 03:43:07.654829 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.654711 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05fab7ac-a166-4e6f-956d-ccf0582a0063-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs\" (UID: \"05fab7ac-a166-4e6f-956d-ccf0582a0063\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs" Sep 18 03:43:07.654829 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.654753 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05fab7ac-a166-4e6f-956d-ccf0582a0063-home\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs\" (UID: \"05fab7ac-a166-4e6f-956d-ccf0582a0063\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs" Sep 18 03:43:07.656511 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.656493 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05fab7ac-a166-4e6f-956d-ccf0582a0063-dshm\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs\" (UID: \"05fab7ac-a166-4e6f-956d-ccf0582a0063\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs" Sep 18 03:43:07.657025 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.657010 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05fab7ac-a166-4e6f-956d-ccf0582a0063-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs\" (UID: \"05fab7ac-a166-4e6f-956d-ccf0582a0063\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs" Sep 18 03:43:07.661843 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.661818 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k2hr\" (UniqueName: \"kubernetes.io/projected/05fab7ac-a166-4e6f-956d-ccf0582a0063-kube-api-access-4k2hr\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs\" (UID: \"05fab7ac-a166-4e6f-956d-ccf0582a0063\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs" Sep 18 03:43:07.768254 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.768173 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-hhmv2\"" Sep 18 03:43:07.775996 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.775971 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs" Sep 18 03:43:07.828468 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.828439 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psksv_1013fa23-5170-4d1f-8722-86bc4b8b8742/ovn-acl-logging/0.log" Sep 18 03:43:07.836491 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:07.835640 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psksv_1013fa23-5170-4d1f-8722-86bc4b8b8742/ovn-acl-logging/0.log" Sep 18 03:43:08.055442 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:08.055416 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs"] Sep 18 03:43:08.058451 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:43:08.058422 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05fab7ac_a166_4e6f_956d_ccf0582a0063.slice/crio-3921a51842c2861994e29d7921658612a1064ad475e3c411bb17eb326c08272e WatchSource:0}: Error finding container 3921a51842c2861994e29d7921658612a1064ad475e3c411bb17eb326c08272e: Status 404 returned error can't find the container with id 3921a51842c2861994e29d7921658612a1064ad475e3c411bb17eb326c08272e Sep 18 03:43:08.442020 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:08.441973 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs" event={"ID":"05fab7ac-a166-4e6f-956d-ccf0582a0063","Type":"ContainerStarted","Data":"3921a51842c2861994e29d7921658612a1064ad475e3c411bb17eb326c08272e"} Sep 18 03:43:09.450825 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:09.450730 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs" event={"ID":"05fab7ac-a166-4e6f-956d-ccf0582a0063","Type":"ContainerStarted","Data":"9810802d622553f64abcdf74f818bc7de011d1f6e26638c9cbe8a1027fef5be0"} Sep 18 03:43:09.467921 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:09.467872 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs" podStartSLOduration=1.339468962 podStartE2EDuration="2.467855913s" totalImagesPullingTime="1.128386951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:43:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:43:09.467251591 +0000 UTC m=+603.750013454" watchObservedRunningTime="2026-09-18 03:43:09.467855913 +0000 UTC m=+603.750617776" Sep 18 03:43:09.776507 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:09.776412 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs" Sep 18 03:43:17.776990 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:17.776955 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs" Sep 18 03:43:17.789116 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:17.789093 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs" Sep 18 03:43:18.506822 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:18.506796 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs" Sep 18 03:43:21.131777 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:21.131700 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk"] Sep 18 03:43:21.135685 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:21.135663 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk" Sep 18 03:43:21.139360 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:21.139338 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Sep 18 03:43:21.147338 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:21.147316 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk"] Sep 18 03:43:21.277537 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:21.277497 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/266193c9-df7d-4254-9343-2b51c3eab275-home\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-whmdk\" (UID: \"266193c9-df7d-4254-9343-2b51c3eab275\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk" Sep 18 03:43:21.277537 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:21.277539 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/266193c9-df7d-4254-9343-2b51c3eab275-tls-certs\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-whmdk\" (UID: \"266193c9-df7d-4254-9343-2b51c3eab275\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk" Sep 18 03:43:21.277755 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:21.277592 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpl28\" (UniqueName: \"kubernetes.io/projected/266193c9-df7d-4254-9343-2b51c3eab275-kube-api-access-gpl28\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-whmdk\" (UID: \"266193c9-df7d-4254-9343-2b51c3eab275\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk" Sep 18 03:43:21.277755 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:21.277619 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/266193c9-df7d-4254-9343-2b51c3eab275-model-cache\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-whmdk\" (UID: \"266193c9-df7d-4254-9343-2b51c3eab275\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk" Sep 18 03:43:21.277755 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:21.277658 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/266193c9-df7d-4254-9343-2b51c3eab275-tmp-dir\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-whmdk\" (UID: \"266193c9-df7d-4254-9343-2b51c3eab275\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk" Sep 18 03:43:21.277755 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:21.277718 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/266193c9-df7d-4254-9343-2b51c3eab275-dshm\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-whmdk\" (UID: \"266193c9-df7d-4254-9343-2b51c3eab275\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk" Sep 18 03:43:21.378982 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:21.378916 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/266193c9-df7d-4254-9343-2b51c3eab275-dshm\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-whmdk\" (UID: \"266193c9-df7d-4254-9343-2b51c3eab275\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk" Sep 18 03:43:21.379137 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:21.379009 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/266193c9-df7d-4254-9343-2b51c3eab275-home\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-whmdk\" (UID: \"266193c9-df7d-4254-9343-2b51c3eab275\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk" Sep 18 03:43:21.379137 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:21.379028 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/266193c9-df7d-4254-9343-2b51c3eab275-tls-certs\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-whmdk\" (UID: \"266193c9-df7d-4254-9343-2b51c3eab275\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk" Sep 18 03:43:21.379137 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:21.379062 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpl28\" (UniqueName: \"kubernetes.io/projected/266193c9-df7d-4254-9343-2b51c3eab275-kube-api-access-gpl28\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-whmdk\" (UID: \"266193c9-df7d-4254-9343-2b51c3eab275\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk" Sep 18 03:43:21.379294 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:21.379206 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/266193c9-df7d-4254-9343-2b51c3eab275-model-cache\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-whmdk\" (UID: \"266193c9-df7d-4254-9343-2b51c3eab275\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk" Sep 18 03:43:21.379346 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:21.379289 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/266193c9-df7d-4254-9343-2b51c3eab275-tmp-dir\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-whmdk\" (UID: \"266193c9-df7d-4254-9343-2b51c3eab275\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk" Sep 18 03:43:21.379428 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:21.379407 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/266193c9-df7d-4254-9343-2b51c3eab275-home\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-whmdk\" (UID: \"266193c9-df7d-4254-9343-2b51c3eab275\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk" Sep 18 03:43:21.379536 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:21.379517 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/266193c9-df7d-4254-9343-2b51c3eab275-model-cache\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-whmdk\" (UID: \"266193c9-df7d-4254-9343-2b51c3eab275\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk" Sep 18 03:43:21.379624 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:21.379604 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/266193c9-df7d-4254-9343-2b51c3eab275-tmp-dir\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-whmdk\" (UID: \"266193c9-df7d-4254-9343-2b51c3eab275\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk" Sep 18 03:43:21.381311 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:21.381284 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/266193c9-df7d-4254-9343-2b51c3eab275-dshm\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-whmdk\" (UID: \"266193c9-df7d-4254-9343-2b51c3eab275\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk" Sep 18 03:43:21.381498 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:21.381483 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/266193c9-df7d-4254-9343-2b51c3eab275-tls-certs\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-whmdk\" (UID: \"266193c9-df7d-4254-9343-2b51c3eab275\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk" Sep 18 03:43:21.386506 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:21.386454 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpl28\" (UniqueName: \"kubernetes.io/projected/266193c9-df7d-4254-9343-2b51c3eab275-kube-api-access-gpl28\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-whmdk\" (UID: \"266193c9-df7d-4254-9343-2b51c3eab275\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk" Sep 18 03:43:21.446436 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:21.446410 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk" Sep 18 03:43:21.729500 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:21.726255 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk"] Sep 18 03:43:22.522837 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:22.522793 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk" event={"ID":"266193c9-df7d-4254-9343-2b51c3eab275","Type":"ContainerStarted","Data":"599e2209e08a1bb6ebfe6912fa226b22f124eb420e7205fe9d4d01275049a694"} Sep 18 03:43:22.523290 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:22.522871 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk" event={"ID":"266193c9-df7d-4254-9343-2b51c3eab275","Type":"ContainerStarted","Data":"b420f481788bd257ad0410862603be0f69ee6433494e81ba26e3207ce9693717"} Sep 18 03:43:22.541142 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:22.541093 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk" podStartSLOduration=1.363416622 podStartE2EDuration="1.541078259s" totalImagesPullingTime="177.661637ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:43:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:43:22.538529496 +0000 UTC m=+616.821291370" watchObservedRunningTime="2026-09-18 03:43:22.541078259 +0000 UTC m=+616.823840122" Sep 18 03:43:23.446810 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:23.446774 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk" Sep 18 03:43:31.447225 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:31.447183 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk" Sep 18 03:43:31.459736 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:31.459703 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk" Sep 18 03:43:31.578880 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:43:31.578848 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-whmdk" Sep 18 03:44:17.240244 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:17.240213 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl"] Sep 18 03:44:17.244201 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:17.244179 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl" Sep 18 03:44:17.246563 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:17.246545 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Sep 18 03:44:17.259174 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:17.259147 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl"] Sep 18 03:44:17.387349 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:17.387308 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20dc98be-e3fc-42da-8e1b-54711e229f3b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl\" (UID: \"20dc98be-e3fc-42da-8e1b-54711e229f3b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl" Sep 18 03:44:17.387349 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:17.387349 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20dc98be-e3fc-42da-8e1b-54711e229f3b-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl\" (UID: \"20dc98be-e3fc-42da-8e1b-54711e229f3b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl" Sep 18 03:44:17.387624 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:17.387504 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20dc98be-e3fc-42da-8e1b-54711e229f3b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl\" (UID: \"20dc98be-e3fc-42da-8e1b-54711e229f3b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl" Sep 18 03:44:17.387624 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:17.387548 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20dc98be-e3fc-42da-8e1b-54711e229f3b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl\" (UID: \"20dc98be-e3fc-42da-8e1b-54711e229f3b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl" Sep 18 03:44:17.387744 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:17.387641 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42t6m\" (UniqueName: \"kubernetes.io/projected/20dc98be-e3fc-42da-8e1b-54711e229f3b-kube-api-access-42t6m\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl\" (UID: \"20dc98be-e3fc-42da-8e1b-54711e229f3b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl" Sep 18 03:44:17.387744 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:17.387674 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20dc98be-e3fc-42da-8e1b-54711e229f3b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl\" (UID: \"20dc98be-e3fc-42da-8e1b-54711e229f3b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl" Sep 18 03:44:17.488720 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:17.488672 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20dc98be-e3fc-42da-8e1b-54711e229f3b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl\" (UID: \"20dc98be-e3fc-42da-8e1b-54711e229f3b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl" Sep 18 03:44:17.488720 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:17.488713 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20dc98be-e3fc-42da-8e1b-54711e229f3b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl\" (UID: \"20dc98be-e3fc-42da-8e1b-54711e229f3b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl" Sep 18 03:44:17.488985 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:17.488758 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42t6m\" (UniqueName: \"kubernetes.io/projected/20dc98be-e3fc-42da-8e1b-54711e229f3b-kube-api-access-42t6m\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl\" (UID: \"20dc98be-e3fc-42da-8e1b-54711e229f3b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl" Sep 18 03:44:17.488985 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:17.488778 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20dc98be-e3fc-42da-8e1b-54711e229f3b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl\" (UID: \"20dc98be-e3fc-42da-8e1b-54711e229f3b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl" Sep 18 03:44:17.488985 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:17.488946 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20dc98be-e3fc-42da-8e1b-54711e229f3b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl\" (UID: \"20dc98be-e3fc-42da-8e1b-54711e229f3b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl" Sep 18 03:44:17.489194 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:17.488987 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20dc98be-e3fc-42da-8e1b-54711e229f3b-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl\" (UID: \"20dc98be-e3fc-42da-8e1b-54711e229f3b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl" Sep 18 03:44:17.489194 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:17.489138 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20dc98be-e3fc-42da-8e1b-54711e229f3b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl\" (UID: \"20dc98be-e3fc-42da-8e1b-54711e229f3b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl" Sep 18 03:44:17.489305 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:17.489262 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20dc98be-e3fc-42da-8e1b-54711e229f3b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl\" (UID: \"20dc98be-e3fc-42da-8e1b-54711e229f3b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl" Sep 18 03:44:17.489384 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:17.489360 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20dc98be-e3fc-42da-8e1b-54711e229f3b-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl\" (UID: \"20dc98be-e3fc-42da-8e1b-54711e229f3b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl" Sep 18 03:44:17.491182 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:17.491129 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20dc98be-e3fc-42da-8e1b-54711e229f3b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl\" (UID: \"20dc98be-e3fc-42da-8e1b-54711e229f3b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl" Sep 18 03:44:17.491283 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:17.491233 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20dc98be-e3fc-42da-8e1b-54711e229f3b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl\" (UID: \"20dc98be-e3fc-42da-8e1b-54711e229f3b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl" Sep 18 03:44:17.496812 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:17.496789 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42t6m\" (UniqueName: \"kubernetes.io/projected/20dc98be-e3fc-42da-8e1b-54711e229f3b-kube-api-access-42t6m\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl\" (UID: \"20dc98be-e3fc-42da-8e1b-54711e229f3b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl" Sep 18 03:44:17.556371 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:17.556335 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl" Sep 18 03:44:17.827841 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:17.827818 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl"] Sep 18 03:44:17.830126 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:44:17.830091 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20dc98be_e3fc_42da_8e1b_54711e229f3b.slice/crio-9d71580b8123f6ce51512346e5aed404bb2fca6fda762333cf6bba5c9f3c88dd WatchSource:0}: Error finding container 9d71580b8123f6ce51512346e5aed404bb2fca6fda762333cf6bba5c9f3c88dd: Status 404 returned error can't find the container with id 9d71580b8123f6ce51512346e5aed404bb2fca6fda762333cf6bba5c9f3c88dd Sep 18 03:44:18.811154 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:18.811112 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl" event={"ID":"20dc98be-e3fc-42da-8e1b-54711e229f3b","Type":"ContainerStarted","Data":"ca062b48745f37054530c9cbc91da1984cc57765b150199e209f6bc0065c3b9d"} Sep 18 03:44:18.811557 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:18.811172 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl" event={"ID":"20dc98be-e3fc-42da-8e1b-54711e229f3b","Type":"ContainerStarted","Data":"9d71580b8123f6ce51512346e5aed404bb2fca6fda762333cf6bba5c9f3c88dd"} Sep 18 03:44:18.832775 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:18.832725 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl" podStartSLOduration=1.666107053 podStartE2EDuration="1.832710253s" totalImagesPullingTime="166.6032ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:44:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:44:18.830490466 +0000 UTC m=+673.113252330" watchObservedRunningTime="2026-09-18 03:44:18.832710253 +0000 UTC m=+673.115472116" Sep 18 03:44:19.557378 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:19.557343 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl" Sep 18 03:44:27.556731 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:27.556690 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl" Sep 18 03:44:27.569091 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:27.569065 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl" Sep 18 03:44:27.866629 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:44:27.866543 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl" Sep 18 03:45:42.555753 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:42.555717 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58dc94667-lmz88"] Sep 18 03:45:42.559589 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:42.559571 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58dc94667-lmz88" Sep 18 03:45:42.564720 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:42.564695 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58dc94667-lmz88"] Sep 18 03:45:42.659882 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:42.659849 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5bf5f787-0abb-4a3d-8414-8d83f7a19e2f-tls-cert\") pod \"authorino-58dc94667-lmz88\" (UID: \"5bf5f787-0abb-4a3d-8414-8d83f7a19e2f\") " pod="kuadrant-system/authorino-58dc94667-lmz88" Sep 18 03:45:42.660088 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:42.659897 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rtfr\" (UniqueName: \"kubernetes.io/projected/5bf5f787-0abb-4a3d-8414-8d83f7a19e2f-kube-api-access-7rtfr\") pod \"authorino-58dc94667-lmz88\" (UID: \"5bf5f787-0abb-4a3d-8414-8d83f7a19e2f\") " pod="kuadrant-system/authorino-58dc94667-lmz88" Sep 18 03:45:42.760738 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:42.760701 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5bf5f787-0abb-4a3d-8414-8d83f7a19e2f-tls-cert\") pod \"authorino-58dc94667-lmz88\" (UID: \"5bf5f787-0abb-4a3d-8414-8d83f7a19e2f\") " pod="kuadrant-system/authorino-58dc94667-lmz88" Sep 18 03:45:42.760912 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:42.760753 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rtfr\" (UniqueName: \"kubernetes.io/projected/5bf5f787-0abb-4a3d-8414-8d83f7a19e2f-kube-api-access-7rtfr\") pod \"authorino-58dc94667-lmz88\" (UID: \"5bf5f787-0abb-4a3d-8414-8d83f7a19e2f\") " pod="kuadrant-system/authorino-58dc94667-lmz88" Sep 18 03:45:42.763229 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:42.763197 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5bf5f787-0abb-4a3d-8414-8d83f7a19e2f-tls-cert\") pod \"authorino-58dc94667-lmz88\" (UID: \"5bf5f787-0abb-4a3d-8414-8d83f7a19e2f\") " pod="kuadrant-system/authorino-58dc94667-lmz88" Sep 18 03:45:42.768093 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:42.768068 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rtfr\" (UniqueName: \"kubernetes.io/projected/5bf5f787-0abb-4a3d-8414-8d83f7a19e2f-kube-api-access-7rtfr\") pod \"authorino-58dc94667-lmz88\" (UID: \"5bf5f787-0abb-4a3d-8414-8d83f7a19e2f\") " pod="kuadrant-system/authorino-58dc94667-lmz88" Sep 18 03:45:42.870260 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:42.870180 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58dc94667-lmz88" Sep 18 03:45:43.137296 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:43.137271 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58dc94667-lmz88"] Sep 18 03:45:43.139409 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:45:43.139381 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5bf5f787_0abb_4a3d_8414_8d83f7a19e2f.slice/crio-284d738c2f8759e4de06e80688ad840bb30411bdee4337bef4e49520892fe6e5 WatchSource:0}: Error finding container 284d738c2f8759e4de06e80688ad840bb30411bdee4337bef4e49520892fe6e5: Status 404 returned error can't find the container with id 284d738c2f8759e4de06e80688ad840bb30411bdee4337bef4e49520892fe6e5 Sep 18 03:45:43.140754 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:43.140731 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 18 03:45:43.232949 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:43.232904 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58dc94667-lmz88" event={"ID":"5bf5f787-0abb-4a3d-8414-8d83f7a19e2f","Type":"ContainerStarted","Data":"284d738c2f8759e4de06e80688ad840bb30411bdee4337bef4e49520892fe6e5"} Sep 18 03:45:44.241764 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:44.241668 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58dc94667-lmz88" event={"ID":"5bf5f787-0abb-4a3d-8414-8d83f7a19e2f","Type":"ContainerStarted","Data":"4c9fde5ef515672f7661c5bf3576b1477809c3ea858e349526d291cd98eb6d48"} Sep 18 03:45:44.260296 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:44.260245 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58dc94667-lmz88" podStartSLOduration=1.430641495 podStartE2EDuration="2.260228704s" totalImagesPullingTime="829.587209ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:45:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:45:44.258692066 +0000 UTC m=+758.541453942" watchObservedRunningTime="2026-09-18 03:45:44.260228704 +0000 UTC m=+758.542990595" Sep 18 03:45:44.294751 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:44.294714 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75fd986f6b-nkbhs"] Sep 18 03:45:44.295229 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:44.295194 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75fd986f6b-nkbhs" podUID="5779c92a-ab5d-43d5-9eb8-864ae4facd0e" containerName="authorino" containerID="cri-o://1bf22cf3d7c24d6b402cdce5b9f0d160ba61b12be2dbbee5aece5aab73999d4b" gracePeriod=30 Sep 18 03:45:44.559907 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:44.559881 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75fd986f6b-nkbhs" Sep 18 03:45:44.680775 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:44.680743 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5779c92a-ab5d-43d5-9eb8-864ae4facd0e-tls-cert\") pod \"5779c92a-ab5d-43d5-9eb8-864ae4facd0e\" (UID: \"5779c92a-ab5d-43d5-9eb8-864ae4facd0e\") " Sep 18 03:45:44.680971 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:44.680793 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6b5sn\" (UniqueName: \"kubernetes.io/projected/5779c92a-ab5d-43d5-9eb8-864ae4facd0e-kube-api-access-6b5sn\") pod \"5779c92a-ab5d-43d5-9eb8-864ae4facd0e\" (UID: \"5779c92a-ab5d-43d5-9eb8-864ae4facd0e\") " Sep 18 03:45:44.682884 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:44.682849 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5779c92a-ab5d-43d5-9eb8-864ae4facd0e-kube-api-access-6b5sn" (OuterVolumeSpecName: "kube-api-access-6b5sn") pod "5779c92a-ab5d-43d5-9eb8-864ae4facd0e" (UID: "5779c92a-ab5d-43d5-9eb8-864ae4facd0e"). InnerVolumeSpecName "kube-api-access-6b5sn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 03:45:44.692564 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:44.692522 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5779c92a-ab5d-43d5-9eb8-864ae4facd0e-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5779c92a-ab5d-43d5-9eb8-864ae4facd0e" (UID: "5779c92a-ab5d-43d5-9eb8-864ae4facd0e"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:45:44.781946 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:44.781882 2568 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5779c92a-ab5d-43d5-9eb8-864ae4facd0e-tls-cert\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:45:44.781946 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:44.781921 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6b5sn\" (UniqueName: \"kubernetes.io/projected/5779c92a-ab5d-43d5-9eb8-864ae4facd0e-kube-api-access-6b5sn\") on node \"ip-10-0-134-80.ec2.internal\" DevicePath \"\"" Sep 18 03:45:45.249002 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:45.248904 2568 generic.go:356] "Generic (PLEG): container finished" podID="5779c92a-ab5d-43d5-9eb8-864ae4facd0e" containerID="1bf22cf3d7c24d6b402cdce5b9f0d160ba61b12be2dbbee5aece5aab73999d4b" exitCode=0 Sep 18 03:45:45.249002 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:45.248971 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75fd986f6b-nkbhs" Sep 18 03:45:45.249002 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:45.248990 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75fd986f6b-nkbhs" event={"ID":"5779c92a-ab5d-43d5-9eb8-864ae4facd0e","Type":"ContainerDied","Data":"1bf22cf3d7c24d6b402cdce5b9f0d160ba61b12be2dbbee5aece5aab73999d4b"} Sep 18 03:45:45.249553 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:45.249038 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75fd986f6b-nkbhs" event={"ID":"5779c92a-ab5d-43d5-9eb8-864ae4facd0e","Type":"ContainerDied","Data":"219fc3cfbad13daba89aadbb00df2091e9e799190469f3d929ef20d6af2296b3"} Sep 18 03:45:45.249553 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:45.249058 2568 scope.go:117] "RemoveContainer" containerID="1bf22cf3d7c24d6b402cdce5b9f0d160ba61b12be2dbbee5aece5aab73999d4b" Sep 18 03:45:45.261016 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:45.260999 2568 scope.go:117] "RemoveContainer" containerID="1bf22cf3d7c24d6b402cdce5b9f0d160ba61b12be2dbbee5aece5aab73999d4b" Sep 18 03:45:45.261323 ip-10-0-134-80 kubenswrapper[2568]: E0918 03:45:45.261299 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bf22cf3d7c24d6b402cdce5b9f0d160ba61b12be2dbbee5aece5aab73999d4b\": container with ID starting with 1bf22cf3d7c24d6b402cdce5b9f0d160ba61b12be2dbbee5aece5aab73999d4b not found: ID does not exist" containerID="1bf22cf3d7c24d6b402cdce5b9f0d160ba61b12be2dbbee5aece5aab73999d4b" Sep 18 03:45:45.261444 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:45.261326 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bf22cf3d7c24d6b402cdce5b9f0d160ba61b12be2dbbee5aece5aab73999d4b"} err="failed to get container status \"1bf22cf3d7c24d6b402cdce5b9f0d160ba61b12be2dbbee5aece5aab73999d4b\": rpc error: code = NotFound desc = could not find container \"1bf22cf3d7c24d6b402cdce5b9f0d160ba61b12be2dbbee5aece5aab73999d4b\": container with ID starting with 1bf22cf3d7c24d6b402cdce5b9f0d160ba61b12be2dbbee5aece5aab73999d4b not found: ID does not exist" Sep 18 03:45:45.278138 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:45.278097 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75fd986f6b-nkbhs"] Sep 18 03:45:45.279896 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:45.279863 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75fd986f6b-nkbhs"] Sep 18 03:45:45.885073 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:45.885043 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5779c92a-ab5d-43d5-9eb8-864ae4facd0e" path="/var/lib/kubelet/pods/5779c92a-ab5d-43d5-9eb8-864ae4facd0e/volumes" Sep 18 03:45:53.618634 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:53.618609 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-58dc94667-lmz88_5bf5f787-0abb-4a3d-8414-8d83f7a19e2f/authorino/0.log" Sep 18 03:45:58.598890 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:58.598861 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-7645b9c6f5-9mwtv_40c85b4f-853b-43b4-9dc7-d0f1196b8e09/manager/0.log" Sep 18 03:45:58.709861 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:58.709832 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-54765d59fd-tnkfn_e7958393-fde7-4dee-947c-4da8a09069ce/manager/0.log" Sep 18 03:45:59.063330 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:59.063304 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-77df98bf98-qhq74_7b65c1dd-90a7-4030-8d8e-4860bd7313bf/server/0.log" Sep 18 03:45:59.523006 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:45:59.522974 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-8f86fbdb5-6z6xq_63753bbc-18cb-4bce-a30c-4fe9bfe7ab94/maas-api/0.log" Sep 18 03:46:00.462757 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:00.462727 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-58dc94667-lmz88_5bf5f787-0abb-4a3d-8414-8d83f7a19e2f/authorino/0.log" Sep 18 03:46:00.910117 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:00.910086 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-54b6d5c989-d854p_731a1bc6-8fc0-408e-9ef8-d9bab9c10783/manager/0.log" Sep 18 03:46:01.697496 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:01.697465 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c6dc6c678-4v5hd_427558c4-95a1-41e3-a4cf-ff272e522caf/kube-auth-proxy/0.log" Sep 18 03:46:02.134499 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:02.134420 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-xk5gl_4c378e07-18c0-4165-81ef-6ab860b4aa7f/payload-pre-processing/0.log" Sep 18 03:46:02.340338 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:02.340313 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-59cb5d4fd6-68pct_a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95/router/0.log" Sep 18 03:46:02.677507 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:02.677482 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-5fc5ddd44f-gwrbs_05fab7ac-a166-4e6f-956d-ccf0582a0063/main/0.log" Sep 18 03:46:02.788700 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:02.788669 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5d57fc946d-whmdk_266193c9-df7d-4254-9343-2b51c3eab275/main/0.log" Sep 18 03:46:03.126776 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:03.126700 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddff6prl_20dc98be-e3fc-42da-8e1b-54711e229f3b/main/0.log" Sep 18 03:46:11.738096 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:11.738062 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hv4gk_34b777d9-80fc-4885-a29b-0b1f1e7d7415/global-pull-secret-syncer/0.log" Sep 18 03:46:11.837246 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:11.837216 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hs82j_d9a7ff5b-3ca7-43f4-907d-3eff238968f1/konnectivity-agent/0.log" Sep 18 03:46:11.885171 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:11.885141 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-80.ec2.internal_20ab69c906b5fa2b928622de52fe5786/haproxy/0.log" Sep 18 03:46:16.553099 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:16.553017 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-58dc94667-lmz88_5bf5f787-0abb-4a3d-8414-8d83f7a19e2f/authorino/0.log" Sep 18 03:46:16.676012 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:16.675984 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-54b6d5c989-d854p_731a1bc6-8fc0-408e-9ef8-d9bab9c10783/manager/0.log" Sep 18 03:46:18.244978 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:18.244946 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-847b8bd8d7-vcm9j_ff4a6710-b06e-4c27-b451-aed9164d0ffd/cluster-monitoring-operator/0.log" Sep 18 03:46:18.274317 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:18.274283 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-55dzd_c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87/kube-state-metrics/0.log" Sep 18 03:46:18.292529 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:18.292490 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-55dzd_c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87/kube-rbac-proxy-main/0.log" Sep 18 03:46:18.314864 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:18.314820 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-55dzd_c1cb9ddf-10a4-4e3b-b1fd-f7bddd8e4c87/kube-rbac-proxy-self/0.log" Sep 18 03:46:18.370683 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:18.370662 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5984459ff8-qv7cp_130cb992-f186-4a31-9d7d-f75af513570f/monitoring-plugin/0.log" Sep 18 03:46:18.565962 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:18.565806 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wtl48_f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b/node-exporter/0.log" Sep 18 03:46:18.590708 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:18.590675 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wtl48_f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b/kube-rbac-proxy/0.log" Sep 18 03:46:18.613979 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:18.613960 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wtl48_f0436d35-1fd8-4785-8ee0-a0a64d6c5c8b/init-textfile/0.log" Sep 18 03:46:18.730762 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:18.730741 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d/prometheus/0.log" Sep 18 03:46:18.756274 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:18.756249 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d/config-reloader/0.log" Sep 18 03:46:18.776477 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:18.776457 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d/thanos-sidecar/0.log" Sep 18 03:46:18.798256 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:18.798235 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d/kube-rbac-proxy-web/0.log" Sep 18 03:46:18.818542 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:18.818486 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d/kube-rbac-proxy/0.log" Sep 18 03:46:18.840732 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:18.840717 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d/kube-rbac-proxy-thanos/0.log" Sep 18 03:46:18.859691 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:18.859672 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d8e8f63f-c1a3-45d1-b3a5-01bbca654b6d/init-config-reloader/0.log" Sep 18 03:46:18.892396 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:18.892364 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-v72z7_088dec8d-8500-44f7-bf61-8540ac607fc2/prometheus-operator/0.log" Sep 18 03:46:18.909578 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:18.909559 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-v72z7_088dec8d-8500-44f7-bf61-8540ac607fc2/kube-rbac-proxy/0.log" Sep 18 03:46:18.933626 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:18.933602 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5c6747dc7b-pdpn7_93874dc2-35da-4afd-9016-216a9c5afdc9/prometheus-operator-admission-webhook/0.log" Sep 18 03:46:18.961084 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:18.961065 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-784749b4b9-vsm6m_3c785c79-caab-473c-9aa3-10e177f1bf0d/telemeter-client/0.log" Sep 18 03:46:18.980229 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:18.980214 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-784749b4b9-vsm6m_3c785c79-caab-473c-9aa3-10e177f1bf0d/reload/0.log" Sep 18 03:46:19.002167 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:19.002151 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-784749b4b9-vsm6m_3c785c79-caab-473c-9aa3-10e177f1bf0d/kube-rbac-proxy/0.log" Sep 18 03:46:21.605163 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:21.605078 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6cb42/perf-node-gather-daemonset-fcx2n"] Sep 18 03:46:21.605746 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:21.605681 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5779c92a-ab5d-43d5-9eb8-864ae4facd0e" containerName="authorino" Sep 18 03:46:21.605746 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:21.605702 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="5779c92a-ab5d-43d5-9eb8-864ae4facd0e" containerName="authorino" Sep 18 03:46:21.605922 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:21.605831 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="5779c92a-ab5d-43d5-9eb8-864ae4facd0e" containerName="authorino" Sep 18 03:46:21.610461 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:21.610441 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6cb42/perf-node-gather-daemonset-fcx2n" Sep 18 03:46:21.612623 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:21.612598 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6cb42\"/\"openshift-service-ca.crt\"" Sep 18 03:46:21.612623 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:21.612617 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6cb42\"/\"kube-root-ca.crt\"" Sep 18 03:46:21.613464 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:21.613449 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6cb42\"/\"default-dockercfg-hl7tk\"" Sep 18 03:46:21.617398 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:21.617373 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6cb42/perf-node-gather-daemonset-fcx2n"] Sep 18 03:46:21.713881 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:21.713850 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p47bb\" (UniqueName: \"kubernetes.io/projected/eb1e4d36-bb84-4dcb-a37f-953375e9c2fa-kube-api-access-p47bb\") pod \"perf-node-gather-daemonset-fcx2n\" (UID: \"eb1e4d36-bb84-4dcb-a37f-953375e9c2fa\") " pod="openshift-must-gather-6cb42/perf-node-gather-daemonset-fcx2n" Sep 18 03:46:21.714045 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:21.713891 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eb1e4d36-bb84-4dcb-a37f-953375e9c2fa-lib-modules\") pod \"perf-node-gather-daemonset-fcx2n\" (UID: \"eb1e4d36-bb84-4dcb-a37f-953375e9c2fa\") " pod="openshift-must-gather-6cb42/perf-node-gather-daemonset-fcx2n" Sep 18 03:46:21.714045 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:21.714003 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eb1e4d36-bb84-4dcb-a37f-953375e9c2fa-sys\") pod \"perf-node-gather-daemonset-fcx2n\" (UID: \"eb1e4d36-bb84-4dcb-a37f-953375e9c2fa\") " pod="openshift-must-gather-6cb42/perf-node-gather-daemonset-fcx2n" Sep 18 03:46:21.714045 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:21.714034 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/eb1e4d36-bb84-4dcb-a37f-953375e9c2fa-podres\") pod \"perf-node-gather-daemonset-fcx2n\" (UID: \"eb1e4d36-bb84-4dcb-a37f-953375e9c2fa\") " pod="openshift-must-gather-6cb42/perf-node-gather-daemonset-fcx2n" Sep 18 03:46:21.714203 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:21.714054 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/eb1e4d36-bb84-4dcb-a37f-953375e9c2fa-proc\") pod \"perf-node-gather-daemonset-fcx2n\" (UID: \"eb1e4d36-bb84-4dcb-a37f-953375e9c2fa\") " pod="openshift-must-gather-6cb42/perf-node-gather-daemonset-fcx2n" Sep 18 03:46:21.815082 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:21.815047 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p47bb\" (UniqueName: \"kubernetes.io/projected/eb1e4d36-bb84-4dcb-a37f-953375e9c2fa-kube-api-access-p47bb\") pod \"perf-node-gather-daemonset-fcx2n\" (UID: \"eb1e4d36-bb84-4dcb-a37f-953375e9c2fa\") " pod="openshift-must-gather-6cb42/perf-node-gather-daemonset-fcx2n" Sep 18 03:46:21.815082 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:21.815084 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eb1e4d36-bb84-4dcb-a37f-953375e9c2fa-lib-modules\") pod \"perf-node-gather-daemonset-fcx2n\" (UID: \"eb1e4d36-bb84-4dcb-a37f-953375e9c2fa\") " pod="openshift-must-gather-6cb42/perf-node-gather-daemonset-fcx2n" Sep 18 03:46:21.815317 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:21.815131 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eb1e4d36-bb84-4dcb-a37f-953375e9c2fa-sys\") pod \"perf-node-gather-daemonset-fcx2n\" (UID: \"eb1e4d36-bb84-4dcb-a37f-953375e9c2fa\") " pod="openshift-must-gather-6cb42/perf-node-gather-daemonset-fcx2n" Sep 18 03:46:21.815317 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:21.815151 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/eb1e4d36-bb84-4dcb-a37f-953375e9c2fa-podres\") pod \"perf-node-gather-daemonset-fcx2n\" (UID: \"eb1e4d36-bb84-4dcb-a37f-953375e9c2fa\") " pod="openshift-must-gather-6cb42/perf-node-gather-daemonset-fcx2n" Sep 18 03:46:21.815317 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:21.815182 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/eb1e4d36-bb84-4dcb-a37f-953375e9c2fa-proc\") pod \"perf-node-gather-daemonset-fcx2n\" (UID: \"eb1e4d36-bb84-4dcb-a37f-953375e9c2fa\") " pod="openshift-must-gather-6cb42/perf-node-gather-daemonset-fcx2n" Sep 18 03:46:21.815317 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:21.815255 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/eb1e4d36-bb84-4dcb-a37f-953375e9c2fa-proc\") pod \"perf-node-gather-daemonset-fcx2n\" (UID: \"eb1e4d36-bb84-4dcb-a37f-953375e9c2fa\") " pod="openshift-must-gather-6cb42/perf-node-gather-daemonset-fcx2n" Sep 18 03:46:21.815317 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:21.815259 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eb1e4d36-bb84-4dcb-a37f-953375e9c2fa-sys\") pod \"perf-node-gather-daemonset-fcx2n\" (UID: \"eb1e4d36-bb84-4dcb-a37f-953375e9c2fa\") " pod="openshift-must-gather-6cb42/perf-node-gather-daemonset-fcx2n" Sep 18 03:46:21.815495 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:21.815322 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eb1e4d36-bb84-4dcb-a37f-953375e9c2fa-lib-modules\") pod \"perf-node-gather-daemonset-fcx2n\" (UID: \"eb1e4d36-bb84-4dcb-a37f-953375e9c2fa\") " pod="openshift-must-gather-6cb42/perf-node-gather-daemonset-fcx2n" Sep 18 03:46:21.815495 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:21.815319 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/eb1e4d36-bb84-4dcb-a37f-953375e9c2fa-podres\") pod \"perf-node-gather-daemonset-fcx2n\" (UID: \"eb1e4d36-bb84-4dcb-a37f-953375e9c2fa\") " pod="openshift-must-gather-6cb42/perf-node-gather-daemonset-fcx2n" Sep 18 03:46:21.822501 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:21.822478 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p47bb\" (UniqueName: \"kubernetes.io/projected/eb1e4d36-bb84-4dcb-a37f-953375e9c2fa-kube-api-access-p47bb\") pod \"perf-node-gather-daemonset-fcx2n\" (UID: \"eb1e4d36-bb84-4dcb-a37f-953375e9c2fa\") " pod="openshift-must-gather-6cb42/perf-node-gather-daemonset-fcx2n" Sep 18 03:46:21.921276 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:21.921248 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6cb42/perf-node-gather-daemonset-fcx2n" Sep 18 03:46:22.187300 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:22.187270 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6cb42/perf-node-gather-daemonset-fcx2n"] Sep 18 03:46:22.189308 ip-10-0-134-80 kubenswrapper[2568]: W0918 03:46:22.189279 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podeb1e4d36_bb84_4dcb_a37f_953375e9c2fa.slice/crio-f41a215f77992cdf99d63d9529c84ed9a6f21c6fc65d57c9f4f243b127a858f2 WatchSource:0}: Error finding container f41a215f77992cdf99d63d9529c84ed9a6f21c6fc65d57c9f4f243b127a858f2: Status 404 returned error can't find the container with id f41a215f77992cdf99d63d9529c84ed9a6f21c6fc65d57c9f4f243b127a858f2 Sep 18 03:46:22.343628 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:22.343599 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5c4db8bdf9-qq8kb_c80472e0-3955-48d0-8029-34f8b355f4f3/volume-data-source-validator/0.log" Sep 18 03:46:22.430543 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:22.430509 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6cb42/perf-node-gather-daemonset-fcx2n" event={"ID":"eb1e4d36-bb84-4dcb-a37f-953375e9c2fa","Type":"ContainerStarted","Data":"8cb55f7aa90d37f3483289474bca7dc2942e4f34eb9f1f13cfa336ad2a8cc7e0"} Sep 18 03:46:22.430697 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:22.430565 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6cb42/perf-node-gather-daemonset-fcx2n" event={"ID":"eb1e4d36-bb84-4dcb-a37f-953375e9c2fa","Type":"ContainerStarted","Data":"f41a215f77992cdf99d63d9529c84ed9a6f21c6fc65d57c9f4f243b127a858f2"} Sep 18 03:46:22.430697 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:22.430645 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6cb42/perf-node-gather-daemonset-fcx2n" Sep 18 03:46:22.445435 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:22.445334 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6cb42/perf-node-gather-daemonset-fcx2n" podStartSLOduration=1.445319251 podStartE2EDuration="1.445319251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 03:46:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:46:22.444151921 +0000 UTC m=+796.726913785" watchObservedRunningTime="2026-09-18 03:46:22.445319251 +0000 UTC m=+796.728081115" Sep 18 03:46:23.390013 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:23.389983 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dwxzz_d5d06eb4-28cf-4325-8e91-19c6ee8b5b75/dns/0.log" Sep 18 03:46:23.409459 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:23.409437 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dwxzz_d5d06eb4-28cf-4325-8e91-19c6ee8b5b75/kube-rbac-proxy/0.log" Sep 18 03:46:23.433731 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:23.433709 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-87l8w_fcf69b36-5d73-458d-9324-6da53a9f1c73/dns-node-resolver/0.log" Sep 18 03:46:23.966010 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:23.965972 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6d95fdc8fb-hgkfh_212ab307-3717-41ab-93e2-1a92ae45fecf/registry/0.log" Sep 18 03:46:23.985959 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:23.985923 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-54ct2_d5694211-42dd-4f1c-984b-fc2a748e8759/node-ca/0.log" Sep 18 03:46:24.989804 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:24.989781 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c6dc6c678-4v5hd_427558c4-95a1-41e3-a4cf-ff272e522caf/kube-auth-proxy/0.log" Sep 18 03:46:25.089199 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:25.089168 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-xk5gl_4c378e07-18c0-4165-81ef-6ab860b4aa7f/payload-pre-processing/0.log" Sep 18 03:46:25.132773 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:25.132753 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-59cb5d4fd6-68pct_a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95/router/0.log" Sep 18 03:46:25.762544 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:25.762517 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5ccx6_2ad38ce3-1483-4747-a9b3-7ccbfbb71957/serve-healthcheck-canary/0.log" Sep 18 03:46:26.213260 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:26.213236 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-p6bc5_afa0807e-f31c-4d12-ab3a-35e9be7066f5/insights-operator/1.log" Sep 18 03:46:26.213616 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:26.213567 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-p6bc5_afa0807e-f31c-4d12-ab3a-35e9be7066f5/insights-operator/0.log" Sep 18 03:46:26.303157 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:26.303129 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m5g7q_4b99b8fd-1388-4e40-b69a-fcc3d901a224/kube-rbac-proxy/0.log" Sep 18 03:46:26.321311 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:26.321293 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m5g7q_4b99b8fd-1388-4e40-b69a-fcc3d901a224/exporter/0.log" Sep 18 03:46:26.339894 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:26.339872 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m5g7q_4b99b8fd-1388-4e40-b69a-fcc3d901a224/extractor/0.log" Sep 18 03:46:28.445500 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:28.445473 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6cb42/perf-node-gather-daemonset-fcx2n" Sep 18 03:46:28.591082 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:28.591056 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-7b6444b484-ngxtf_ccfa48da-d34b-4810-804d-ea7b8fa219c8/manager/0.log" Sep 18 03:46:28.609516 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:28.609487 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-7b6444b484-ngxtf_ccfa48da-d34b-4810-804d-ea7b8fa219c8/copy-manifests/0.log" Sep 18 03:46:30.318487 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:30.318460 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c6dc6c678-4v5hd_427558c4-95a1-41e3-a4cf-ff272e522caf/kube-auth-proxy/0.log" Sep 18 03:46:30.323948 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:30.323916 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-7645b9c6f5-9mwtv_40c85b4f-853b-43b4-9dc7-d0f1196b8e09/manager/0.log" Sep 18 03:46:30.350552 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:30.350528 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-54765d59fd-tnkfn_e7958393-fde7-4dee-947c-4da8a09069ce/manager/0.log" Sep 18 03:46:30.373556 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:30.373536 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-xk5gl_4c378e07-18c0-4165-81ef-6ab860b4aa7f/payload-pre-processing/0.log" Sep 18 03:46:30.403609 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:30.403577 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-59cb5d4fd6-68pct_a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95/router/0.log" Sep 18 03:46:30.458316 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:30.458294 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-77df98bf98-qhq74_7b65c1dd-90a7-4030-8d8e-4860bd7313bf/server/0.log" Sep 18 03:46:31.711416 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:31.711388 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c6dc6c678-4v5hd_427558c4-95a1-41e3-a4cf-ff272e522caf/kube-auth-proxy/0.log" Sep 18 03:46:31.777964 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:31.777864 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-xk5gl_4c378e07-18c0-4165-81ef-6ab860b4aa7f/payload-pre-processing/0.log" Sep 18 03:46:31.804916 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:31.804885 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-59cb5d4fd6-68pct_a827da8d-aca7-4b1e-8cb3-88ea5c4d3e95/router/0.log" Sep 18 03:46:35.447291 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:35.447267 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-5qbkt_44c39f94-93e8-46a5-aa1c-0d16cc984015/migrator/0.log" Sep 18 03:46:35.467171 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:35.467148 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-5qbkt_44c39f94-93e8-46a5-aa1c-0d16cc984015/graceful-termination/0.log" Sep 18 03:46:36.872073 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:36.872041 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fhzm9_afa7f816-ccc9-406c-bd67-55e4d933d144/kube-multus-additional-cni-plugins/0.log" Sep 18 03:46:36.890352 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:36.890307 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fhzm9_afa7f816-ccc9-406c-bd67-55e4d933d144/egress-router-binary-copy/0.log" Sep 18 03:46:36.915003 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:36.914977 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fhzm9_afa7f816-ccc9-406c-bd67-55e4d933d144/cni-plugins/0.log" Sep 18 03:46:36.932829 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:36.932806 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fhzm9_afa7f816-ccc9-406c-bd67-55e4d933d144/bond-cni-plugin/0.log" Sep 18 03:46:36.951012 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:36.950989 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fhzm9_afa7f816-ccc9-406c-bd67-55e4d933d144/routeoverride-cni/0.log" Sep 18 03:46:36.969470 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:36.969450 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fhzm9_afa7f816-ccc9-406c-bd67-55e4d933d144/whereabouts-cni-bincopy/0.log" Sep 18 03:46:36.988053 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:36.988036 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fhzm9_afa7f816-ccc9-406c-bd67-55e4d933d144/whereabouts-cni/0.log" Sep 18 03:46:37.264745 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:37.264696 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-tlvwc_2fbf3341-a0d6-4123-8bc2-799fb6b913ae/kube-multus/0.log" Sep 18 03:46:37.422604 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:37.422575 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tf6sq_9fdcb7aa-5f4c-4d16-a250-b272513204da/network-metrics-daemon/0.log" Sep 18 03:46:37.439848 ip-10-0-134-80 kubenswrapper[2568]: I0918 03:46:37.439825 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tf6sq_9fdcb7aa-5f4c-4d16-a250-b272513204da/kube-rbac-proxy/0.log"