Sep 13 20:10:53.175371 ip-10-0-134-99 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 13 20:10:53.175383 ip-10-0-134-99 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 13 20:10:53.175392 ip-10-0-134-99 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 13 20:10:53.175693 ip-10-0-134-99 systemd[1]: Failed to start Kubernetes Kubelet. Sep 13 20:11:02.532603 ip-10-0-134-99 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 13 20:11:02.532616 ip-10-0-134-99 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 395fb4bc6b774a4eb1ba0629144a9bf8 -- Sep 13 20:12:45.726343 ip-10-0-134-99 systemd[1]: Starting Kubernetes Kubelet... Sep 13 20:12:46.141771 ip-10-0-134-99 kubenswrapper[2557]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 13 20:12:46.141771 ip-10-0-134-99 kubenswrapper[2557]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 13 20:12:46.141771 ip-10-0-134-99 kubenswrapper[2557]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 13 20:12:46.141771 ip-10-0-134-99 kubenswrapper[2557]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 13 20:12:46.141771 ip-10-0-134-99 kubenswrapper[2557]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 13 20:12:46.144690 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.144600 2557 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 13 20:12:46.149701 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149684 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 13 20:12:46.149701 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149701 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 13 20:12:46.149766 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149705 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 13 20:12:46.149766 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149708 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 13 20:12:46.149766 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149711 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 13 20:12:46.149766 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149715 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 13 20:12:46.149766 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149718 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 13 20:12:46.149766 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149721 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 13 20:12:46.149766 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149723 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 13 20:12:46.149766 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149726 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 13 20:12:46.149766 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149728 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 13 20:12:46.149766 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149731 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 13 20:12:46.149766 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149733 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 13 20:12:46.149766 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149736 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 13 20:12:46.149766 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149739 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 13 20:12:46.149766 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149741 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 13 20:12:46.149766 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149749 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 13 20:12:46.149766 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149752 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 13 20:12:46.149766 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149754 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 13 20:12:46.149766 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149759 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 13 20:12:46.149766 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149763 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 13 20:12:46.150245 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149766 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 13 20:12:46.150245 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149770 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 13 20:12:46.150245 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149774 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 13 20:12:46.150245 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149779 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 13 20:12:46.150245 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149782 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 13 20:12:46.150245 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149785 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 13 20:12:46.150245 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149788 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 13 20:12:46.150245 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149791 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 13 20:12:46.150245 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149793 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 13 20:12:46.150245 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149796 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 13 20:12:46.150245 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149798 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 13 20:12:46.150245 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149801 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 13 20:12:46.150245 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149804 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 13 20:12:46.150245 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149806 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 13 20:12:46.150245 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149809 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 13 20:12:46.150245 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149812 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 13 20:12:46.150245 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149814 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 13 20:12:46.150245 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149825 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 13 20:12:46.150245 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149828 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 13 20:12:46.150709 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149830 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 13 20:12:46.150709 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149833 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 13 20:12:46.150709 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149840 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 13 20:12:46.150709 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149843 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 13 20:12:46.150709 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149845 2557 feature_gate.go:328] unrecognized feature gate: Example2 Sep 13 20:12:46.150709 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149849 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 13 20:12:46.150709 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149852 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 13 20:12:46.150709 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149854 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 13 20:12:46.150709 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149857 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 13 20:12:46.150709 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149859 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 13 20:12:46.150709 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149862 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 13 20:12:46.150709 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149864 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 13 20:12:46.150709 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149867 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 13 20:12:46.150709 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149869 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 13 20:12:46.150709 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149872 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 13 20:12:46.150709 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149875 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 13 20:12:46.150709 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149878 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 13 20:12:46.150709 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149880 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 13 20:12:46.150709 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149883 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 13 20:12:46.150709 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149885 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 13 20:12:46.151214 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149888 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 13 20:12:46.151214 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149890 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 13 20:12:46.151214 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149893 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 13 20:12:46.151214 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149895 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 13 20:12:46.151214 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149898 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 13 20:12:46.151214 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149900 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 13 20:12:46.151214 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149902 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 13 20:12:46.151214 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149905 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 13 20:12:46.151214 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149908 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 13 20:12:46.151214 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149910 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 13 20:12:46.151214 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149913 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 13 20:12:46.151214 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149921 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 13 20:12:46.151214 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149924 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 13 20:12:46.151214 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149927 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 13 20:12:46.151214 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149929 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 13 20:12:46.151214 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149932 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 13 20:12:46.151214 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149934 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 13 20:12:46.151214 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149937 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 13 20:12:46.151214 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149939 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 13 20:12:46.151214 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149942 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 13 20:12:46.151696 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149944 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 13 20:12:46.151696 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149947 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 13 20:12:46.151696 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149949 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 13 20:12:46.151696 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149952 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 13 20:12:46.151696 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149954 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 13 20:12:46.151696 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149957 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 13 20:12:46.151696 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149961 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 13 20:12:46.151696 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149963 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 13 20:12:46.151696 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149966 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 13 20:12:46.151696 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149969 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 13 20:12:46.151696 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149972 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 13 20:12:46.151696 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149974 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 13 20:12:46.151696 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149977 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 13 20:12:46.151696 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149979 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 13 20:12:46.151696 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149982 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 13 20:12:46.151696 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149984 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 13 20:12:46.151696 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149987 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 13 20:12:46.151696 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149989 2557 feature_gate.go:328] unrecognized feature gate: Example Sep 13 20:12:46.151696 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.149992 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 13 20:12:46.151696 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150778 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 13 20:12:46.152236 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150794 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 13 20:12:46.152236 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150798 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 13 20:12:46.152236 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150800 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 13 20:12:46.152236 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150803 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 13 20:12:46.152236 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150807 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 13 20:12:46.152236 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150810 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 13 20:12:46.152236 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150813 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 13 20:12:46.152236 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150816 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 13 20:12:46.152236 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150819 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 13 20:12:46.152236 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150821 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 13 20:12:46.152236 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150824 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 13 20:12:46.152236 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150826 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 13 20:12:46.152236 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150829 2557 feature_gate.go:328] unrecognized feature gate: Example2 Sep 13 20:12:46.152236 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150832 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 13 20:12:46.152236 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150835 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 13 20:12:46.152236 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150837 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 13 20:12:46.152236 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150840 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 13 20:12:46.152236 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150842 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 13 20:12:46.152236 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150846 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 13 20:12:46.152236 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150849 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 13 20:12:46.152722 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150852 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 13 20:12:46.152722 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150854 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 13 20:12:46.152722 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150857 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 13 20:12:46.152722 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150860 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 13 20:12:46.152722 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150862 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 13 20:12:46.152722 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150865 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 13 20:12:46.152722 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150867 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 13 20:12:46.152722 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150870 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 13 20:12:46.152722 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150872 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 13 20:12:46.152722 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150875 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 13 20:12:46.152722 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150877 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 13 20:12:46.152722 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150880 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 13 20:12:46.152722 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150882 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 13 20:12:46.152722 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150885 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 13 20:12:46.152722 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150887 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 13 20:12:46.152722 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150890 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 13 20:12:46.152722 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150893 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 13 20:12:46.152722 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150896 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 13 20:12:46.152722 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150898 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 13 20:12:46.152722 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150901 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 13 20:12:46.153230 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150903 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 13 20:12:46.153230 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150906 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 13 20:12:46.153230 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150908 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 13 20:12:46.153230 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150911 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 13 20:12:46.153230 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150913 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 13 20:12:46.153230 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150916 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 13 20:12:46.153230 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150918 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 13 20:12:46.153230 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150921 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 13 20:12:46.153230 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150923 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 13 20:12:46.153230 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150926 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 13 20:12:46.153230 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150930 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 13 20:12:46.153230 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150933 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 13 20:12:46.153230 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150937 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 13 20:12:46.153230 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150941 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 13 20:12:46.153230 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150944 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 13 20:12:46.153230 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150947 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 13 20:12:46.153230 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150949 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 13 20:12:46.153230 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150952 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 13 20:12:46.153230 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150955 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 13 20:12:46.153698 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150957 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 13 20:12:46.153698 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150960 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 13 20:12:46.153698 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150962 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 13 20:12:46.153698 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150965 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 13 20:12:46.153698 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150968 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 13 20:12:46.153698 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150970 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 13 20:12:46.153698 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150973 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 13 20:12:46.153698 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150975 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 13 20:12:46.153698 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150978 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 13 20:12:46.153698 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150981 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 13 20:12:46.153698 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150984 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 13 20:12:46.153698 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150987 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 13 20:12:46.153698 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150990 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 13 20:12:46.153698 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150993 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 13 20:12:46.153698 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150995 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 13 20:12:46.153698 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.150998 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 13 20:12:46.153698 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151000 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 13 20:12:46.153698 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151003 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 13 20:12:46.153698 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151005 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 13 20:12:46.153698 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151008 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 13 20:12:46.154197 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151012 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 13 20:12:46.154197 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151015 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 13 20:12:46.154197 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151018 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 13 20:12:46.154197 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151023 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 13 20:12:46.154197 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151027 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 13 20:12:46.154197 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151031 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 13 20:12:46.154197 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151033 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 13 20:12:46.154197 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151036 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 13 20:12:46.154197 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151039 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 13 20:12:46.154197 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151055 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 13 20:12:46.154197 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151058 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 13 20:12:46.154197 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151061 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 13 20:12:46.154197 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151064 2557 feature_gate.go:328] unrecognized feature gate: Example Sep 13 20:12:46.154197 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151066 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 13 20:12:46.154197 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151069 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 13 20:12:46.154197 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151071 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 13 20:12:46.154197 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151074 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 13 20:12:46.154197 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151077 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 13 20:12:46.154197 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151079 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 13 20:12:46.154197 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151502 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 13 20:12:46.154681 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151507 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 13 20:12:46.154681 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151510 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 13 20:12:46.154681 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151514 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 13 20:12:46.154681 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151517 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 13 20:12:46.154681 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151520 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 13 20:12:46.154681 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151522 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 13 20:12:46.154681 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151525 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 13 20:12:46.154681 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151527 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 13 20:12:46.154681 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151529 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 13 20:12:46.154681 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151532 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 13 20:12:46.154681 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151535 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 13 20:12:46.154681 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151537 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 13 20:12:46.154681 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151540 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 13 20:12:46.154681 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151542 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 13 20:12:46.154681 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151545 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 13 20:12:46.154681 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151548 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 13 20:12:46.154681 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151550 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 13 20:12:46.154681 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151553 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 13 20:12:46.154681 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151556 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 13 20:12:46.154681 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151558 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 13 20:12:46.155206 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151561 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 13 20:12:46.155206 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151565 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 13 20:12:46.155206 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151569 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 13 20:12:46.155206 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151572 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 13 20:12:46.155206 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151575 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 13 20:12:46.155206 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151578 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 13 20:12:46.155206 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151580 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 13 20:12:46.155206 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151582 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 13 20:12:46.155206 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151585 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 13 20:12:46.155206 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151588 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 13 20:12:46.155206 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151590 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 13 20:12:46.155206 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151593 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 13 20:12:46.155206 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151596 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 13 20:12:46.155206 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151598 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 13 20:12:46.155206 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151601 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 13 20:12:46.155206 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151603 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 13 20:12:46.155206 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151606 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 13 20:12:46.155206 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151609 2557 feature_gate.go:328] unrecognized feature gate: Example2 Sep 13 20:12:46.155206 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151612 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 13 20:12:46.155206 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151615 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 13 20:12:46.155694 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151617 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 13 20:12:46.155694 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151620 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 13 20:12:46.155694 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151622 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 13 20:12:46.155694 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151625 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 13 20:12:46.155694 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151627 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 13 20:12:46.155694 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151630 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 13 20:12:46.155694 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151632 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 13 20:12:46.155694 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151638 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 13 20:12:46.155694 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151641 2557 feature_gate.go:328] unrecognized feature gate: Example Sep 13 20:12:46.155694 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151643 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 13 20:12:46.155694 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151646 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 13 20:12:46.155694 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151649 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 13 20:12:46.155694 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151652 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 13 20:12:46.155694 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151656 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 13 20:12:46.155694 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151659 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 13 20:12:46.155694 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151662 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 13 20:12:46.155694 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151666 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 13 20:12:46.155694 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151669 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 13 20:12:46.155694 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151671 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 13 20:12:46.156170 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151674 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 13 20:12:46.156170 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151676 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 13 20:12:46.156170 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151679 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 13 20:12:46.156170 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151681 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 13 20:12:46.156170 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151684 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 13 20:12:46.156170 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151686 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 13 20:12:46.156170 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151688 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 13 20:12:46.156170 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151692 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 13 20:12:46.156170 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151695 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 13 20:12:46.156170 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151698 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 13 20:12:46.156170 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151702 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 13 20:12:46.156170 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151704 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 13 20:12:46.156170 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151707 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 13 20:12:46.156170 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151710 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 13 20:12:46.156170 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151713 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 13 20:12:46.156170 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151715 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 13 20:12:46.156170 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151718 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 13 20:12:46.156170 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151720 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 13 20:12:46.156170 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151723 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 13 20:12:46.156170 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151726 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 13 20:12:46.156708 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151729 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 13 20:12:46.156708 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151731 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 13 20:12:46.156708 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151734 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 13 20:12:46.156708 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151736 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 13 20:12:46.156708 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151739 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 13 20:12:46.156708 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151741 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 13 20:12:46.156708 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151744 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 13 20:12:46.156708 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151746 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 13 20:12:46.156708 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151749 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 13 20:12:46.156708 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151751 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 13 20:12:46.156708 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151754 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 13 20:12:46.156708 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151756 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 13 20:12:46.156708 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151758 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 13 20:12:46.156708 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151761 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 13 20:12:46.156708 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151763 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 13 20:12:46.156708 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151766 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 13 20:12:46.156708 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151769 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 13 20:12:46.156708 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151771 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 13 20:12:46.156708 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.151773 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 13 20:12:46.156708 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151846 2557 flags.go:64] FLAG: --address="0.0.0.0" Sep 13 20:12:46.157208 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151894 2557 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 13 20:12:46.157208 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151902 2557 flags.go:64] FLAG: --anonymous-auth="true" Sep 13 20:12:46.157208 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151907 2557 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 13 20:12:46.157208 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151911 2557 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 13 20:12:46.157208 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151915 2557 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 13 20:12:46.157208 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151920 2557 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 13 20:12:46.157208 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151924 2557 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 13 20:12:46.157208 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151927 2557 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 13 20:12:46.157208 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151930 2557 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 13 20:12:46.157208 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151934 2557 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 13 20:12:46.157208 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151937 2557 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 13 20:12:46.157208 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151943 2557 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 13 20:12:46.157208 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151947 2557 flags.go:64] FLAG: --cgroup-root="" Sep 13 20:12:46.157208 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151950 2557 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 13 20:12:46.157208 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151953 2557 flags.go:64] FLAG: --client-ca-file="" Sep 13 20:12:46.157208 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151956 2557 flags.go:64] FLAG: --cloud-provider="external" Sep 13 20:12:46.157208 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151959 2557 flags.go:64] FLAG: --cluster-dns="[]" Sep 13 20:12:46.157208 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151964 2557 flags.go:64] FLAG: --cluster-domain="" Sep 13 20:12:46.157208 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151967 2557 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 13 20:12:46.157208 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151970 2557 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 13 20:12:46.157208 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151973 2557 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 13 20:12:46.157208 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151976 2557 flags.go:64] FLAG: --container-log-max-files="5" Sep 13 20:12:46.157208 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151980 2557 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 13 20:12:46.157208 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151983 2557 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 13 20:12:46.157803 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151986 2557 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 13 20:12:46.157803 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151990 2557 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 13 20:12:46.157803 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151994 2557 flags.go:64] FLAG: --contention-profiling="false" Sep 13 20:12:46.157803 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.151997 2557 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 13 20:12:46.157803 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152000 2557 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 13 20:12:46.157803 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152003 2557 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 13 20:12:46.157803 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152006 2557 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 13 20:12:46.157803 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152010 2557 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 13 20:12:46.157803 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152013 2557 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 13 20:12:46.157803 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152016 2557 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 13 20:12:46.157803 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152019 2557 flags.go:64] FLAG: --enable-load-reader="false" Sep 13 20:12:46.157803 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152022 2557 flags.go:64] FLAG: --enable-server="true" Sep 13 20:12:46.157803 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152026 2557 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 13 20:12:46.157803 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152030 2557 flags.go:64] FLAG: --event-burst="100" Sep 13 20:12:46.157803 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152033 2557 flags.go:64] FLAG: --event-qps="50" Sep 13 20:12:46.157803 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152036 2557 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 13 20:12:46.157803 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152039 2557 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 13 20:12:46.157803 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152054 2557 flags.go:64] FLAG: --eviction-hard="" Sep 13 20:12:46.157803 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152058 2557 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 13 20:12:46.157803 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152062 2557 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 13 20:12:46.157803 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152065 2557 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 13 20:12:46.157803 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152068 2557 flags.go:64] FLAG: --eviction-soft="" Sep 13 20:12:46.157803 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152071 2557 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 13 20:12:46.157803 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152075 2557 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 13 20:12:46.157803 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152078 2557 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 13 20:12:46.158504 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152080 2557 flags.go:64] FLAG: --experimental-mounter-path="" Sep 13 20:12:46.158504 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152083 2557 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 13 20:12:46.158504 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152086 2557 flags.go:64] FLAG: --fail-swap-on="true" Sep 13 20:12:46.158504 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152089 2557 flags.go:64] FLAG: --feature-gates="" Sep 13 20:12:46.158504 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152093 2557 flags.go:64] FLAG: --file-check-frequency="20s" Sep 13 20:12:46.158504 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152096 2557 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 13 20:12:46.158504 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152099 2557 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 13 20:12:46.158504 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152102 2557 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 13 20:12:46.158504 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152106 2557 flags.go:64] FLAG: --healthz-port="10248" Sep 13 20:12:46.158504 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152109 2557 flags.go:64] FLAG: --help="false" Sep 13 20:12:46.158504 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152112 2557 flags.go:64] FLAG: --hostname-override="ip-10-0-134-99.ec2.internal" Sep 13 20:12:46.158504 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152115 2557 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 13 20:12:46.158504 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152118 2557 flags.go:64] FLAG: --http-check-frequency="20s" Sep 13 20:12:46.158504 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152121 2557 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 13 20:12:46.158504 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152124 2557 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 13 20:12:46.158504 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152127 2557 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 13 20:12:46.158504 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152129 2557 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 13 20:12:46.158504 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152132 2557 flags.go:64] FLAG: --image-service-endpoint="" Sep 13 20:12:46.158504 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152135 2557 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 13 20:12:46.158504 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152139 2557 flags.go:64] FLAG: --kube-api-burst="100" Sep 13 20:12:46.158504 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152142 2557 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 13 20:12:46.158504 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152146 2557 flags.go:64] FLAG: --kube-api-qps="50" Sep 13 20:12:46.158504 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152149 2557 flags.go:64] FLAG: --kube-reserved="" Sep 13 20:12:46.158504 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152152 2557 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 13 20:12:46.159131 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152154 2557 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 13 20:12:46.159131 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152157 2557 flags.go:64] FLAG: --kubelet-cgroups="" Sep 13 20:12:46.159131 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152162 2557 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 13 20:12:46.159131 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152164 2557 flags.go:64] FLAG: --lock-file="" Sep 13 20:12:46.159131 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152167 2557 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 13 20:12:46.159131 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152170 2557 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 13 20:12:46.159131 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152173 2557 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 13 20:12:46.159131 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152179 2557 flags.go:64] FLAG: --log-json-split-stream="false" Sep 13 20:12:46.159131 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152182 2557 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 13 20:12:46.159131 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152185 2557 flags.go:64] FLAG: --log-text-split-stream="false" Sep 13 20:12:46.159131 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152187 2557 flags.go:64] FLAG: --logging-format="text" Sep 13 20:12:46.159131 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152190 2557 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 13 20:12:46.159131 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152193 2557 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 13 20:12:46.159131 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152196 2557 flags.go:64] FLAG: --manifest-url="" Sep 13 20:12:46.159131 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152199 2557 flags.go:64] FLAG: --manifest-url-header="" Sep 13 20:12:46.159131 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152204 2557 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 13 20:12:46.159131 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152207 2557 flags.go:64] FLAG: --max-open-files="1000000" Sep 13 20:12:46.159131 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152211 2557 flags.go:64] FLAG: --max-pods="110" Sep 13 20:12:46.159131 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152214 2557 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 13 20:12:46.159131 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152217 2557 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 13 20:12:46.159131 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152220 2557 flags.go:64] FLAG: --memory-manager-policy="None" Sep 13 20:12:46.159131 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152223 2557 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 13 20:12:46.159131 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152226 2557 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 13 20:12:46.159131 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152229 2557 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 13 20:12:46.159131 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152232 2557 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 13 20:12:46.159747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152238 2557 flags.go:64] FLAG: --node-status-max-images="50" Sep 13 20:12:46.159747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152242 2557 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 13 20:12:46.159747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152246 2557 flags.go:64] FLAG: --oom-score-adj="-999" Sep 13 20:12:46.159747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152250 2557 flags.go:64] FLAG: --pod-cidr="" Sep 13 20:12:46.159747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152253 2557 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10" Sep 13 20:12:46.159747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152259 2557 flags.go:64] FLAG: --pod-manifest-path="" Sep 13 20:12:46.159747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152262 2557 flags.go:64] FLAG: --pod-max-pids="-1" Sep 13 20:12:46.159747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152265 2557 flags.go:64] FLAG: --pods-per-core="0" Sep 13 20:12:46.159747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152268 2557 flags.go:64] FLAG: --port="10250" Sep 13 20:12:46.159747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152272 2557 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 13 20:12:46.159747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152275 2557 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0509a1151146fa260" Sep 13 20:12:46.159747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152278 2557 flags.go:64] FLAG: --qos-reserved="" Sep 13 20:12:46.159747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152281 2557 flags.go:64] FLAG: --read-only-port="10255" Sep 13 20:12:46.159747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152284 2557 flags.go:64] FLAG: --register-node="true" Sep 13 20:12:46.159747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152287 2557 flags.go:64] FLAG: --register-with-taints="" Sep 13 20:12:46.159747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152290 2557 flags.go:64] FLAG: --registry-burst="10" Sep 13 20:12:46.159747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152293 2557 flags.go:64] FLAG: --registry-qps="5" Sep 13 20:12:46.159747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152296 2557 flags.go:64] FLAG: --reserved-cpus="" Sep 13 20:12:46.159747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152299 2557 flags.go:64] FLAG: --reserved-memory="" Sep 13 20:12:46.159747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152303 2557 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 13 20:12:46.159747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152306 2557 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 13 20:12:46.159747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152309 2557 flags.go:64] FLAG: --rotate-certificates="false" Sep 13 20:12:46.159747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152312 2557 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 13 20:12:46.159747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152315 2557 flags.go:64] FLAG: --runonce="false" Sep 13 20:12:46.159747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152318 2557 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 13 20:12:46.160404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152321 2557 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 13 20:12:46.160404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152327 2557 flags.go:64] FLAG: --seccomp-default="false" Sep 13 20:12:46.160404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152330 2557 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 13 20:12:46.160404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152333 2557 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 13 20:12:46.160404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152336 2557 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 13 20:12:46.160404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152339 2557 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 13 20:12:46.160404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152343 2557 flags.go:64] FLAG: --storage-driver-password="root" Sep 13 20:12:46.160404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152346 2557 flags.go:64] FLAG: --storage-driver-secure="false" Sep 13 20:12:46.160404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152349 2557 flags.go:64] FLAG: --storage-driver-table="stats" Sep 13 20:12:46.160404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152352 2557 flags.go:64] FLAG: --storage-driver-user="root" Sep 13 20:12:46.160404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152356 2557 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 13 20:12:46.160404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152360 2557 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 13 20:12:46.160404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152363 2557 flags.go:64] FLAG: --system-cgroups="" Sep 13 20:12:46.160404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152366 2557 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 13 20:12:46.160404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152372 2557 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 13 20:12:46.160404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152375 2557 flags.go:64] FLAG: --tls-cert-file="" Sep 13 20:12:46.160404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152379 2557 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 13 20:12:46.160404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152383 2557 flags.go:64] FLAG: --tls-min-version="" Sep 13 20:12:46.160404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152386 2557 flags.go:64] FLAG: --tls-private-key-file="" Sep 13 20:12:46.160404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152390 2557 flags.go:64] FLAG: --topology-manager-policy="none" Sep 13 20:12:46.160404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152392 2557 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 13 20:12:46.160404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152395 2557 flags.go:64] FLAG: --topology-manager-scope="container" Sep 13 20:12:46.160404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152398 2557 flags.go:64] FLAG: --v="2" Sep 13 20:12:46.160404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152405 2557 flags.go:64] FLAG: --version="false" Sep 13 20:12:46.160404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152409 2557 flags.go:64] FLAG: --vmodule="" Sep 13 20:12:46.161281 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152413 2557 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 13 20:12:46.161281 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.152416 2557 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 13 20:12:46.161281 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152554 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 13 20:12:46.161281 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152559 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 13 20:12:46.161281 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152562 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 13 20:12:46.161281 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152565 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 13 20:12:46.161281 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152568 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 13 20:12:46.161281 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152571 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 13 20:12:46.161281 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152573 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 13 20:12:46.161281 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152576 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 13 20:12:46.161281 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152579 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 13 20:12:46.161281 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152581 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 13 20:12:46.161281 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152584 2557 feature_gate.go:328] unrecognized feature gate: Example2 Sep 13 20:12:46.161281 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152587 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 13 20:12:46.161281 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152589 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 13 20:12:46.161281 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152592 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 13 20:12:46.161281 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152594 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 13 20:12:46.161281 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152598 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 13 20:12:46.161281 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152601 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 13 20:12:46.161281 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152603 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 13 20:12:46.162053 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152607 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 13 20:12:46.162053 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152610 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 13 20:12:46.162053 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152612 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 13 20:12:46.162053 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152617 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 13 20:12:46.162053 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152619 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 13 20:12:46.162053 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152622 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 13 20:12:46.162053 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152625 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 13 20:12:46.162053 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152627 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 13 20:12:46.162053 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152630 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 13 20:12:46.162053 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152632 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 13 20:12:46.162053 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152635 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 13 20:12:46.162053 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152639 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 13 20:12:46.162053 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152642 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 13 20:12:46.162053 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152645 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 13 20:12:46.162053 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152648 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 13 20:12:46.162053 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152650 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 13 20:12:46.162053 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152652 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 13 20:12:46.162053 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152655 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 13 20:12:46.162053 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152657 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 13 20:12:46.162534 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152660 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 13 20:12:46.162534 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152663 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 13 20:12:46.162534 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152665 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 13 20:12:46.162534 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152668 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 13 20:12:46.162534 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152670 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 13 20:12:46.162534 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152673 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 13 20:12:46.162534 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152675 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 13 20:12:46.162534 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152678 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 13 20:12:46.162534 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152680 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 13 20:12:46.162534 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152683 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 13 20:12:46.162534 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152687 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 13 20:12:46.162534 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152689 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 13 20:12:46.162534 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152692 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 13 20:12:46.162534 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152694 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 13 20:12:46.162534 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152697 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 13 20:12:46.162534 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152700 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 13 20:12:46.162534 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152704 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 13 20:12:46.162534 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152706 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 13 20:12:46.162534 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152709 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 13 20:12:46.163016 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152711 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 13 20:12:46.163016 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152714 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 13 20:12:46.163016 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152718 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 13 20:12:46.163016 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152722 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 13 20:12:46.163016 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152725 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 13 20:12:46.163016 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152728 2557 feature_gate.go:328] unrecognized feature gate: Example Sep 13 20:12:46.163016 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152731 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 13 20:12:46.163016 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152734 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 13 20:12:46.163016 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152736 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 13 20:12:46.163016 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152739 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 13 20:12:46.163016 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152742 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 13 20:12:46.163016 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152745 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 13 20:12:46.163016 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152747 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 13 20:12:46.163016 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152750 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 13 20:12:46.163016 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152752 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 13 20:12:46.163016 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152755 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 13 20:12:46.163016 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152757 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 13 20:12:46.163016 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152760 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 13 20:12:46.163016 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152763 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 13 20:12:46.163016 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152765 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 13 20:12:46.163523 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152768 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 13 20:12:46.163523 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152770 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 13 20:12:46.163523 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152773 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 13 20:12:46.163523 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152776 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 13 20:12:46.163523 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152779 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 13 20:12:46.163523 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152782 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 13 20:12:46.163523 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152784 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 13 20:12:46.163523 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152786 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 13 20:12:46.163523 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152790 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 13 20:12:46.163523 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152793 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 13 20:12:46.163523 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152796 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 13 20:12:46.163523 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152798 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 13 20:12:46.163523 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152801 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 13 20:12:46.163523 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152803 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 13 20:12:46.163523 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152806 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 13 20:12:46.163523 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152809 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 13 20:12:46.163523 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152814 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 13 20:12:46.163523 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152817 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 13 20:12:46.163523 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152821 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 13 20:12:46.164006 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152825 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 13 20:12:46.164006 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152829 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 13 20:12:46.164006 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152833 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 13 20:12:46.164006 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.152836 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 13 20:12:46.164006 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.153521 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 13 20:12:46.164006 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.163361 2557 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 13 20:12:46.164006 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.163378 2557 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 13 20:12:46.164343 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.164330 2557 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 13 20:12:46.164408 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.164343 2557 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 13 20:12:46.164408 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164406 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 13 20:12:46.164465 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164413 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 13 20:12:46.164465 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164417 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 13 20:12:46.164465 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164420 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 13 20:12:46.164465 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164423 2557 feature_gate.go:328] unrecognized feature gate: Example2 Sep 13 20:12:46.164465 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164426 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 13 20:12:46.164465 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164429 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 13 20:12:46.164465 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164432 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 13 20:12:46.164465 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164434 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 13 20:12:46.164465 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164438 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 13 20:12:46.164465 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164441 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 13 20:12:46.164465 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164443 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 13 20:12:46.164465 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164446 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 13 20:12:46.164465 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164449 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 13 20:12:46.164465 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164452 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 13 20:12:46.164465 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164455 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 13 20:12:46.164465 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164457 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 13 20:12:46.164465 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164460 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 13 20:12:46.164465 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164463 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 13 20:12:46.164465 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164466 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 13 20:12:46.164465 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164469 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 13 20:12:46.164957 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164471 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 13 20:12:46.164957 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164475 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 13 20:12:46.164957 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164477 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 13 20:12:46.164957 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164480 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 13 20:12:46.164957 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164483 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 13 20:12:46.164957 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164486 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 13 20:12:46.164957 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164489 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 13 20:12:46.164957 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164492 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 13 20:12:46.164957 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164494 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 13 20:12:46.164957 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164497 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 13 20:12:46.164957 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164499 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 13 20:12:46.164957 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164502 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 13 20:12:46.164957 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164505 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 13 20:12:46.164957 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164507 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 13 20:12:46.164957 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164510 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 13 20:12:46.164957 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164512 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 13 20:12:46.164957 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164515 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 13 20:12:46.164957 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164517 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 13 20:12:46.164957 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164520 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 13 20:12:46.164957 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164522 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 13 20:12:46.165454 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164525 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 13 20:12:46.165454 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164528 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 13 20:12:46.165454 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164530 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 13 20:12:46.165454 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164533 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 13 20:12:46.165454 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164535 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 13 20:12:46.165454 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164538 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 13 20:12:46.165454 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164540 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 13 20:12:46.165454 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164543 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 13 20:12:46.165454 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164545 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 13 20:12:46.165454 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164548 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 13 20:12:46.165454 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164551 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 13 20:12:46.165454 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164553 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 13 20:12:46.165454 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164556 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 13 20:12:46.165454 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164559 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 13 20:12:46.165454 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164562 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 13 20:12:46.165454 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164565 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 13 20:12:46.165454 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164568 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 13 20:12:46.165454 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164570 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 13 20:12:46.165454 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164573 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 13 20:12:46.165454 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164575 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 13 20:12:46.165968 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164578 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 13 20:12:46.165968 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164581 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 13 20:12:46.165968 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164583 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 13 20:12:46.165968 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164586 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 13 20:12:46.165968 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164589 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 13 20:12:46.165968 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164591 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 13 20:12:46.165968 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164594 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 13 20:12:46.165968 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164596 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 13 20:12:46.165968 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164599 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 13 20:12:46.165968 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164601 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 13 20:12:46.165968 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164604 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 13 20:12:46.165968 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164607 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 13 20:12:46.165968 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164609 2557 feature_gate.go:328] unrecognized feature gate: Example Sep 13 20:12:46.165968 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164612 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 13 20:12:46.165968 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164614 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 13 20:12:46.165968 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164617 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 13 20:12:46.165968 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164619 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 13 20:12:46.165968 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164622 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 13 20:12:46.165968 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164625 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 13 20:12:46.166451 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164629 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 13 20:12:46.166451 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164633 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 13 20:12:46.166451 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164636 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 13 20:12:46.166451 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164639 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 13 20:12:46.166451 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164642 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 13 20:12:46.166451 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164645 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 13 20:12:46.166451 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164648 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 13 20:12:46.166451 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164651 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 13 20:12:46.166451 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164653 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 13 20:12:46.166451 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164656 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 13 20:12:46.166451 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164659 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 13 20:12:46.166451 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164662 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 13 20:12:46.166451 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164664 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 13 20:12:46.166451 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164668 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 13 20:12:46.166451 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164671 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 13 20:12:46.166451 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164675 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 13 20:12:46.166451 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164678 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 13 20:12:46.166451 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164681 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 13 20:12:46.166451 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164684 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 13 20:12:46.166930 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.164690 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 13 20:12:46.166930 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164811 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 13 20:12:46.166930 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164816 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 13 20:12:46.166930 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164820 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 13 20:12:46.166930 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164823 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 13 20:12:46.166930 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164826 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 13 20:12:46.166930 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164829 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 13 20:12:46.166930 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164831 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 13 20:12:46.166930 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164834 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 13 20:12:46.166930 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164837 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 13 20:12:46.166930 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164839 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 13 20:12:46.166930 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164842 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 13 20:12:46.166930 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164844 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 13 20:12:46.166930 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164849 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 13 20:12:46.166930 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164852 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 13 20:12:46.167336 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164854 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 13 20:12:46.167336 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164857 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 13 20:12:46.167336 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164864 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 13 20:12:46.167336 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164867 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 13 20:12:46.167336 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164870 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 13 20:12:46.167336 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164873 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 13 20:12:46.167336 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164876 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 13 20:12:46.167336 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164879 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 13 20:12:46.167336 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164882 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 13 20:12:46.167336 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164884 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 13 20:12:46.167336 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164887 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 13 20:12:46.167336 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164890 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 13 20:12:46.167336 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164893 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 13 20:12:46.167336 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164895 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 13 20:12:46.167336 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164898 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 13 20:12:46.167336 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164901 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 13 20:12:46.167336 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164904 2557 feature_gate.go:328] unrecognized feature gate: Example2 Sep 13 20:12:46.167336 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164907 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 13 20:12:46.167336 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164909 2557 feature_gate.go:328] unrecognized feature gate: Example Sep 13 20:12:46.167336 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164912 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 13 20:12:46.167826 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164914 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 13 20:12:46.167826 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164917 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 13 20:12:46.167826 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164920 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 13 20:12:46.167826 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164923 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 13 20:12:46.167826 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164926 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 13 20:12:46.167826 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164928 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 13 20:12:46.167826 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164931 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 13 20:12:46.167826 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164933 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 13 20:12:46.167826 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164936 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 13 20:12:46.167826 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164938 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 13 20:12:46.167826 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164942 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 13 20:12:46.167826 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164946 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 13 20:12:46.167826 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164949 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 13 20:12:46.167826 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164952 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 13 20:12:46.167826 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164955 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 13 20:12:46.167826 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164958 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 13 20:12:46.167826 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164961 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 13 20:12:46.167826 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164964 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 13 20:12:46.167826 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164967 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 13 20:12:46.167826 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164970 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 13 20:12:46.168429 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164973 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 13 20:12:46.168429 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164976 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 13 20:12:46.168429 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164978 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 13 20:12:46.168429 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164981 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 13 20:12:46.168429 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164983 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 13 20:12:46.168429 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164986 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 13 20:12:46.168429 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164988 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 13 20:12:46.168429 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164991 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 13 20:12:46.168429 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164994 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 13 20:12:46.168429 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164996 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 13 20:12:46.168429 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.164999 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 13 20:12:46.168429 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165001 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 13 20:12:46.168429 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165004 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 13 20:12:46.168429 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165006 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 13 20:12:46.168429 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165009 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 13 20:12:46.168429 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165012 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 13 20:12:46.168429 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165014 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 13 20:12:46.168429 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165017 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 13 20:12:46.168429 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165019 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 13 20:12:46.168429 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165022 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 13 20:12:46.168931 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165024 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 13 20:12:46.168931 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165027 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 13 20:12:46.168931 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165029 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 13 20:12:46.168931 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165032 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 13 20:12:46.168931 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165035 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 13 20:12:46.168931 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165038 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 13 20:12:46.168931 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165053 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 13 20:12:46.168931 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165058 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 13 20:12:46.168931 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165061 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 13 20:12:46.168931 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165063 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 13 20:12:46.168931 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165066 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 13 20:12:46.168931 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165069 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 13 20:12:46.168931 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165072 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 13 20:12:46.168931 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165074 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 13 20:12:46.168931 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165077 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 13 20:12:46.168931 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165080 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 13 20:12:46.168931 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165082 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 13 20:12:46.168931 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165085 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 13 20:12:46.168931 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165089 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 13 20:12:46.168931 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165093 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 13 20:12:46.169431 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165096 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 13 20:12:46.169431 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165099 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 13 20:12:46.169431 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165101 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 13 20:12:46.169431 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165104 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 13 20:12:46.169431 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:46.165108 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 13 20:12:46.169431 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.165113 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 13 20:12:46.169431 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.165226 2557 server.go:962] "Client rotation is on, will bootstrap in background" Sep 13 20:12:46.169431 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.167694 2557 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 13 20:12:46.169431 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.168768 2557 server.go:1019] "Starting client certificate rotation" Sep 13 20:12:46.169431 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.168869 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 13 20:12:46.169431 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.168909 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 13 20:12:46.171692 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.171674 2557 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 13 20:12:46.172932 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.172913 2557 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 13 20:12:46.186269 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.186250 2557 log.go:25] "Validated CRI v1 runtime API" Sep 13 20:12:46.189760 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.189745 2557 log.go:25] "Validated CRI v1 image API" Sep 13 20:12:46.190630 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.190613 2557 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 13 20:12:46.194498 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.194477 2557 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8feeaae7-0324-4e5e-807d-9d2826eaced3:/dev/nvme0n1p3 c2e0e882-b57c-4a18-ab9a-a2dd7a27c835:/dev/nvme0n1p4] Sep 13 20:12:46.194568 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.194497 2557 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 13 20:12:46.200532 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.200234 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 13 20:12:46.202055 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.201689 2557 manager.go:216] Machine: {Timestamp:2026-09-13 20:12:46.19977693 +0000 UTC m=+0.369664522 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101619 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e67043fc9c02d98fb154d8fa00159 SystemUUID:ec2e6704-3fc9-c02d-98fb-154d8fa00159 BootID:395fb4bc-6b77-4a4e-b1ba-0629144a9bf8 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:cf:8d:4b:79 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:cf:8d:4b:79 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:22:ba:46:35:d4:24 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 13 20:12:46.202121 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.202060 2557 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 13 20:12:46.202199 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.202160 2557 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 13 20:12:46.205893 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.205865 2557 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 13 20:12:46.206080 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.205893 2557 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-99.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 13 20:12:46.206818 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.206808 2557 topology_manager.go:138] "Creating topology manager with none policy" Sep 13 20:12:46.206849 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.206821 2557 container_manager_linux.go:309] "Creating device plugin manager" Sep 13 20:12:46.206849 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.206829 2557 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 13 20:12:46.206849 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.206844 2557 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 13 20:12:46.208765 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.208755 2557 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 13 20:12:46.213436 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.213425 2557 state_mem.go:36] "Initialized new in-memory state store" Sep 13 20:12:46.213556 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.213546 2557 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 13 20:12:46.213588 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.213575 2557 kubelet.go:486] "Attempting to sync node with API server" Sep 13 20:12:46.213588 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.213584 2557 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 13 20:12:46.213645 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.213601 2557 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 13 20:12:46.213645 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.213619 2557 kubelet.go:398] "Adding apiserver pod source" Sep 13 20:12:46.213645 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.213629 2557 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 13 20:12:46.213992 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.213980 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 13 20:12:46.214061 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.213997 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 13 20:12:46.216508 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.216488 2557 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 13 20:12:46.219449 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.219435 2557 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 13 20:12:46.219528 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.219465 2557 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 13 20:12:46.222509 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.222495 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 13 20:12:46.222583 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.222515 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 13 20:12:46.222583 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.222522 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 13 20:12:46.222583 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.222527 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 13 20:12:46.222583 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.222536 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 13 20:12:46.222583 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.222542 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 13 20:12:46.222583 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.222556 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 13 20:12:46.222583 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.222561 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 13 20:12:46.222583 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.222568 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 13 20:12:46.222583 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.222574 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 13 20:12:46.222583 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.222583 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 13 20:12:46.222583 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.222588 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 13 20:12:46.223055 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:46.223018 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 13 20:12:46.223123 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:46.223022 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-99.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 13 20:12:46.224265 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.224253 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 13 20:12:46.224265 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.224266 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 13 20:12:46.228059 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.228032 2557 server.go:1268] "Started kubelet" Sep 13 20:12:46.228170 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.228129 2557 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 13 20:12:46.228205 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.228147 2557 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 13 20:12:46.228250 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.228236 2557 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 13 20:12:46.228878 ip-10-0-134-99 systemd[1]: Started Kubernetes Kubelet. Sep 13 20:12:46.229931 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.229908 2557 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 13 20:12:46.230021 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.229990 2557 server.go:310] "Adding debug handlers to kubelet server" Sep 13 20:12:46.232621 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:46.232601 2557 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 13 20:12:46.234685 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.234668 2557 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-99.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 13 20:12:46.235892 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:46.234749 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-99.ec2.internal.18d4fa7ce0d7052c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-99.ec2.internal,UID:ip-10-0-134-99.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-99.ec2.internal,},FirstTimestamp:2026-09-13 20:12:46.228006188 +0000 UTC m=+0.397893778,LastTimestamp:2026-09-13 20:12:46.228006188 +0000 UTC m=+0.397893778,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-99.ec2.internal,}" Sep 13 20:12:46.239258 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.239239 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 13 20:12:46.239970 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.239951 2557 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 13 20:12:46.240484 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:46.240462 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-99.ec2.internal\" not found" Sep 13 20:12:46.240801 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.240768 2557 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 13 20:12:46.240801 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.240768 2557 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 13 20:12:46.240928 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.240813 2557 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 13 20:12:46.240928 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.240891 2557 reconstruct.go:97] "Volume reconstruction finished" Sep 13 20:12:46.240928 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.240903 2557 reconciler.go:29] "Reconciler: start to sync state" Sep 13 20:12:46.241363 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.241348 2557 factory.go:55] Registering systemd factory Sep 13 20:12:46.241421 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.241384 2557 factory.go:223] Registration of the systemd container factory successfully Sep 13 20:12:46.241603 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.241586 2557 factory.go:153] Registering CRI-O factory Sep 13 20:12:46.241603 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.241600 2557 factory.go:223] Registration of the crio container factory successfully Sep 13 20:12:46.241674 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.241650 2557 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 13 20:12:46.241674 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.241671 2557 factory.go:103] Registering Raw factory Sep 13 20:12:46.241733 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.241688 2557 manager.go:1196] Started watching for new ooms in manager Sep 13 20:12:46.242242 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.242226 2557 manager.go:349] Starting recovery of all containers Sep 13 20:12:46.243271 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:46.243245 2557 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-99.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 13 20:12:46.244232 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:46.243673 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 13 20:12:46.247124 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.247099 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9dxsx" Sep 13 20:12:46.255236 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.255218 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9dxsx" Sep 13 20:12:46.256174 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.256154 2557 manager.go:354] Recovery completed Sep 13 20:12:46.260851 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.260836 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 13 20:12:46.262643 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.262629 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-99.ec2.internal" event="NodeHasSufficientMemory" Sep 13 20:12:46.262706 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.262656 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-99.ec2.internal" event="NodeHasNoDiskPressure" Sep 13 20:12:46.262706 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.262667 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-99.ec2.internal" event="NodeHasSufficientPID" Sep 13 20:12:46.263280 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.263263 2557 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 13 20:12:46.263280 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.263275 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 13 20:12:46.263404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.263278 2557 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 13 20:12:46.263404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.263316 2557 state_mem.go:36] "Initialized new in-memory state store" Sep 13 20:12:46.264877 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:46.264802 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-99.ec2.internal.18d4fa7ce2e78ded default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-99.ec2.internal,UID:ip-10-0-134-99.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-99.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-99.ec2.internal,},FirstTimestamp:2026-09-13 20:12:46.262644205 +0000 UTC m=+0.432531796,LastTimestamp:2026-09-13 20:12:46.262644205 +0000 UTC m=+0.432531796,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-99.ec2.internal,}" Sep 13 20:12:46.265072 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.265054 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-99.ec2.internal" event="NodeHasSufficientMemory" Sep 13 20:12:46.265150 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.265089 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-99.ec2.internal" event="NodeHasNoDiskPressure" Sep 13 20:12:46.265150 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.265103 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-99.ec2.internal" event="NodeHasSufficientPID" Sep 13 20:12:46.265369 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.265355 2557 policy_none.go:49] "None policy: Start" Sep 13 20:12:46.265434 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.265372 2557 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 13 20:12:46.265434 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.265385 2557 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 13 20:12:46.267439 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.267425 2557 policy_none.go:47] "Start" Sep 13 20:12:46.296798 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.296781 2557 manager.go:337] "Starting Device Plugin manager" Sep 13 20:12:46.321305 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:46.296834 2557 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 13 20:12:46.321305 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.296848 2557 server.go:87] "Starting device plugin registration server" Sep 13 20:12:46.321305 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.297198 2557 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 13 20:12:46.321305 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.297209 2557 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 13 20:12:46.321305 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.297419 2557 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 13 20:12:46.321305 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.297477 2557 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 13 20:12:46.321305 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.297485 2557 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 13 20:12:46.321305 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:46.298340 2557 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 13 20:12:46.321305 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:46.298400 2557 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-99.ec2.internal\" not found" Sep 13 20:12:46.321305 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.310762 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 13 20:12:46.321305 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.311871 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 13 20:12:46.321305 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.311928 2557 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 13 20:12:46.321305 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.311956 2557 kubelet.go:2448] "Starting kubelet main sync loop" Sep 13 20:12:46.321305 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:46.311999 2557 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 13 20:12:46.321305 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.313603 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 13 20:12:46.398123 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.398012 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 13 20:12:46.399032 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.399010 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-99.ec2.internal" event="NodeHasSufficientMemory" Sep 13 20:12:46.399152 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.399061 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-99.ec2.internal" event="NodeHasNoDiskPressure" Sep 13 20:12:46.399152 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.399075 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-99.ec2.internal" event="NodeHasSufficientPID" Sep 13 20:12:46.399152 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.399110 2557 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-99.ec2.internal" Sep 13 20:12:46.407480 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.407461 2557 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-99.ec2.internal" Sep 13 20:12:46.407560 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:46.407486 2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-99.ec2.internal\": node \"ip-10-0-134-99.ec2.internal\" not found" Sep 13 20:12:46.413035 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.413017 2557 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-99.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-99.ec2.internal"] Sep 13 20:12:46.413129 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.413091 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 13 20:12:46.414533 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.414516 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-99.ec2.internal" event="NodeHasSufficientMemory" Sep 13 20:12:46.414617 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.414549 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-99.ec2.internal" event="NodeHasNoDiskPressure" Sep 13 20:12:46.414617 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.414563 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-99.ec2.internal" event="NodeHasSufficientPID" Sep 13 20:12:46.416867 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.416852 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 13 20:12:46.416985 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.416959 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-99.ec2.internal" Sep 13 20:12:46.417028 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.416997 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 13 20:12:46.417538 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.417523 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-99.ec2.internal" event="NodeHasSufficientMemory" Sep 13 20:12:46.417538 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.417535 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-99.ec2.internal" event="NodeHasSufficientMemory" Sep 13 20:12:46.417662 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.417546 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-99.ec2.internal" event="NodeHasNoDiskPressure" Sep 13 20:12:46.417662 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.417554 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-99.ec2.internal" event="NodeHasNoDiskPressure" Sep 13 20:12:46.417662 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.417563 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-99.ec2.internal" event="NodeHasSufficientPID" Sep 13 20:12:46.417662 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.417566 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-99.ec2.internal" event="NodeHasSufficientPID" Sep 13 20:12:46.419867 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.419851 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-99.ec2.internal" Sep 13 20:12:46.419956 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.419880 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 13 20:12:46.420582 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.420565 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-99.ec2.internal" event="NodeHasSufficientMemory" Sep 13 20:12:46.420658 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.420596 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-99.ec2.internal" event="NodeHasNoDiskPressure" Sep 13 20:12:46.420658 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.420610 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-99.ec2.internal" event="NodeHasSufficientPID" Sep 13 20:12:46.434963 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:46.434942 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-99.ec2.internal\" not found" Sep 13 20:12:46.442136 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.442119 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7e971f67b78934963c4de7819bc2ede9-config\") pod \"kube-apiserver-proxy-ip-10-0-134-99.ec2.internal\" (UID: \"7e971f67b78934963c4de7819bc2ede9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-99.ec2.internal" Sep 13 20:12:46.442195 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.442144 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0202662ce3dd5b921974fd3ab12651d7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-99.ec2.internal\" (UID: \"0202662ce3dd5b921974fd3ab12651d7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-99.ec2.internal" Sep 13 20:12:46.442195 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.442162 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0202662ce3dd5b921974fd3ab12651d7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-99.ec2.internal\" (UID: \"0202662ce3dd5b921974fd3ab12651d7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-99.ec2.internal" Sep 13 20:12:46.449899 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:46.449878 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-99.ec2.internal\" not found" node="ip-10-0-134-99.ec2.internal" Sep 13 20:12:46.454686 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:46.454671 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-99.ec2.internal\" not found" node="ip-10-0-134-99.ec2.internal" Sep 13 20:12:46.535279 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:46.535242 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-99.ec2.internal\" not found" Sep 13 20:12:46.542615 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.542593 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7e971f67b78934963c4de7819bc2ede9-config\") pod \"kube-apiserver-proxy-ip-10-0-134-99.ec2.internal\" (UID: \"7e971f67b78934963c4de7819bc2ede9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-99.ec2.internal" Sep 13 20:12:46.542682 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.542621 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0202662ce3dd5b921974fd3ab12651d7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-99.ec2.internal\" (UID: \"0202662ce3dd5b921974fd3ab12651d7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-99.ec2.internal" Sep 13 20:12:46.542682 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.542638 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0202662ce3dd5b921974fd3ab12651d7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-99.ec2.internal\" (UID: \"0202662ce3dd5b921974fd3ab12651d7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-99.ec2.internal" Sep 13 20:12:46.542682 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.542668 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0202662ce3dd5b921974fd3ab12651d7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-99.ec2.internal\" (UID: \"0202662ce3dd5b921974fd3ab12651d7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-99.ec2.internal" Sep 13 20:12:46.542784 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.542680 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7e971f67b78934963c4de7819bc2ede9-config\") pod \"kube-apiserver-proxy-ip-10-0-134-99.ec2.internal\" (UID: \"7e971f67b78934963c4de7819bc2ede9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-99.ec2.internal" Sep 13 20:12:46.542784 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.542698 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0202662ce3dd5b921974fd3ab12651d7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-99.ec2.internal\" (UID: \"0202662ce3dd5b921974fd3ab12651d7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-99.ec2.internal" Sep 13 20:12:46.635774 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:46.635742 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-99.ec2.internal\" not found" Sep 13 20:12:46.736547 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:46.736472 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-99.ec2.internal\" not found" Sep 13 20:12:46.752968 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.752945 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-99.ec2.internal" Sep 13 20:12:46.759344 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:46.759329 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-99.ec2.internal" Sep 13 20:12:46.836614 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:46.836571 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-99.ec2.internal\" not found" Sep 13 20:12:46.937101 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:46.937070 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-99.ec2.internal\" not found" Sep 13 20:12:47.037658 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:47.037576 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-99.ec2.internal\" not found" Sep 13 20:12:49.131632 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:49.131590 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-99.ec2.internal\" not found" Sep 13 20:12:49.162134 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:49.162113 2557 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 13 20:12:49.162565 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:49.162260 2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 13 20:12:49.225202 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:49.225173 2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 13 20:12:49.232702 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:49.232680 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-99.ec2.internal\" not found" Sep 13 20:12:49.240061 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:49.240035 2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 13 20:12:49.252257 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:49.252216 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-12 20:07:48 +0000 UTC" deadline="2028-05-22 02:19:09.10099859 +0000 UTC" Sep 13 20:12:49.252257 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:49.252250 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14790h6m19.848752389s" Sep 13 20:12:49.333664 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:49.333636 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-99.ec2.internal" Sep 13 20:12:49.340511 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:49.340482 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 13 20:12:49.342593 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:49.342578 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-99.ec2.internal" Sep 13 20:12:49.352531 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:49.352513 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 13 20:12:49.438549 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:49.438520 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0202662ce3dd5b921974fd3ab12651d7.slice/crio-d3169dd512809be89ab497be0a4dacb105db9fc829356545f989d7fe976c3e97 WatchSource:0}: Error finding container d3169dd512809be89ab497be0a4dacb105db9fc829356545f989d7fe976c3e97: Status 404 returned error can't find the container with id d3169dd512809be89ab497be0a4dacb105db9fc829356545f989d7fe976c3e97 Sep 13 20:12:49.438787 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:49.438763 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e971f67b78934963c4de7819bc2ede9.slice/crio-de144baf9ca350601b3b395a0b4b54e102232653b1370f28360ee13c9fc94323 WatchSource:0}: Error finding container de144baf9ca350601b3b395a0b4b54e102232653b1370f28360ee13c9fc94323: Status 404 returned error can't find the container with id de144baf9ca350601b3b395a0b4b54e102232653b1370f28360ee13c9fc94323 Sep 13 20:12:49.443422 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:49.443407 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 13 20:12:49.688990 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:49.688916 2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 13 20:12:50.174772 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.174697 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 13 20:12:50.207518 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.207450 2557 apiserver.go:52] "Watching apiserver" Sep 13 20:12:50.214658 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.214635 2557 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 13 20:12:50.215074 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.215047 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb","openshift-cluster-node-tuning-operator/tuned-7clqq","openshift-dns/node-resolver-mgvpg","openshift-image-registry/node-ca-xdjrr","openshift-multus/multus-additional-cni-plugins-vkdmh","openshift-multus/network-metrics-daemon-95zg2","openshift-network-diagnostics/network-check-target-6zzm8","openshift-network-operator/iptables-alerter-cqvzm","kube-system/global-pull-secret-syncer-6shg5","kube-system/konnectivity-agent-9zrsq","kube-system/kube-apiserver-proxy-ip-10-0-134-99.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-99.ec2.internal","openshift-multus/multus-w4brn","openshift-ovn-kubernetes/ovnkube-node-5h748"] Sep 13 20:12:50.217778 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.217760 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:12:50.217882 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:50.217835 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zzm8" podUID="ee8bb811-1f06-4ca4-9766-3406ac59ee27" Sep 13 20:12:50.219759 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.219741 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.222090 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.221954 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 13 20:12:50.222367 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.222347 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 13 20:12:50.222788 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.222610 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 13 20:12:50.222788 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.222646 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 13 20:12:50.222788 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.222670 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-p7tpt\"" Sep 13 20:12:50.222788 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.222789 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mgvpg" Sep 13 20:12:50.224372 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.224352 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xdjrr" Sep 13 20:12:50.224865 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.224843 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 13 20:12:50.225183 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.225155 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 13 20:12:50.225183 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.225170 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qgr8f\"" Sep 13 20:12:50.226413 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.226392 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 13 20:12:50.226595 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.226538 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 13 20:12:50.226713 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.226681 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4q8xq\"" Sep 13 20:12:50.227169 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.226403 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 13 20:12:50.227169 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.227025 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vkdmh" Sep 13 20:12:50.229475 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.229083 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 13 20:12:50.229475 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.229287 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9hhzd\"" Sep 13 20:12:50.235618 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.235594 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:12:50.235716 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:50.235676 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95zg2" podUID="32f68c96-97d3-4904-a675-45f93df87572" Sep 13 20:12:50.238315 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.238295 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 13 20:12:50.241206 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.241187 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 13 20:12:50.243549 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.243527 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmmd2\" (UniqueName: \"kubernetes.io/projected/2a742e26-18ce-4bcb-8735-a69419ffb81a-kube-api-access-xmmd2\") pod \"node-ca-xdjrr\" (UID: \"2a742e26-18ce-4bcb-8735-a69419ffb81a\") " pod="openshift-image-registry/node-ca-xdjrr" Sep 13 20:12:50.243632 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.243565 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-cni-binary-copy\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.243632 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.243590 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-host-var-lib-kubelet\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.243632 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.243613 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f071dc13-4600-49f7-9396-4ee02d4a2c6f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vkdmh\" (UID: \"f071dc13-4600-49f7-9396-4ee02d4a2c6f\") " pod="openshift-multus/multus-additional-cni-plugins-vkdmh" Sep 13 20:12:50.243792 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.243661 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f071dc13-4600-49f7-9396-4ee02d4a2c6f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vkdmh\" (UID: \"f071dc13-4600-49f7-9396-4ee02d4a2c6f\") " pod="openshift-multus/multus-additional-cni-plugins-vkdmh" Sep 13 20:12:50.243792 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.243687 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4tpx\" (UniqueName: \"kubernetes.io/projected/f071dc13-4600-49f7-9396-4ee02d4a2c6f-kube-api-access-h4tpx\") pod \"multus-additional-cni-plugins-vkdmh\" (UID: \"f071dc13-4600-49f7-9396-4ee02d4a2c6f\") " pod="openshift-multus/multus-additional-cni-plugins-vkdmh" Sep 13 20:12:50.243792 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.243709 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xldlf\" (UniqueName: \"kubernetes.io/projected/ee8bb811-1f06-4ca4-9766-3406ac59ee27-kube-api-access-xldlf\") pod \"network-check-target-6zzm8\" (UID: \"ee8bb811-1f06-4ca4-9766-3406ac59ee27\") " pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:12:50.243792 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.243730 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-system-cni-dir\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.243792 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.243753 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-multus-cni-dir\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.243792 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.243775 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-cnibin\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.244076 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.243797 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-host-var-lib-cni-multus\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.244076 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.243818 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-host-run-multus-certs\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.244076 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.243842 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-etc-kubernetes\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.244076 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.243859 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx8pz\" (UniqueName: \"kubernetes.io/projected/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-kube-api-access-nx8pz\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.244076 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.243881 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e80af952-0069-4b10-9b75-2bbef4b3a126-hosts-file\") pod \"node-resolver-mgvpg\" (UID: \"e80af952-0069-4b10-9b75-2bbef4b3a126\") " pod="openshift-dns/node-resolver-mgvpg" Sep 13 20:12:50.244076 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.243894 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2a742e26-18ce-4bcb-8735-a69419ffb81a-host\") pod \"node-ca-xdjrr\" (UID: \"2a742e26-18ce-4bcb-8735-a69419ffb81a\") " pod="openshift-image-registry/node-ca-xdjrr" Sep 13 20:12:50.244076 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.243909 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f071dc13-4600-49f7-9396-4ee02d4a2c6f-system-cni-dir\") pod \"multus-additional-cni-plugins-vkdmh\" (UID: \"f071dc13-4600-49f7-9396-4ee02d4a2c6f\") " pod="openshift-multus/multus-additional-cni-plugins-vkdmh" Sep 13 20:12:50.244076 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.243925 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f071dc13-4600-49f7-9396-4ee02d4a2c6f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vkdmh\" (UID: \"f071dc13-4600-49f7-9396-4ee02d4a2c6f\") " pod="openshift-multus/multus-additional-cni-plugins-vkdmh" Sep 13 20:12:50.244076 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.243946 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2a742e26-18ce-4bcb-8735-a69419ffb81a-serviceca\") pod \"node-ca-xdjrr\" (UID: \"2a742e26-18ce-4bcb-8735-a69419ffb81a\") " pod="openshift-image-registry/node-ca-xdjrr" Sep 13 20:12:50.244076 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.243969 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-multus-socket-dir-parent\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.244076 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.243992 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-host-run-k8s-cni-cncf-io\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.244076 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.244016 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/855047d0-780f-4f76-a546-e77902ec5399-kubelet-config\") pod \"global-pull-secret-syncer-6shg5\" (UID: \"855047d0-780f-4f76-a546-e77902ec5399\") " pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:12:50.244076 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.244030 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/855047d0-780f-4f76-a546-e77902ec5399-dbus\") pod \"global-pull-secret-syncer-6shg5\" (UID: \"855047d0-780f-4f76-a546-e77902ec5399\") " pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:12:50.244076 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.244055 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/855047d0-780f-4f76-a546-e77902ec5399-original-pull-secret\") pod \"global-pull-secret-syncer-6shg5\" (UID: \"855047d0-780f-4f76-a546-e77902ec5399\") " pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:12:50.244076 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.244076 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxt47\" (UniqueName: \"kubernetes.io/projected/32f68c96-97d3-4904-a675-45f93df87572-kube-api-access-zxt47\") pod \"network-metrics-daemon-95zg2\" (UID: \"32f68c96-97d3-4904-a675-45f93df87572\") " pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:12:50.244781 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.244094 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-host-run-netns\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.244781 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.244112 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-host-var-lib-cni-bin\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.244781 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.244126 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f071dc13-4600-49f7-9396-4ee02d4a2c6f-cnibin\") pod \"multus-additional-cni-plugins-vkdmh\" (UID: \"f071dc13-4600-49f7-9396-4ee02d4a2c6f\") " pod="openshift-multus/multus-additional-cni-plugins-vkdmh" Sep 13 20:12:50.244781 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.244140 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-hostroot\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.244781 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.244162 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f071dc13-4600-49f7-9396-4ee02d4a2c6f-cni-binary-copy\") pod \"multus-additional-cni-plugins-vkdmh\" (UID: \"f071dc13-4600-49f7-9396-4ee02d4a2c6f\") " pod="openshift-multus/multus-additional-cni-plugins-vkdmh" Sep 13 20:12:50.244781 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.244185 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf4sv\" (UniqueName: \"kubernetes.io/projected/e80af952-0069-4b10-9b75-2bbef4b3a126-kube-api-access-jf4sv\") pod \"node-resolver-mgvpg\" (UID: \"e80af952-0069-4b10-9b75-2bbef4b3a126\") " pod="openshift-dns/node-resolver-mgvpg" Sep 13 20:12:50.244781 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.244199 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs\") pod \"network-metrics-daemon-95zg2\" (UID: \"32f68c96-97d3-4904-a675-45f93df87572\") " pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:12:50.244781 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.244213 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-multus-conf-dir\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.244781 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.244227 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-multus-daemon-config\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.244781 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.244263 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e80af952-0069-4b10-9b75-2bbef4b3a126-tmp-dir\") pod \"node-resolver-mgvpg\" (UID: \"e80af952-0069-4b10-9b75-2bbef4b3a126\") " pod="openshift-dns/node-resolver-mgvpg" Sep 13 20:12:50.244781 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.244588 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:12:50.244781 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:50.244652 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6shg5" podUID="855047d0-780f-4f76-a546-e77902ec5399" Sep 13 20:12:50.246036 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.246016 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" Sep 13 20:12:50.248115 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.248096 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 13 20:12:50.248464 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.248448 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.249183 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.249161 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 13 20:12:50.249409 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.249376 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 13 20:12:50.249477 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.249428 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hvcxw\"" Sep 13 20:12:50.251157 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.250641 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 13 20:12:50.251157 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.250799 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rlbhp\"" Sep 13 20:12:50.251157 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.250896 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9zrsq" Sep 13 20:12:50.251157 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.250980 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 13 20:12:50.253544 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.253526 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 13 20:12:50.254007 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.253990 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.255319 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.255304 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 13 20:12:50.255728 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.255708 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cqvzm" Sep 13 20:12:50.256410 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.256387 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 13 20:12:50.256486 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.256412 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 13 20:12:50.257010 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.256992 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 13 20:12:50.257156 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.257139 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 13 20:12:50.257210 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.257191 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 13 20:12:50.257416 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.257404 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 13 20:12:50.257606 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.257573 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-twtks\"" Sep 13 20:12:50.257606 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.257588 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6zk24\"" Sep 13 20:12:50.257833 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.257803 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 13 20:12:50.257833 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.257815 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 13 20:12:50.258688 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.258671 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 13 20:12:50.259274 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.259257 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5mkw8\"" Sep 13 20:12:50.259369 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.259352 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 13 20:12:50.273441 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.273419 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-jhzmm" Sep 13 20:12:50.281226 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.281203 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-jhzmm" Sep 13 20:12:50.311483 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.311435 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-99.ec2.internal" event={"ID":"7e971f67b78934963c4de7819bc2ede9","Type":"ContainerStarted","Data":"de144baf9ca350601b3b395a0b4b54e102232653b1370f28360ee13c9fc94323"} Sep 13 20:12:50.312921 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.312895 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-99.ec2.internal" event={"ID":"0202662ce3dd5b921974fd3ab12651d7","Type":"ContainerStarted","Data":"d3169dd512809be89ab497be0a4dacb105db9fc829356545f989d7fe976c3e97"} Sep 13 20:12:50.335226 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.335142 2557 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 13 20:12:50.344599 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.344556 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-host-run-netns\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.344724 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.344599 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-var-lib-openvswitch\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.344724 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.344627 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ef5aa942-a400-499d-8f53-c81653879c77-registration-dir\") pod \"aws-ebs-csi-driver-node-mnvjb\" (UID: \"ef5aa942-a400-499d-8f53-c81653879c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" Sep 13 20:12:50.344724 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.344656 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-hostroot\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.344724 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.344680 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-run\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.344724 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.344678 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-host-run-netns\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.344724 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.344702 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-host-slash\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.345008 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.344729 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf4sv\" (UniqueName: \"kubernetes.io/projected/e80af952-0069-4b10-9b75-2bbef4b3a126-kube-api-access-jf4sv\") pod \"node-resolver-mgvpg\" (UID: \"e80af952-0069-4b10-9b75-2bbef4b3a126\") " pod="openshift-dns/node-resolver-mgvpg" Sep 13 20:12:50.345008 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.344753 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e80af952-0069-4b10-9b75-2bbef4b3a126-tmp-dir\") pod \"node-resolver-mgvpg\" (UID: \"e80af952-0069-4b10-9b75-2bbef4b3a126\") " pod="openshift-dns/node-resolver-mgvpg" Sep 13 20:12:50.345008 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.344771 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-hostroot\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.345008 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.344798 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-sys\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.345008 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.344833 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-host-run-netns\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.345008 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.344862 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.345008 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.344891 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh9rh\" (UniqueName: \"kubernetes.io/projected/ef5aa942-a400-499d-8f53-c81653879c77-kube-api-access-zh9rh\") pod \"aws-ebs-csi-driver-node-mnvjb\" (UID: \"ef5aa942-a400-499d-8f53-c81653879c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" Sep 13 20:12:50.345008 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.344923 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmmd2\" (UniqueName: \"kubernetes.io/projected/2a742e26-18ce-4bcb-8735-a69419ffb81a-kube-api-access-xmmd2\") pod \"node-ca-xdjrr\" (UID: \"2a742e26-18ce-4bcb-8735-a69419ffb81a\") " pod="openshift-image-registry/node-ca-xdjrr" Sep 13 20:12:50.345008 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.344958 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-cni-binary-copy\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.345008 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.344989 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-var-lib-kubelet\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.345447 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345022 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-run-ovn\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.345447 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345047 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-host-cni-bin\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.345447 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345071 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xldlf\" (UniqueName: \"kubernetes.io/projected/ee8bb811-1f06-4ca4-9766-3406ac59ee27-kube-api-access-xldlf\") pod \"network-check-target-6zzm8\" (UID: \"ee8bb811-1f06-4ca4-9766-3406ac59ee27\") " pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:12:50.345447 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345104 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-system-cni-dir\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.345447 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345130 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-multus-cni-dir\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.345447 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345152 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-cnibin\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.345447 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345182 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-host-var-lib-cni-multus\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.345447 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345201 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-multus-cni-dir\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.345447 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345074 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e80af952-0069-4b10-9b75-2bbef4b3a126-tmp-dir\") pod \"node-resolver-mgvpg\" (UID: \"e80af952-0069-4b10-9b75-2bbef4b3a126\") " pod="openshift-dns/node-resolver-mgvpg" Sep 13 20:12:50.345447 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345249 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-host-run-multus-certs\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.345447 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345254 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-system-cni-dir\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.345447 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345204 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-host-run-multus-certs\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.345447 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345289 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-host-var-lib-cni-multus\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.345447 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345300 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-etc-kubernetes\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.345447 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345326 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-etc-systemd\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.345447 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345327 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-cnibin\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.345447 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345351 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2a742e26-18ce-4bcb-8735-a69419ffb81a-host\") pod \"node-ca-xdjrr\" (UID: \"2a742e26-18ce-4bcb-8735-a69419ffb81a\") " pod="openshift-image-registry/node-ca-xdjrr" Sep 13 20:12:50.345447 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345363 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-etc-kubernetes\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.346309 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345380 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-host\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.346309 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345413 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2a742e26-18ce-4bcb-8735-a69419ffb81a-host\") pod \"node-ca-xdjrr\" (UID: \"2a742e26-18ce-4bcb-8735-a69419ffb81a\") " pod="openshift-image-registry/node-ca-xdjrr" Sep 13 20:12:50.346309 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345449 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/819c8604-01b9-4333-a61d-9b47a1b6356d-etc-tuned\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.346309 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345475 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-host-kubelet\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.346309 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345527 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-cni-binary-copy\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.346309 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345543 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-log-socket\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.346309 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345567 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f997e865-7a2e-4898-a3ed-056bf8a4124d-ovn-node-metrics-cert\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.346309 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345610 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ef5aa942-a400-499d-8f53-c81653879c77-socket-dir\") pod \"aws-ebs-csi-driver-node-mnvjb\" (UID: \"ef5aa942-a400-499d-8f53-c81653879c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" Sep 13 20:12:50.346309 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345641 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-host-run-k8s-cni-cncf-io\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.346309 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345665 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/855047d0-780f-4f76-a546-e77902ec5399-dbus\") pod \"global-pull-secret-syncer-6shg5\" (UID: \"855047d0-780f-4f76-a546-e77902ec5399\") " pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:12:50.346309 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345690 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/855047d0-780f-4f76-a546-e77902ec5399-original-pull-secret\") pod \"global-pull-secret-syncer-6shg5\" (UID: \"855047d0-780f-4f76-a546-e77902ec5399\") " pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:12:50.346309 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345714 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-lib-modules\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.346309 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345737 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq5qx\" (UniqueName: \"kubernetes.io/projected/819c8604-01b9-4333-a61d-9b47a1b6356d-kube-api-access-xq5qx\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.346309 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345759 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-node-log\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.346309 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345765 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/855047d0-780f-4f76-a546-e77902ec5399-dbus\") pod \"global-pull-secret-syncer-6shg5\" (UID: \"855047d0-780f-4f76-a546-e77902ec5399\") " pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:12:50.346309 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345784 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxt47\" (UniqueName: \"kubernetes.io/projected/32f68c96-97d3-4904-a675-45f93df87572-kube-api-access-zxt47\") pod \"network-metrics-daemon-95zg2\" (UID: \"32f68c96-97d3-4904-a675-45f93df87572\") " pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:12:50.346309 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345812 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-host-run-k8s-cni-cncf-io\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.347099 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345809 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-host-var-lib-cni-bin\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.347099 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345852 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f071dc13-4600-49f7-9396-4ee02d4a2c6f-cnibin\") pod \"multus-additional-cni-plugins-vkdmh\" (UID: \"f071dc13-4600-49f7-9396-4ee02d4a2c6f\") " pod="openshift-multus/multus-additional-cni-plugins-vkdmh" Sep 13 20:12:50.347099 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345876 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-etc-sysctl-conf\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.347099 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345900 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f997e865-7a2e-4898-a3ed-056bf8a4124d-ovnkube-script-lib\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.347099 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:50.345908 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 13 20:12:50.347099 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345923 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq7tc\" (UniqueName: \"kubernetes.io/projected/f997e865-7a2e-4898-a3ed-056bf8a4124d-kube-api-access-vq7tc\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.347099 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345947 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ef5aa942-a400-499d-8f53-c81653879c77-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mnvjb\" (UID: \"ef5aa942-a400-499d-8f53-c81653879c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" Sep 13 20:12:50.347099 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.345972 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f071dc13-4600-49f7-9396-4ee02d4a2c6f-cni-binary-copy\") pod \"multus-additional-cni-plugins-vkdmh\" (UID: \"f071dc13-4600-49f7-9396-4ee02d4a2c6f\") " pod="openshift-multus/multus-additional-cni-plugins-vkdmh" Sep 13 20:12:50.347099 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:50.345997 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/855047d0-780f-4f76-a546-e77902ec5399-original-pull-secret podName:855047d0-780f-4f76-a546-e77902ec5399 nodeName:}" failed. No retries permitted until 2026-09-13 20:12:50.845980343 +0000 UTC m=+3.022416373 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/855047d0-780f-4f76-a546-e77902ec5399-original-pull-secret") pod "global-pull-secret-syncer-6shg5" (UID: "855047d0-780f-4f76-a546-e77902ec5399") : object "kube-system"/"original-pull-secret" not registered Sep 13 20:12:50.347099 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346027 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-etc-modprobe-d\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.347099 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346048 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/819c8604-01b9-4333-a61d-9b47a1b6356d-tmp\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.347099 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346062 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-run-systemd\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.347099 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346080 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs\") pod \"network-metrics-daemon-95zg2\" (UID: \"32f68c96-97d3-4904-a675-45f93df87572\") " pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:12:50.347099 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346109 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-multus-conf-dir\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.347099 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346130 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-multus-daemon-config\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.347099 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346166 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ca71999d-6b6c-46b6-9c32-819b8b72807c-agent-certs\") pod \"konnectivity-agent-9zrsq\" (UID: \"ca71999d-6b6c-46b6-9c32-819b8b72807c\") " pod="kube-system/konnectivity-agent-9zrsq" Sep 13 20:12:50.347821 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346176 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f071dc13-4600-49f7-9396-4ee02d4a2c6f-cnibin\") pod \"multus-additional-cni-plugins-vkdmh\" (UID: \"f071dc13-4600-49f7-9396-4ee02d4a2c6f\") " pod="openshift-multus/multus-additional-cni-plugins-vkdmh" Sep 13 20:12:50.347821 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346188 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-systemd-units\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.347821 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346210 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-run-openvswitch\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.347821 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346229 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-host-var-lib-cni-bin\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.347821 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346256 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f997e865-7a2e-4898-a3ed-056bf8a4124d-ovnkube-config\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.347821 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346283 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-etc-sysconfig\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.347821 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346306 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-etc-sysctl-d\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.347821 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:50.346362 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 20:12:50.347821 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:50.346419 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs podName:32f68c96-97d3-4904-a675-45f93df87572 nodeName:}" failed. No retries permitted until 2026-09-13 20:12:50.846402863 +0000 UTC m=+3.022838876 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs") pod "network-metrics-daemon-95zg2" (UID: "32f68c96-97d3-4904-a675-45f93df87572") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 20:12:50.347821 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346420 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-multus-conf-dir\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.347821 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346449 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-etc-openvswitch\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.347821 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346476 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/08f8b302-b359-4b7b-a213-4f91e6bba9f0-iptables-alerter-script\") pod \"iptables-alerter-cqvzm\" (UID: \"08f8b302-b359-4b7b-a213-4f91e6bba9f0\") " pod="openshift-network-operator/iptables-alerter-cqvzm" Sep 13 20:12:50.347821 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346522 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ef5aa942-a400-499d-8f53-c81653879c77-device-dir\") pod \"aws-ebs-csi-driver-node-mnvjb\" (UID: \"ef5aa942-a400-499d-8f53-c81653879c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" Sep 13 20:12:50.347821 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346545 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ef5aa942-a400-499d-8f53-c81653879c77-etc-selinux\") pod \"aws-ebs-csi-driver-node-mnvjb\" (UID: \"ef5aa942-a400-499d-8f53-c81653879c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" Sep 13 20:12:50.347821 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346584 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-host-var-lib-kubelet\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.347821 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346611 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f071dc13-4600-49f7-9396-4ee02d4a2c6f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vkdmh\" (UID: \"f071dc13-4600-49f7-9396-4ee02d4a2c6f\") " pod="openshift-multus/multus-additional-cni-plugins-vkdmh" Sep 13 20:12:50.348471 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346638 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f071dc13-4600-49f7-9396-4ee02d4a2c6f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vkdmh\" (UID: \"f071dc13-4600-49f7-9396-4ee02d4a2c6f\") " pod="openshift-multus/multus-additional-cni-plugins-vkdmh" Sep 13 20:12:50.348471 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346664 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4tpx\" (UniqueName: \"kubernetes.io/projected/f071dc13-4600-49f7-9396-4ee02d4a2c6f-kube-api-access-h4tpx\") pod \"multus-additional-cni-plugins-vkdmh\" (UID: \"f071dc13-4600-49f7-9396-4ee02d4a2c6f\") " pod="openshift-multus/multus-additional-cni-plugins-vkdmh" Sep 13 20:12:50.348471 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346663 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-host-var-lib-kubelet\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.348471 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346688 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/08f8b302-b359-4b7b-a213-4f91e6bba9f0-host-slash\") pod \"iptables-alerter-cqvzm\" (UID: \"08f8b302-b359-4b7b-a213-4f91e6bba9f0\") " pod="openshift-network-operator/iptables-alerter-cqvzm" Sep 13 20:12:50.348471 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346715 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx8pz\" (UniqueName: \"kubernetes.io/projected/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-kube-api-access-nx8pz\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.348471 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346738 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f071dc13-4600-49f7-9396-4ee02d4a2c6f-system-cni-dir\") pod \"multus-additional-cni-plugins-vkdmh\" (UID: \"f071dc13-4600-49f7-9396-4ee02d4a2c6f\") " pod="openshift-multus/multus-additional-cni-plugins-vkdmh" Sep 13 20:12:50.348471 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346761 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-host-run-ovn-kubernetes\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.348471 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346786 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e80af952-0069-4b10-9b75-2bbef4b3a126-hosts-file\") pod \"node-resolver-mgvpg\" (UID: \"e80af952-0069-4b10-9b75-2bbef4b3a126\") " pod="openshift-dns/node-resolver-mgvpg" Sep 13 20:12:50.348471 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346805 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f071dc13-4600-49f7-9396-4ee02d4a2c6f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vkdmh\" (UID: \"f071dc13-4600-49f7-9396-4ee02d4a2c6f\") " pod="openshift-multus/multus-additional-cni-plugins-vkdmh" Sep 13 20:12:50.348471 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346810 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f071dc13-4600-49f7-9396-4ee02d4a2c6f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vkdmh\" (UID: \"f071dc13-4600-49f7-9396-4ee02d4a2c6f\") " pod="openshift-multus/multus-additional-cni-plugins-vkdmh" Sep 13 20:12:50.348471 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346839 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-multus-daemon-config\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.348471 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346854 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ca71999d-6b6c-46b6-9c32-819b8b72807c-konnectivity-ca\") pod \"konnectivity-agent-9zrsq\" (UID: \"ca71999d-6b6c-46b6-9c32-819b8b72807c\") " pod="kube-system/konnectivity-agent-9zrsq" Sep 13 20:12:50.348471 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346883 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-host-cni-netd\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.348471 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346901 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f071dc13-4600-49f7-9396-4ee02d4a2c6f-system-cni-dir\") pod \"multus-additional-cni-plugins-vkdmh\" (UID: \"f071dc13-4600-49f7-9396-4ee02d4a2c6f\") " pod="openshift-multus/multus-additional-cni-plugins-vkdmh" Sep 13 20:12:50.348471 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.346977 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f997e865-7a2e-4898-a3ed-056bf8a4124d-env-overrides\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.348471 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.347024 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e80af952-0069-4b10-9b75-2bbef4b3a126-hosts-file\") pod \"node-resolver-mgvpg\" (UID: \"e80af952-0069-4b10-9b75-2bbef4b3a126\") " pod="openshift-dns/node-resolver-mgvpg" Sep 13 20:12:50.348471 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.347063 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjfdn\" (UniqueName: \"kubernetes.io/projected/08f8b302-b359-4b7b-a213-4f91e6bba9f0-kube-api-access-pjfdn\") pod \"iptables-alerter-cqvzm\" (UID: \"08f8b302-b359-4b7b-a213-4f91e6bba9f0\") " pod="openshift-network-operator/iptables-alerter-cqvzm" Sep 13 20:12:50.349272 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.347111 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2a742e26-18ce-4bcb-8735-a69419ffb81a-serviceca\") pod \"node-ca-xdjrr\" (UID: \"2a742e26-18ce-4bcb-8735-a69419ffb81a\") " pod="openshift-image-registry/node-ca-xdjrr" Sep 13 20:12:50.349272 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.347146 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-multus-socket-dir-parent\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.349272 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.347211 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-multus-socket-dir-parent\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.349272 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.347245 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/855047d0-780f-4f76-a546-e77902ec5399-kubelet-config\") pod \"global-pull-secret-syncer-6shg5\" (UID: \"855047d0-780f-4f76-a546-e77902ec5399\") " pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:12:50.349272 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.347304 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/855047d0-780f-4f76-a546-e77902ec5399-kubelet-config\") pod \"global-pull-secret-syncer-6shg5\" (UID: \"855047d0-780f-4f76-a546-e77902ec5399\") " pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:12:50.349272 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.347334 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-etc-kubernetes\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.349272 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.347364 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ef5aa942-a400-499d-8f53-c81653879c77-sys-fs\") pod \"aws-ebs-csi-driver-node-mnvjb\" (UID: \"ef5aa942-a400-499d-8f53-c81653879c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" Sep 13 20:12:50.349272 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.347456 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f071dc13-4600-49f7-9396-4ee02d4a2c6f-cni-binary-copy\") pod \"multus-additional-cni-plugins-vkdmh\" (UID: \"f071dc13-4600-49f7-9396-4ee02d4a2c6f\") " pod="openshift-multus/multus-additional-cni-plugins-vkdmh" Sep 13 20:12:50.349272 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.347456 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f071dc13-4600-49f7-9396-4ee02d4a2c6f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vkdmh\" (UID: \"f071dc13-4600-49f7-9396-4ee02d4a2c6f\") " pod="openshift-multus/multus-additional-cni-plugins-vkdmh" Sep 13 20:12:50.349272 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.347577 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2a742e26-18ce-4bcb-8735-a69419ffb81a-serviceca\") pod \"node-ca-xdjrr\" (UID: \"2a742e26-18ce-4bcb-8735-a69419ffb81a\") " pod="openshift-image-registry/node-ca-xdjrr" Sep 13 20:12:50.349272 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.347829 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f071dc13-4600-49f7-9396-4ee02d4a2c6f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vkdmh\" (UID: \"f071dc13-4600-49f7-9396-4ee02d4a2c6f\") " pod="openshift-multus/multus-additional-cni-plugins-vkdmh" Sep 13 20:12:50.350648 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:50.350627 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 13 20:12:50.350749 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:50.350656 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 13 20:12:50.350749 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:50.350670 2557 projected.go:196] Error preparing data for projected volume kube-api-access-xldlf for pod openshift-network-diagnostics/network-check-target-6zzm8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 20:12:50.350749 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:50.350731 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ee8bb811-1f06-4ca4-9766-3406ac59ee27-kube-api-access-xldlf podName:ee8bb811-1f06-4ca4-9766-3406ac59ee27 nodeName:}" failed. No retries permitted until 2026-09-13 20:12:50.850716363 +0000 UTC m=+3.027152376 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xldlf" (UniqueName: "kubernetes.io/projected/ee8bb811-1f06-4ca4-9766-3406ac59ee27-kube-api-access-xldlf") pod "network-check-target-6zzm8" (UID: "ee8bb811-1f06-4ca4-9766-3406ac59ee27") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 20:12:50.350983 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.350963 2557 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 13 20:12:50.356082 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.355805 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxt47\" (UniqueName: \"kubernetes.io/projected/32f68c96-97d3-4904-a675-45f93df87572-kube-api-access-zxt47\") pod \"network-metrics-daemon-95zg2\" (UID: \"32f68c96-97d3-4904-a675-45f93df87572\") " pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:12:50.356082 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.355850 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4tpx\" (UniqueName: \"kubernetes.io/projected/f071dc13-4600-49f7-9396-4ee02d4a2c6f-kube-api-access-h4tpx\") pod \"multus-additional-cni-plugins-vkdmh\" (UID: \"f071dc13-4600-49f7-9396-4ee02d4a2c6f\") " pod="openshift-multus/multus-additional-cni-plugins-vkdmh" Sep 13 20:12:50.356713 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.356663 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf4sv\" (UniqueName: \"kubernetes.io/projected/e80af952-0069-4b10-9b75-2bbef4b3a126-kube-api-access-jf4sv\") pod \"node-resolver-mgvpg\" (UID: \"e80af952-0069-4b10-9b75-2bbef4b3a126\") " pod="openshift-dns/node-resolver-mgvpg" Sep 13 20:12:50.356996 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.356978 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmmd2\" (UniqueName: \"kubernetes.io/projected/2a742e26-18ce-4bcb-8735-a69419ffb81a-kube-api-access-xmmd2\") pod \"node-ca-xdjrr\" (UID: \"2a742e26-18ce-4bcb-8735-a69419ffb81a\") " pod="openshift-image-registry/node-ca-xdjrr" Sep 13 20:12:50.357084 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.357064 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx8pz\" (UniqueName: \"kubernetes.io/projected/5b2436ba-54ce-4ba5-9ad8-48613ecd34aa-kube-api-access-nx8pz\") pod \"multus-w4brn\" (UID: \"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa\") " pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.448135 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448060 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-etc-kubernetes\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.448135 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448103 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ef5aa942-a400-499d-8f53-c81653879c77-sys-fs\") pod \"aws-ebs-csi-driver-node-mnvjb\" (UID: \"ef5aa942-a400-499d-8f53-c81653879c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" Sep 13 20:12:50.448135 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448132 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-var-lib-openvswitch\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.448392 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448165 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ef5aa942-a400-499d-8f53-c81653879c77-registration-dir\") pod \"aws-ebs-csi-driver-node-mnvjb\" (UID: \"ef5aa942-a400-499d-8f53-c81653879c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" Sep 13 20:12:50.448392 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448171 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-etc-kubernetes\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.448392 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448191 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-run\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.448392 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448214 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-var-lib-openvswitch\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.448392 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448238 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-host-slash\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.448392 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448241 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ef5aa942-a400-499d-8f53-c81653879c77-sys-fs\") pod \"aws-ebs-csi-driver-node-mnvjb\" (UID: \"ef5aa942-a400-499d-8f53-c81653879c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" Sep 13 20:12:50.448392 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448267 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-sys\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.448392 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448282 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ef5aa942-a400-499d-8f53-c81653879c77-registration-dir\") pod \"aws-ebs-csi-driver-node-mnvjb\" (UID: \"ef5aa942-a400-499d-8f53-c81653879c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" Sep 13 20:12:50.448392 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448293 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-host-run-netns\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.448392 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448320 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.448392 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448296 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-host-slash\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.448392 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448321 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-host-run-netns\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.448392 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448326 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-sys\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.448392 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448327 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-run\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.448392 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448365 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.448392 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448400 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh9rh\" (UniqueName: \"kubernetes.io/projected/ef5aa942-a400-499d-8f53-c81653879c77-kube-api-access-zh9rh\") pod \"aws-ebs-csi-driver-node-mnvjb\" (UID: \"ef5aa942-a400-499d-8f53-c81653879c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" Sep 13 20:12:50.449176 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448427 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-var-lib-kubelet\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.449176 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448451 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-run-ovn\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.449176 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448472 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-host-cni-bin\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.449176 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448531 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-var-lib-kubelet\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.449176 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448567 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-run-ovn\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.449176 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448581 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-etc-systemd\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.449176 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448608 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-host-cni-bin\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.449176 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448532 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-etc-systemd\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.449176 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448866 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-host\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.449176 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448890 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/819c8604-01b9-4333-a61d-9b47a1b6356d-etc-tuned\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.449176 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448907 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-host-kubelet\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.449176 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448929 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-log-socket\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.449176 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448943 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f997e865-7a2e-4898-a3ed-056bf8a4124d-ovn-node-metrics-cert\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.449176 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448957 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ef5aa942-a400-499d-8f53-c81653879c77-socket-dir\") pod \"aws-ebs-csi-driver-node-mnvjb\" (UID: \"ef5aa942-a400-499d-8f53-c81653879c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" Sep 13 20:12:50.449176 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.448981 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-lib-modules\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.449176 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449005 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq5qx\" (UniqueName: \"kubernetes.io/projected/819c8604-01b9-4333-a61d-9b47a1b6356d-kube-api-access-xq5qx\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.449176 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449030 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-node-log\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.449176 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449058 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-etc-sysctl-conf\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.450115 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449081 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f997e865-7a2e-4898-a3ed-056bf8a4124d-ovnkube-script-lib\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.450115 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449105 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq7tc\" (UniqueName: \"kubernetes.io/projected/f997e865-7a2e-4898-a3ed-056bf8a4124d-kube-api-access-vq7tc\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.450115 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449121 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ef5aa942-a400-499d-8f53-c81653879c77-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mnvjb\" (UID: \"ef5aa942-a400-499d-8f53-c81653879c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" Sep 13 20:12:50.450115 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449139 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-etc-modprobe-d\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.450115 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449161 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/819c8604-01b9-4333-a61d-9b47a1b6356d-tmp\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.450115 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449186 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-run-systemd\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.450115 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449225 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ca71999d-6b6c-46b6-9c32-819b8b72807c-agent-certs\") pod \"konnectivity-agent-9zrsq\" (UID: \"ca71999d-6b6c-46b6-9c32-819b8b72807c\") " pod="kube-system/konnectivity-agent-9zrsq" Sep 13 20:12:50.450115 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449247 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-systemd-units\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.450115 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449279 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-run-openvswitch\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.450115 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449301 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f997e865-7a2e-4898-a3ed-056bf8a4124d-ovnkube-config\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.450115 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449320 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-etc-sysconfig\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.450115 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449327 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-node-log\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.450115 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449387 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-etc-sysconfig\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.450115 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449438 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-host\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.450115 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449452 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-etc-modprobe-d\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.450115 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449482 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-etc-sysctl-d\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.450115 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449543 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-etc-openvswitch\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.450922 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449570 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/08f8b302-b359-4b7b-a213-4f91e6bba9f0-iptables-alerter-script\") pod \"iptables-alerter-cqvzm\" (UID: \"08f8b302-b359-4b7b-a213-4f91e6bba9f0\") " pod="openshift-network-operator/iptables-alerter-cqvzm" Sep 13 20:12:50.450922 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449593 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ef5aa942-a400-499d-8f53-c81653879c77-device-dir\") pod \"aws-ebs-csi-driver-node-mnvjb\" (UID: \"ef5aa942-a400-499d-8f53-c81653879c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" Sep 13 20:12:50.450922 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449618 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ef5aa942-a400-499d-8f53-c81653879c77-etc-selinux\") pod \"aws-ebs-csi-driver-node-mnvjb\" (UID: \"ef5aa942-a400-499d-8f53-c81653879c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" Sep 13 20:12:50.450922 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449647 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/08f8b302-b359-4b7b-a213-4f91e6bba9f0-host-slash\") pod \"iptables-alerter-cqvzm\" (UID: \"08f8b302-b359-4b7b-a213-4f91e6bba9f0\") " pod="openshift-network-operator/iptables-alerter-cqvzm" Sep 13 20:12:50.450922 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449680 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-host-run-ovn-kubernetes\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.450922 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449683 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-systemd-units\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.450922 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449708 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ca71999d-6b6c-46b6-9c32-819b8b72807c-konnectivity-ca\") pod \"konnectivity-agent-9zrsq\" (UID: \"ca71999d-6b6c-46b6-9c32-819b8b72807c\") " pod="kube-system/konnectivity-agent-9zrsq" Sep 13 20:12:50.450922 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449732 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-host-cni-netd\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.450922 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449733 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-run-systemd\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.450922 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449757 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f997e865-7a2e-4898-a3ed-056bf8a4124d-env-overrides\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.450922 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449782 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjfdn\" (UniqueName: \"kubernetes.io/projected/08f8b302-b359-4b7b-a213-4f91e6bba9f0-kube-api-access-pjfdn\") pod \"iptables-alerter-cqvzm\" (UID: \"08f8b302-b359-4b7b-a213-4f91e6bba9f0\") " pod="openshift-network-operator/iptables-alerter-cqvzm" Sep 13 20:12:50.450922 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449966 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f997e865-7a2e-4898-a3ed-056bf8a4124d-ovnkube-script-lib\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.450922 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.449975 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-run-openvswitch\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.450922 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.450122 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-etc-sysctl-conf\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.450922 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.450218 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ef5aa942-a400-499d-8f53-c81653879c77-etc-selinux\") pod \"aws-ebs-csi-driver-node-mnvjb\" (UID: \"ef5aa942-a400-499d-8f53-c81653879c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" Sep 13 20:12:50.450922 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.450232 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/08f8b302-b359-4b7b-a213-4f91e6bba9f0-host-slash\") pod \"iptables-alerter-cqvzm\" (UID: \"08f8b302-b359-4b7b-a213-4f91e6bba9f0\") " pod="openshift-network-operator/iptables-alerter-cqvzm" Sep 13 20:12:50.450922 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.450287 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-host-run-ovn-kubernetes\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.451743 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.450275 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ef5aa942-a400-499d-8f53-c81653879c77-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mnvjb\" (UID: \"ef5aa942-a400-499d-8f53-c81653879c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" Sep 13 20:12:50.451743 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.450313 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-host-cni-netd\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.451743 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.450346 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-etc-openvswitch\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.451743 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.450418 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-etc-sysctl-d\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.451743 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.450467 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ef5aa942-a400-499d-8f53-c81653879c77-device-dir\") pod \"aws-ebs-csi-driver-node-mnvjb\" (UID: \"ef5aa942-a400-499d-8f53-c81653879c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" Sep 13 20:12:50.451743 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.450530 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f997e865-7a2e-4898-a3ed-056bf8a4124d-ovnkube-config\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.451743 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.450589 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-host-kubelet\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.451743 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.450633 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f997e865-7a2e-4898-a3ed-056bf8a4124d-log-socket\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.451743 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.450717 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/819c8604-01b9-4333-a61d-9b47a1b6356d-lib-modules\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.451743 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.450819 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ef5aa942-a400-499d-8f53-c81653879c77-socket-dir\") pod \"aws-ebs-csi-driver-node-mnvjb\" (UID: \"ef5aa942-a400-499d-8f53-c81653879c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" Sep 13 20:12:50.451743 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.450917 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ca71999d-6b6c-46b6-9c32-819b8b72807c-konnectivity-ca\") pod \"konnectivity-agent-9zrsq\" (UID: \"ca71999d-6b6c-46b6-9c32-819b8b72807c\") " pod="kube-system/konnectivity-agent-9zrsq" Sep 13 20:12:50.451743 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.450923 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/08f8b302-b359-4b7b-a213-4f91e6bba9f0-iptables-alerter-script\") pod \"iptables-alerter-cqvzm\" (UID: \"08f8b302-b359-4b7b-a213-4f91e6bba9f0\") " pod="openshift-network-operator/iptables-alerter-cqvzm" Sep 13 20:12:50.451743 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.451254 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f997e865-7a2e-4898-a3ed-056bf8a4124d-env-overrides\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.452309 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.452196 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ca71999d-6b6c-46b6-9c32-819b8b72807c-agent-certs\") pod \"konnectivity-agent-9zrsq\" (UID: \"ca71999d-6b6c-46b6-9c32-819b8b72807c\") " pod="kube-system/konnectivity-agent-9zrsq" Sep 13 20:12:50.453269 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.453245 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f997e865-7a2e-4898-a3ed-056bf8a4124d-ovn-node-metrics-cert\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.453416 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.453348 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/819c8604-01b9-4333-a61d-9b47a1b6356d-tmp\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.453416 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.453403 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/819c8604-01b9-4333-a61d-9b47a1b6356d-etc-tuned\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.456982 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.456955 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjfdn\" (UniqueName: \"kubernetes.io/projected/08f8b302-b359-4b7b-a213-4f91e6bba9f0-kube-api-access-pjfdn\") pod \"iptables-alerter-cqvzm\" (UID: \"08f8b302-b359-4b7b-a213-4f91e6bba9f0\") " pod="openshift-network-operator/iptables-alerter-cqvzm" Sep 13 20:12:50.457450 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.457425 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh9rh\" (UniqueName: \"kubernetes.io/projected/ef5aa942-a400-499d-8f53-c81653879c77-kube-api-access-zh9rh\") pod \"aws-ebs-csi-driver-node-mnvjb\" (UID: \"ef5aa942-a400-499d-8f53-c81653879c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" Sep 13 20:12:50.458808 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.458729 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq7tc\" (UniqueName: \"kubernetes.io/projected/f997e865-7a2e-4898-a3ed-056bf8a4124d-kube-api-access-vq7tc\") pod \"ovnkube-node-5h748\" (UID: \"f997e865-7a2e-4898-a3ed-056bf8a4124d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.458808 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.458754 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq5qx\" (UniqueName: \"kubernetes.io/projected/819c8604-01b9-4333-a61d-9b47a1b6356d-kube-api-access-xq5qx\") pod \"tuned-7clqq\" (UID: \"819c8604-01b9-4333-a61d-9b47a1b6356d\") " pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.545857 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.545823 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-w4brn" Sep 13 20:12:50.547926 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.547898 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mgvpg" Sep 13 20:12:50.561219 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.561193 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xdjrr" Sep 13 20:12:50.566824 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.566799 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7clqq" Sep 13 20:12:50.573554 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.573531 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vkdmh" Sep 13 20:12:50.583263 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.583244 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" Sep 13 20:12:50.590750 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.590728 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9zrsq" Sep 13 20:12:50.597436 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.597418 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:12:50.603131 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.603112 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cqvzm" Sep 13 20:12:50.853375 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.853291 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs\") pod \"network-metrics-daemon-95zg2\" (UID: \"32f68c96-97d3-4904-a675-45f93df87572\") " pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:12:50.853375 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.853341 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xldlf\" (UniqueName: \"kubernetes.io/projected/ee8bb811-1f06-4ca4-9766-3406ac59ee27-kube-api-access-xldlf\") pod \"network-check-target-6zzm8\" (UID: \"ee8bb811-1f06-4ca4-9766-3406ac59ee27\") " pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:12:50.853599 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:50.853431 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 20:12:50.853599 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:50.853457 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/855047d0-780f-4f76-a546-e77902ec5399-original-pull-secret\") pod \"global-pull-secret-syncer-6shg5\" (UID: \"855047d0-780f-4f76-a546-e77902ec5399\") " pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:12:50.853599 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:50.853505 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs podName:32f68c96-97d3-4904-a675-45f93df87572 nodeName:}" failed. No retries permitted until 2026-09-13 20:12:51.853480133 +0000 UTC m=+4.029916146 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs") pod "network-metrics-daemon-95zg2" (UID: "32f68c96-97d3-4904-a675-45f93df87572") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 20:12:50.853599 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:50.853442 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 13 20:12:50.853599 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:50.853537 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 13 20:12:50.853599 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:50.853549 2557 projected.go:196] Error preparing data for projected volume kube-api-access-xldlf for pod openshift-network-diagnostics/network-check-target-6zzm8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 20:12:50.853599 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:50.853556 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 13 20:12:50.853599 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:50.853592 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ee8bb811-1f06-4ca4-9766-3406ac59ee27-kube-api-access-xldlf podName:ee8bb811-1f06-4ca4-9766-3406ac59ee27 nodeName:}" failed. No retries permitted until 2026-09-13 20:12:51.85358044 +0000 UTC m=+4.030016443 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xldlf" (UniqueName: "kubernetes.io/projected/ee8bb811-1f06-4ca4-9766-3406ac59ee27-kube-api-access-xldlf") pod "network-check-target-6zzm8" (UID: "ee8bb811-1f06-4ca4-9766-3406ac59ee27") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 20:12:50.854000 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:50.853609 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/855047d0-780f-4f76-a546-e77902ec5399-original-pull-secret podName:855047d0-780f-4f76-a546-e77902ec5399 nodeName:}" failed. No retries permitted until 2026-09-13 20:12:51.853600341 +0000 UTC m=+4.030036343 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/855047d0-780f-4f76-a546-e77902ec5399-original-pull-secret") pod "global-pull-secret-syncer-6shg5" (UID: "855047d0-780f-4f76-a546-e77902ec5399") : object "kube-system"/"original-pull-secret" not registered Sep 13 20:12:51.285286 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:51.285156 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-12 20:07:50 +0000 UTC" deadline="2028-03-10 16:45:34.272671853 +0000 UTC" Sep 13 20:12:51.285286 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:51.285227 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13052h32m42.98744905s" Sep 13 20:12:51.613697 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:51.613670 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef5aa942_a400_499d_8f53_c81653879c77.slice/crio-7497414c97000415229d3d2bccf9ab7363457975a92dc7be405a912e4e9ca0e1 WatchSource:0}: Error finding container 7497414c97000415229d3d2bccf9ab7363457975a92dc7be405a912e4e9ca0e1: Status 404 returned error can't find the container with id 7497414c97000415229d3d2bccf9ab7363457975a92dc7be405a912e4e9ca0e1 Sep 13 20:12:51.614664 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:51.614642 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf071dc13_4600_49f7_9396_4ee02d4a2c6f.slice/crio-13464dc41b7307b0c0e34cf14f6bac429973ad4c4f5e697ad65a07d20a76dbe9 WatchSource:0}: Error finding container 13464dc41b7307b0c0e34cf14f6bac429973ad4c4f5e697ad65a07d20a76dbe9: Status 404 returned error can't find the container with id 13464dc41b7307b0c0e34cf14f6bac429973ad4c4f5e697ad65a07d20a76dbe9 Sep 13 20:12:51.616032 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:51.615910 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08f8b302_b359_4b7b_a213_4f91e6bba9f0.slice/crio-73f9a33c9a454633d5e2b1e7fd88768aeaa955bf2d5ac2a4ddb5a83429cb8681 WatchSource:0}: Error finding container 73f9a33c9a454633d5e2b1e7fd88768aeaa955bf2d5ac2a4ddb5a83429cb8681: Status 404 returned error can't find the container with id 73f9a33c9a454633d5e2b1e7fd88768aeaa955bf2d5ac2a4ddb5a83429cb8681 Sep 13 20:12:51.617872 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:51.617848 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a742e26_18ce_4bcb_8735_a69419ffb81a.slice/crio-465a3344d6c0d6cb2d28569ae685dee1b9394ddab61533e0f13eda261c1df920 WatchSource:0}: Error finding container 465a3344d6c0d6cb2d28569ae685dee1b9394ddab61533e0f13eda261c1df920: Status 404 returned error can't find the container with id 465a3344d6c0d6cb2d28569ae685dee1b9394ddab61533e0f13eda261c1df920 Sep 13 20:12:51.620098 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:51.620068 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca71999d_6b6c_46b6_9c32_819b8b72807c.slice/crio-2d2d47db664abf538c562b8e9b48b682682a9251ea33c721da81579599c7159b WatchSource:0}: Error finding container 2d2d47db664abf538c562b8e9b48b682682a9251ea33c721da81579599c7159b: Status 404 returned error can't find the container with id 2d2d47db664abf538c562b8e9b48b682682a9251ea33c721da81579599c7159b Sep 13 20:12:51.621745 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:51.621370 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf997e865_7a2e_4898_a3ed_056bf8a4124d.slice/crio-229abe2fa1b8ce1b081dd43dfa3c2d8973b20cdec70bf045ee3779c5bd357cff WatchSource:0}: Error finding container 229abe2fa1b8ce1b081dd43dfa3c2d8973b20cdec70bf045ee3779c5bd357cff: Status 404 returned error can't find the container with id 229abe2fa1b8ce1b081dd43dfa3c2d8973b20cdec70bf045ee3779c5bd357cff Sep 13 20:12:51.622820 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:51.622694 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b2436ba_54ce_4ba5_9ad8_48613ecd34aa.slice/crio-675f57e8f1675ca6f81b86c8edb4a2a0753a025607c04624c0fbf1400a943af0 WatchSource:0}: Error finding container 675f57e8f1675ca6f81b86c8edb4a2a0753a025607c04624c0fbf1400a943af0: Status 404 returned error can't find the container with id 675f57e8f1675ca6f81b86c8edb4a2a0753a025607c04624c0fbf1400a943af0 Sep 13 20:12:51.625655 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:51.625200 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod819c8604_01b9_4333_a61d_9b47a1b6356d.slice/crio-ed631ec5ba94627cbdc18f02353ee4187a95ded2b3bad32daffc2787149aaca6 WatchSource:0}: Error finding container ed631ec5ba94627cbdc18f02353ee4187a95ded2b3bad32daffc2787149aaca6: Status 404 returned error can't find the container with id ed631ec5ba94627cbdc18f02353ee4187a95ded2b3bad32daffc2787149aaca6 Sep 13 20:12:51.626419 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:12:51.626395 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode80af952_0069_4b10_9b75_2bbef4b3a126.slice/crio-5ab80fe1a53f6f74de5e9495733d99bedacb0f455ce11e7d839516d29cef5630 WatchSource:0}: Error finding container 5ab80fe1a53f6f74de5e9495733d99bedacb0f455ce11e7d839516d29cef5630: Status 404 returned error can't find the container with id 5ab80fe1a53f6f74de5e9495733d99bedacb0f455ce11e7d839516d29cef5630 Sep 13 20:12:51.860049 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:51.860023 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xldlf\" (UniqueName: \"kubernetes.io/projected/ee8bb811-1f06-4ca4-9766-3406ac59ee27-kube-api-access-xldlf\") pod \"network-check-target-6zzm8\" (UID: \"ee8bb811-1f06-4ca4-9766-3406ac59ee27\") " pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:12:51.860172 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:51.860062 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/855047d0-780f-4f76-a546-e77902ec5399-original-pull-secret\") pod \"global-pull-secret-syncer-6shg5\" (UID: \"855047d0-780f-4f76-a546-e77902ec5399\") " pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:12:51.860172 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:51.860095 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs\") pod \"network-metrics-daemon-95zg2\" (UID: \"32f68c96-97d3-4904-a675-45f93df87572\") " pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:12:51.860288 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:51.860182 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 20:12:51.860288 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:51.860242 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs podName:32f68c96-97d3-4904-a675-45f93df87572 nodeName:}" failed. No retries permitted until 2026-09-13 20:12:53.860222469 +0000 UTC m=+6.036658468 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs") pod "network-metrics-daemon-95zg2" (UID: "32f68c96-97d3-4904-a675-45f93df87572") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 20:12:51.860288 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:51.860253 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 13 20:12:51.860409 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:51.860326 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/855047d0-780f-4f76-a546-e77902ec5399-original-pull-secret podName:855047d0-780f-4f76-a546-e77902ec5399 nodeName:}" failed. No retries permitted until 2026-09-13 20:12:53.86030659 +0000 UTC m=+6.036742592 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/855047d0-780f-4f76-a546-e77902ec5399-original-pull-secret") pod "global-pull-secret-syncer-6shg5" (UID: "855047d0-780f-4f76-a546-e77902ec5399") : object "kube-system"/"original-pull-secret" not registered Sep 13 20:12:51.860409 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:51.860185 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 13 20:12:51.860409 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:51.860365 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 13 20:12:51.860409 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:51.860378 2557 projected.go:196] Error preparing data for projected volume kube-api-access-xldlf for pod openshift-network-diagnostics/network-check-target-6zzm8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 20:12:51.860607 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:51.860422 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ee8bb811-1f06-4ca4-9766-3406ac59ee27-kube-api-access-xldlf podName:ee8bb811-1f06-4ca4-9766-3406ac59ee27 nodeName:}" failed. No retries permitted until 2026-09-13 20:12:53.860410721 +0000 UTC m=+6.036846737 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xldlf" (UniqueName: "kubernetes.io/projected/ee8bb811-1f06-4ca4-9766-3406ac59ee27-kube-api-access-xldlf") pod "network-check-target-6zzm8" (UID: "ee8bb811-1f06-4ca4-9766-3406ac59ee27") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 20:12:52.285728 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:52.285635 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-12 20:07:50 +0000 UTC" deadline="2028-04-21 08:15:54.157601514 +0000 UTC" Sep 13 20:12:52.285728 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:52.285677 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14052h3m1.871928632s" Sep 13 20:12:52.309302 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:52.308454 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:12:52.309302 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:52.308618 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95zg2" podUID="32f68c96-97d3-4904-a675-45f93df87572" Sep 13 20:12:52.309302 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:52.308996 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:12:52.309302 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:52.309089 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zzm8" podUID="ee8bb811-1f06-4ca4-9766-3406ac59ee27" Sep 13 20:12:52.309302 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:52.309153 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:12:52.309302 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:52.309225 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6shg5" podUID="855047d0-780f-4f76-a546-e77902ec5399" Sep 13 20:12:52.349520 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:52.348941 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-99.ec2.internal" event={"ID":"7e971f67b78934963c4de7819bc2ede9","Type":"ContainerStarted","Data":"6dd15752fe925780cd8ddfc1843b4b84bf2bef0176a32aca79dc93152b4e368d"} Sep 13 20:12:52.360980 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:52.360947 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mgvpg" event={"ID":"e80af952-0069-4b10-9b75-2bbef4b3a126","Type":"ContainerStarted","Data":"5ab80fe1a53f6f74de5e9495733d99bedacb0f455ce11e7d839516d29cef5630"} Sep 13 20:12:52.369752 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:52.369716 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7clqq" event={"ID":"819c8604-01b9-4333-a61d-9b47a1b6356d","Type":"ContainerStarted","Data":"ed631ec5ba94627cbdc18f02353ee4187a95ded2b3bad32daffc2787149aaca6"} Sep 13 20:12:52.379626 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:52.379590 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-w4brn" event={"ID":"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa","Type":"ContainerStarted","Data":"675f57e8f1675ca6f81b86c8edb4a2a0753a025607c04624c0fbf1400a943af0"} Sep 13 20:12:52.385000 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:52.384964 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cqvzm" event={"ID":"08f8b302-b359-4b7b-a213-4f91e6bba9f0","Type":"ContainerStarted","Data":"73f9a33c9a454633d5e2b1e7fd88768aeaa955bf2d5ac2a4ddb5a83429cb8681"} Sep 13 20:12:52.397091 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:52.397034 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" event={"ID":"ef5aa942-a400-499d-8f53-c81653879c77","Type":"ContainerStarted","Data":"7497414c97000415229d3d2bccf9ab7363457975a92dc7be405a912e4e9ca0e1"} Sep 13 20:12:52.411712 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:52.411656 2557 generic.go:356] "Generic (PLEG): container finished" podID="0202662ce3dd5b921974fd3ab12651d7" containerID="c534b64a2922224d4cca91c484eb34ece397375a771014d5fcdb2473bdd59882" exitCode=0 Sep 13 20:12:52.411849 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:52.411801 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-99.ec2.internal" event={"ID":"0202662ce3dd5b921974fd3ab12651d7","Type":"ContainerDied","Data":"c534b64a2922224d4cca91c484eb34ece397375a771014d5fcdb2473bdd59882"} Sep 13 20:12:52.420578 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:52.420549 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9zrsq" event={"ID":"ca71999d-6b6c-46b6-9c32-819b8b72807c","Type":"ContainerStarted","Data":"2d2d47db664abf538c562b8e9b48b682682a9251ea33c721da81579599c7159b"} Sep 13 20:12:52.428910 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:52.428855 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-99.ec2.internal" podStartSLOduration=3.42883759 podStartE2EDuration="3.42883759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-13 20:12:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:12:52.363318598 +0000 UTC m=+4.539754622" watchObservedRunningTime="2026-09-13 20:12:52.42883759 +0000 UTC m=+4.605273624" Sep 13 20:12:52.442153 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:52.442110 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5h748" event={"ID":"f997e865-7a2e-4898-a3ed-056bf8a4124d","Type":"ContainerStarted","Data":"229abe2fa1b8ce1b081dd43dfa3c2d8973b20cdec70bf045ee3779c5bd357cff"} Sep 13 20:12:52.451870 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:52.451840 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xdjrr" event={"ID":"2a742e26-18ce-4bcb-8735-a69419ffb81a","Type":"ContainerStarted","Data":"465a3344d6c0d6cb2d28569ae685dee1b9394ddab61533e0f13eda261c1df920"} Sep 13 20:12:52.462256 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:52.458672 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vkdmh" event={"ID":"f071dc13-4600-49f7-9396-4ee02d4a2c6f","Type":"ContainerStarted","Data":"13464dc41b7307b0c0e34cf14f6bac429973ad4c4f5e697ad65a07d20a76dbe9"} Sep 13 20:12:53.487176 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:53.487140 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-99.ec2.internal" event={"ID":"0202662ce3dd5b921974fd3ab12651d7","Type":"ContainerStarted","Data":"0f64766a5e8e6ebb6ef13a58a853a35fbba8556e54fdf5c8c66c386a04806859"} Sep 13 20:12:53.501729 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:53.501645 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-99.ec2.internal" podStartSLOduration=4.501624746 podStartE2EDuration="4.501624746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-13 20:12:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:12:53.500146461 +0000 UTC m=+5.676582484" watchObservedRunningTime="2026-09-13 20:12:53.501624746 +0000 UTC m=+5.678060784" Sep 13 20:12:53.880301 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:53.880210 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xldlf\" (UniqueName: \"kubernetes.io/projected/ee8bb811-1f06-4ca4-9766-3406ac59ee27-kube-api-access-xldlf\") pod \"network-check-target-6zzm8\" (UID: \"ee8bb811-1f06-4ca4-9766-3406ac59ee27\") " pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:12:53.880301 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:53.880287 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/855047d0-780f-4f76-a546-e77902ec5399-original-pull-secret\") pod \"global-pull-secret-syncer-6shg5\" (UID: \"855047d0-780f-4f76-a546-e77902ec5399\") " pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:12:53.880545 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:53.880327 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs\") pod \"network-metrics-daemon-95zg2\" (UID: \"32f68c96-97d3-4904-a675-45f93df87572\") " pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:12:53.880610 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:53.880554 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 20:12:53.880658 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:53.880622 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs podName:32f68c96-97d3-4904-a675-45f93df87572 nodeName:}" failed. No retries permitted until 2026-09-13 20:12:57.880602108 +0000 UTC m=+10.057038131 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs") pod "network-metrics-daemon-95zg2" (UID: "32f68c96-97d3-4904-a675-45f93df87572") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 20:12:53.881098 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:53.881075 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 13 20:12:53.881204 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:53.881109 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 13 20:12:53.881204 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:53.881124 2557 projected.go:196] Error preparing data for projected volume kube-api-access-xldlf for pod openshift-network-diagnostics/network-check-target-6zzm8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 20:12:53.881204 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:53.881176 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ee8bb811-1f06-4ca4-9766-3406ac59ee27-kube-api-access-xldlf podName:ee8bb811-1f06-4ca4-9766-3406ac59ee27 nodeName:}" failed. No retries permitted until 2026-09-13 20:12:57.881154914 +0000 UTC m=+10.057590919 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xldlf" (UniqueName: "kubernetes.io/projected/ee8bb811-1f06-4ca4-9766-3406ac59ee27-kube-api-access-xldlf") pod "network-check-target-6zzm8" (UID: "ee8bb811-1f06-4ca4-9766-3406ac59ee27") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 20:12:53.881367 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:53.881245 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 13 20:12:53.881367 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:53.881280 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/855047d0-780f-4f76-a546-e77902ec5399-original-pull-secret podName:855047d0-780f-4f76-a546-e77902ec5399 nodeName:}" failed. No retries permitted until 2026-09-13 20:12:57.881269056 +0000 UTC m=+10.057705060 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/855047d0-780f-4f76-a546-e77902ec5399-original-pull-secret") pod "global-pull-secret-syncer-6shg5" (UID: "855047d0-780f-4f76-a546-e77902ec5399") : object "kube-system"/"original-pull-secret" not registered Sep 13 20:12:54.306565 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:54.306476 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:12:54.306726 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:54.306624 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6shg5" podUID="855047d0-780f-4f76-a546-e77902ec5399" Sep 13 20:12:54.307055 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:54.307033 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:12:54.307169 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:54.307147 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95zg2" podUID="32f68c96-97d3-4904-a675-45f93df87572" Sep 13 20:12:54.307229 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:54.307210 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:12:54.307290 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:54.307273 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zzm8" podUID="ee8bb811-1f06-4ca4-9766-3406ac59ee27" Sep 13 20:12:56.306245 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:56.306217 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:12:56.306712 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:56.306352 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95zg2" podUID="32f68c96-97d3-4904-a675-45f93df87572" Sep 13 20:12:56.306712 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:56.306566 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:12:56.306712 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:56.306679 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6shg5" podUID="855047d0-780f-4f76-a546-e77902ec5399" Sep 13 20:12:56.306880 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:56.306745 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:12:56.306880 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:56.306815 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zzm8" podUID="ee8bb811-1f06-4ca4-9766-3406ac59ee27" Sep 13 20:12:57.914434 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:57.914388 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xldlf\" (UniqueName: \"kubernetes.io/projected/ee8bb811-1f06-4ca4-9766-3406ac59ee27-kube-api-access-xldlf\") pod \"network-check-target-6zzm8\" (UID: \"ee8bb811-1f06-4ca4-9766-3406ac59ee27\") " pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:12:57.914880 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:57.914449 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/855047d0-780f-4f76-a546-e77902ec5399-original-pull-secret\") pod \"global-pull-secret-syncer-6shg5\" (UID: \"855047d0-780f-4f76-a546-e77902ec5399\") " pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:12:57.914880 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:57.914485 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs\") pod \"network-metrics-daemon-95zg2\" (UID: \"32f68c96-97d3-4904-a675-45f93df87572\") " pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:12:57.914880 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:57.914577 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 13 20:12:57.914880 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:57.914600 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 13 20:12:57.914880 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:57.914616 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 20:12:57.914880 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:57.914623 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 13 20:12:57.914880 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:57.914638 2557 projected.go:196] Error preparing data for projected volume kube-api-access-xldlf for pod openshift-network-diagnostics/network-check-target-6zzm8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 20:12:57.914880 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:57.914656 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/855047d0-780f-4f76-a546-e77902ec5399-original-pull-secret podName:855047d0-780f-4f76-a546-e77902ec5399 nodeName:}" failed. No retries permitted until 2026-09-13 20:13:05.914633918 +0000 UTC m=+18.091069933 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/855047d0-780f-4f76-a546-e77902ec5399-original-pull-secret") pod "global-pull-secret-syncer-6shg5" (UID: "855047d0-780f-4f76-a546-e77902ec5399") : object "kube-system"/"original-pull-secret" not registered Sep 13 20:12:57.914880 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:57.914676 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs podName:32f68c96-97d3-4904-a675-45f93df87572 nodeName:}" failed. No retries permitted until 2026-09-13 20:13:05.914666227 +0000 UTC m=+18.091102243 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs") pod "network-metrics-daemon-95zg2" (UID: "32f68c96-97d3-4904-a675-45f93df87572") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 20:12:57.914880 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:57.914691 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ee8bb811-1f06-4ca4-9766-3406ac59ee27-kube-api-access-xldlf podName:ee8bb811-1f06-4ca4-9766-3406ac59ee27 nodeName:}" failed. No retries permitted until 2026-09-13 20:13:05.91468218 +0000 UTC m=+18.091118184 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xldlf" (UniqueName: "kubernetes.io/projected/ee8bb811-1f06-4ca4-9766-3406ac59ee27-kube-api-access-xldlf") pod "network-check-target-6zzm8" (UID: "ee8bb811-1f06-4ca4-9766-3406ac59ee27") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 20:12:58.308917 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:58.308654 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:12:58.309199 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:58.309173 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zzm8" podUID="ee8bb811-1f06-4ca4-9766-3406ac59ee27" Sep 13 20:12:58.309470 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:58.308748 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:12:58.309470 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:58.309371 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95zg2" podUID="32f68c96-97d3-4904-a675-45f93df87572" Sep 13 20:12:58.309470 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:12:58.308718 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:12:58.309470 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:12:58.309445 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6shg5" podUID="855047d0-780f-4f76-a546-e77902ec5399" Sep 13 20:13:00.310453 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:00.310419 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:13:00.310453 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:00.310447 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:13:00.311063 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:00.310421 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:13:00.311063 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:00.310558 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6shg5" podUID="855047d0-780f-4f76-a546-e77902ec5399" Sep 13 20:13:00.311063 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:00.310675 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95zg2" podUID="32f68c96-97d3-4904-a675-45f93df87572" Sep 13 20:13:00.311063 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:00.310740 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zzm8" podUID="ee8bb811-1f06-4ca4-9766-3406ac59ee27" Sep 13 20:13:02.305920 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:02.305804 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:13:02.306357 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:02.305810 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:13:02.306357 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:02.305947 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95zg2" podUID="32f68c96-97d3-4904-a675-45f93df87572" Sep 13 20:13:02.306357 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:02.306041 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6shg5" podUID="855047d0-780f-4f76-a546-e77902ec5399" Sep 13 20:13:02.306357 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:02.305810 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:13:02.306357 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:02.306133 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zzm8" podUID="ee8bb811-1f06-4ca4-9766-3406ac59ee27" Sep 13 20:13:04.308734 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:04.308706 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:13:04.309132 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:04.308715 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:13:04.309132 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:04.308820 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6shg5" podUID="855047d0-780f-4f76-a546-e77902ec5399" Sep 13 20:13:04.309132 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:04.308708 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:13:04.309132 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:04.308896 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95zg2" podUID="32f68c96-97d3-4904-a675-45f93df87572" Sep 13 20:13:04.309132 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:04.309006 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zzm8" podUID="ee8bb811-1f06-4ca4-9766-3406ac59ee27" Sep 13 20:13:05.974589 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:05.974557 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xldlf\" (UniqueName: \"kubernetes.io/projected/ee8bb811-1f06-4ca4-9766-3406ac59ee27-kube-api-access-xldlf\") pod \"network-check-target-6zzm8\" (UID: \"ee8bb811-1f06-4ca4-9766-3406ac59ee27\") " pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:13:05.974589 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:05.974595 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/855047d0-780f-4f76-a546-e77902ec5399-original-pull-secret\") pod \"global-pull-secret-syncer-6shg5\" (UID: \"855047d0-780f-4f76-a546-e77902ec5399\") " pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:13:05.975060 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:05.974617 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs\") pod \"network-metrics-daemon-95zg2\" (UID: \"32f68c96-97d3-4904-a675-45f93df87572\") " pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:13:05.975060 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:05.974721 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 20:13:05.975060 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:05.974722 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 13 20:13:05.975060 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:05.974743 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 13 20:13:05.975060 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:05.974763 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 13 20:13:05.975060 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:05.974776 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs podName:32f68c96-97d3-4904-a675-45f93df87572 nodeName:}" failed. No retries permitted until 2026-09-13 20:13:21.974759341 +0000 UTC m=+34.151195342 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs") pod "network-metrics-daemon-95zg2" (UID: "32f68c96-97d3-4904-a675-45f93df87572") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 20:13:05.975060 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:05.974777 2557 projected.go:196] Error preparing data for projected volume kube-api-access-xldlf for pod openshift-network-diagnostics/network-check-target-6zzm8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 20:13:05.975060 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:05.974793 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/855047d0-780f-4f76-a546-e77902ec5399-original-pull-secret podName:855047d0-780f-4f76-a546-e77902ec5399 nodeName:}" failed. No retries permitted until 2026-09-13 20:13:21.974784306 +0000 UTC m=+34.151220307 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/855047d0-780f-4f76-a546-e77902ec5399-original-pull-secret") pod "global-pull-secret-syncer-6shg5" (UID: "855047d0-780f-4f76-a546-e77902ec5399") : object "kube-system"/"original-pull-secret" not registered Sep 13 20:13:05.975060 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:05.974828 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ee8bb811-1f06-4ca4-9766-3406ac59ee27-kube-api-access-xldlf podName:ee8bb811-1f06-4ca4-9766-3406ac59ee27 nodeName:}" failed. No retries permitted until 2026-09-13 20:13:21.974812449 +0000 UTC m=+34.151248453 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xldlf" (UniqueName: "kubernetes.io/projected/ee8bb811-1f06-4ca4-9766-3406ac59ee27-kube-api-access-xldlf") pod "network-check-target-6zzm8" (UID: "ee8bb811-1f06-4ca4-9766-3406ac59ee27") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 20:13:06.308798 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:06.308697 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:13:06.308798 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:06.308734 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:13:06.308798 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:06.308786 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:13:06.309027 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:06.308809 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zzm8" podUID="ee8bb811-1f06-4ca4-9766-3406ac59ee27" Sep 13 20:13:06.309027 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:06.308884 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95zg2" podUID="32f68c96-97d3-4904-a675-45f93df87572" Sep 13 20:13:06.309027 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:06.308962 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6shg5" podUID="855047d0-780f-4f76-a546-e77902ec5399" Sep 13 20:13:08.309466 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:08.309436 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:13:08.309963 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:08.309471 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:13:08.309963 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:08.309436 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:13:08.309963 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:08.309574 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6shg5" podUID="855047d0-780f-4f76-a546-e77902ec5399" Sep 13 20:13:08.309963 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:08.309647 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95zg2" podUID="32f68c96-97d3-4904-a675-45f93df87572" Sep 13 20:13:08.309963 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:08.309733 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zzm8" podUID="ee8bb811-1f06-4ca4-9766-3406ac59ee27" Sep 13 20:13:09.519747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:09.519555 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9zrsq" event={"ID":"ca71999d-6b6c-46b6-9c32-819b8b72807c","Type":"ContainerStarted","Data":"72fdbe402417dca3c34f039b7f29c2a2c5a0e3a6e9ae5df5d35732924e7338bd"} Sep 13 20:13:09.522362 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:09.522208 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5h748" event={"ID":"f997e865-7a2e-4898-a3ed-056bf8a4124d","Type":"ContainerStarted","Data":"6ea4afc6f09faedd4ef12a13a694777bacd2a436d13d900837589ea5616e693a"} Sep 13 20:13:09.522362 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:09.522305 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5h748" event={"ID":"f997e865-7a2e-4898-a3ed-056bf8a4124d","Type":"ContainerStarted","Data":"1d51eca3e8a90159e2bd7357891d0102d296473a9939314480d9b4e80e5199dd"} Sep 13 20:13:09.524186 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:09.524158 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xdjrr" event={"ID":"2a742e26-18ce-4bcb-8735-a69419ffb81a","Type":"ContainerStarted","Data":"45c0667f43289ed7699a1720ec49c40730265c0ccc1718446b115d046af4de1e"} Sep 13 20:13:09.526026 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:09.525997 2557 generic.go:356] "Generic (PLEG): container finished" podID="f071dc13-4600-49f7-9396-4ee02d4a2c6f" containerID="f931bfd4fad78b05ff8a55f6b4fa02ee7924e6f549b7b8d351c9b99188395658" exitCode=0 Sep 13 20:13:09.526138 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:09.526083 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vkdmh" event={"ID":"f071dc13-4600-49f7-9396-4ee02d4a2c6f","Type":"ContainerDied","Data":"f931bfd4fad78b05ff8a55f6b4fa02ee7924e6f549b7b8d351c9b99188395658"} Sep 13 20:13:09.528048 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:09.528019 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mgvpg" event={"ID":"e80af952-0069-4b10-9b75-2bbef4b3a126","Type":"ContainerStarted","Data":"0e96ddfa0ff1914e8d32cbd893dbbce8aaf5933cceebdfc9877c57d30fbadbf3"} Sep 13 20:13:09.529819 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:09.529790 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7clqq" event={"ID":"819c8604-01b9-4333-a61d-9b47a1b6356d","Type":"ContainerStarted","Data":"7d94691ac565955423483037a312ad8fd138ebe70413be6f9ad2605d187bde2c"} Sep 13 20:13:09.531508 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:09.531468 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-w4brn" event={"ID":"5b2436ba-54ce-4ba5-9ad8-48613ecd34aa","Type":"ContainerStarted","Data":"465767a7708a84561388812669b16d96f0e1c172a414a405edf95c1cfc7bb292"} Sep 13 20:13:09.533196 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:09.533175 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" event={"ID":"ef5aa942-a400-499d-8f53-c81653879c77","Type":"ContainerStarted","Data":"090a7092bdd22c8d35dfe96f75fb5cf03caaddd10a83fdd3550319475e6834b6"} Sep 13 20:13:09.544387 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:09.544349 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mgvpg" podStartSLOduration=3.986098604 podStartE2EDuration="21.544335059s" totalImagesPullingTime="17.558236455s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-13 20:12:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:13:09.543772996 +0000 UTC m=+21.720209018" watchObservedRunningTime="2026-09-13 20:13:09.544335059 +0000 UTC m=+21.720771081" Sep 13 20:13:09.544678 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:09.544647 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9zrsq" podStartSLOduration=11.181400422 podStartE2EDuration="21.544642052s" totalImagesPullingTime="10.36324163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:12:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:13:09.532071985 +0000 UTC m=+21.708508007" watchObservedRunningTime="2026-09-13 20:13:09.544642052 +0000 UTC m=+21.721078074" Sep 13 20:13:09.560730 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:09.560677 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xdjrr" podStartSLOduration=3.997075173 podStartE2EDuration="21.560658164s" totalImagesPullingTime="17.563582991s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-13 20:12:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:13:09.560267055 +0000 UTC m=+21.736703068" watchObservedRunningTime="2026-09-13 20:13:09.560658164 +0000 UTC m=+21.737094189" Sep 13 20:13:09.574741 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:09.574699 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7clqq" podStartSLOduration=4.01440851 podStartE2EDuration="21.574685373s" totalImagesPullingTime="17.560276863s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-13 20:12:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:13:09.573847496 +0000 UTC m=+21.750283575" watchObservedRunningTime="2026-09-13 20:13:09.574685373 +0000 UTC m=+21.751121395" Sep 13 20:13:09.591131 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:09.591095 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-w4brn" podStartSLOduration=3.972817796 podStartE2EDuration="21.591082645s" totalImagesPullingTime="17.618264849s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-13 20:12:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:13:09.590577271 +0000 UTC m=+21.767013294" watchObservedRunningTime="2026-09-13 20:13:09.591082645 +0000 UTC m=+21.767518666" Sep 13 20:13:10.306487 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:10.306187 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:13:10.306754 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:10.306552 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6shg5" podUID="855047d0-780f-4f76-a546-e77902ec5399" Sep 13 20:13:10.306754 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:10.306260 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:13:10.306754 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:10.306646 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zzm8" podUID="ee8bb811-1f06-4ca4-9766-3406ac59ee27" Sep 13 20:13:10.306754 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:10.306218 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:13:10.306754 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:10.306731 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95zg2" podUID="32f68c96-97d3-4904-a675-45f93df87572" Sep 13 20:13:10.538875 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:10.538835 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5h748" event={"ID":"f997e865-7a2e-4898-a3ed-056bf8a4124d","Type":"ContainerStarted","Data":"4679e081a00e1054a69c150654df54c11f3e9addc7641d230ae294b7108d64cb"} Sep 13 20:13:10.539301 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:10.538920 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5h748" event={"ID":"f997e865-7a2e-4898-a3ed-056bf8a4124d","Type":"ContainerStarted","Data":"3132dc9fdff01814e3c07af898453667a2af39f3409c5195fc6ee9e37dde5e0e"} Sep 13 20:13:10.539301 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:10.538984 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5h748" event={"ID":"f997e865-7a2e-4898-a3ed-056bf8a4124d","Type":"ContainerStarted","Data":"19e017015f319d3837d519da408c2c71fdf24864eb8c970636b19236d3483363"} Sep 13 20:13:10.539301 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:10.539018 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5h748" event={"ID":"f997e865-7a2e-4898-a3ed-056bf8a4124d","Type":"ContainerStarted","Data":"9803bbdd6f5b9321ef9b04242f08891848b2ff0fe3414e8d32d77e20cad1eb68"} Sep 13 20:13:10.540397 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:10.540359 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cqvzm" event={"ID":"08f8b302-b359-4b7b-a213-4f91e6bba9f0","Type":"ContainerStarted","Data":"1360aeb216b43efa87bf506c66e56c72bc7cd7c924682983a9547b40b48275d6"} Sep 13 20:13:10.554813 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:10.554772 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-cqvzm" podStartSLOduration=4.987948528 podStartE2EDuration="22.55475957s" totalImagesPullingTime="17.566811042s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-13 20:12:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:13:10.554367315 +0000 UTC m=+22.730803348" watchObservedRunningTime="2026-09-13 20:13:10.55475957 +0000 UTC m=+22.731195591" Sep 13 20:13:10.824906 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:10.824871 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9zrsq" Sep 13 20:13:10.825741 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:10.825707 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9zrsq" Sep 13 20:13:10.913823 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:10.913773 2557 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 13 20:13:11.302985 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:11.302858 2557 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-13T20:13:10.913794328Z","UUID":"9df1574d-f6aa-4268-8853-8e9413dfb82f","Handler":null,"Name":"","Endpoint":""} Sep 13 20:13:11.304355 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:11.304334 2557 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 13 20:13:11.304479 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:11.304378 2557 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 13 20:13:11.545740 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:11.545700 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" event={"ID":"ef5aa942-a400-499d-8f53-c81653879c77","Type":"ContainerStarted","Data":"08d451bbefd1f7b9c45aa6c3c6aa96989a404d6cb8052f83b7053a53c25afe43"} Sep 13 20:13:12.306193 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:12.306162 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:13:12.306361 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:12.306165 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:13:12.306361 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:12.306287 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6shg5" podUID="855047d0-780f-4f76-a546-e77902ec5399" Sep 13 20:13:12.306361 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:12.306306 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:13:12.306578 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:12.306408 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95zg2" podUID="32f68c96-97d3-4904-a675-45f93df87572" Sep 13 20:13:12.306578 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:12.306482 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zzm8" podUID="ee8bb811-1f06-4ca4-9766-3406ac59ee27" Sep 13 20:13:12.552488 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:12.552440 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" event={"ID":"ef5aa942-a400-499d-8f53-c81653879c77","Type":"ContainerStarted","Data":"2a035948ee91671cd6ca9dc24e2986daa1837ebcded044db8e1c012497085a53"} Sep 13 20:13:12.556446 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:12.556415 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5h748" event={"ID":"f997e865-7a2e-4898-a3ed-056bf8a4124d","Type":"ContainerStarted","Data":"8df69ad2824d7b0fb5958eb985bda77b5bd142ac678d0ab1e31ea5a3b09644b9"} Sep 13 20:13:12.556446 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:12.556455 2557 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 13 20:13:12.570260 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:12.570219 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnvjb" podStartSLOduration=4.297903025 podStartE2EDuration="24.570204236s" totalImagesPullingTime="20.272301211s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-13 20:12:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:13:12.569697683 +0000 UTC m=+24.746133704" watchObservedRunningTime="2026-09-13 20:13:12.570204236 +0000 UTC m=+24.746640257" Sep 13 20:13:14.309335 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:14.309137 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:13:14.309813 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:14.309201 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:13:14.309813 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:14.309422 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6shg5" podUID="855047d0-780f-4f76-a546-e77902ec5399" Sep 13 20:13:14.309813 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:14.309208 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:13:14.309813 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:14.309508 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95zg2" podUID="32f68c96-97d3-4904-a675-45f93df87572" Sep 13 20:13:14.309813 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:14.309604 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zzm8" podUID="ee8bb811-1f06-4ca4-9766-3406ac59ee27" Sep 13 20:13:15.566827 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:15.566794 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5h748" event={"ID":"f997e865-7a2e-4898-a3ed-056bf8a4124d","Type":"ContainerStarted","Data":"facbc24ac42569d2de572f5b597387d5d9c68c2dc10e635de1afb1497dd3ffbd"} Sep 13 20:13:15.567667 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:15.567069 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:13:15.568921 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:15.568896 2557 generic.go:356] "Generic (PLEG): container finished" podID="f071dc13-4600-49f7-9396-4ee02d4a2c6f" containerID="7e5ef18fa31ead1187789f3cbcbd860f98e2621a6434486030c95cf51d165feb" exitCode=0 Sep 13 20:13:15.569044 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:15.568932 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vkdmh" event={"ID":"f071dc13-4600-49f7-9396-4ee02d4a2c6f","Type":"ContainerDied","Data":"7e5ef18fa31ead1187789f3cbcbd860f98e2621a6434486030c95cf51d165feb"} Sep 13 20:13:15.582703 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:15.582682 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:13:15.595013 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:15.594966 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5h748" podStartSLOduration=9.985477027 podStartE2EDuration="27.594949896s" totalImagesPullingTime="17.609472869s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-13 20:12:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:13:15.594354534 +0000 UTC m=+27.770790559" watchObservedRunningTime="2026-09-13 20:13:15.594949896 +0000 UTC m=+27.771385918" Sep 13 20:13:16.306054 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:16.305875 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:13:16.306197 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:16.305876 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:13:16.306267 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:16.305924 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:13:16.306267 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:16.306230 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zzm8" podUID="ee8bb811-1f06-4ca4-9766-3406ac59ee27" Sep 13 20:13:16.306360 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:16.306128 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6shg5" podUID="855047d0-780f-4f76-a546-e77902ec5399" Sep 13 20:13:16.306360 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:16.306332 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95zg2" podUID="32f68c96-97d3-4904-a675-45f93df87572" Sep 13 20:13:16.573515 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:16.573408 2557 generic.go:356] "Generic (PLEG): container finished" podID="f071dc13-4600-49f7-9396-4ee02d4a2c6f" containerID="0ccb7f4a6b8b235d066cbc3dd1b2b8fd6c278c551915a0a54b6438117f908712" exitCode=0 Sep 13 20:13:16.573927 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:16.573513 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vkdmh" event={"ID":"f071dc13-4600-49f7-9396-4ee02d4a2c6f","Type":"ContainerDied","Data":"0ccb7f4a6b8b235d066cbc3dd1b2b8fd6c278c551915a0a54b6438117f908712"} Sep 13 20:13:16.574077 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:16.574057 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:13:16.574216 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:16.574136 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:13:16.589114 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:16.589090 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:13:17.096412 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:17.096374 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6shg5"] Sep 13 20:13:17.096621 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:17.096515 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:13:17.096692 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:17.096628 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6shg5" podUID="855047d0-780f-4f76-a546-e77902ec5399" Sep 13 20:13:17.097129 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:17.097104 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6zzm8"] Sep 13 20:13:17.097226 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:17.097185 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:13:17.097294 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:17.097269 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zzm8" podUID="ee8bb811-1f06-4ca4-9766-3406ac59ee27" Sep 13 20:13:17.097699 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:17.097682 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-95zg2"] Sep 13 20:13:17.097760 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:17.097750 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:13:17.097834 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:17.097817 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95zg2" podUID="32f68c96-97d3-4904-a675-45f93df87572" Sep 13 20:13:17.577762 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:17.577732 2557 generic.go:356] "Generic (PLEG): container finished" podID="f071dc13-4600-49f7-9396-4ee02d4a2c6f" containerID="2d5c8c7ec316ddd6823c65e1115869d7579963137ebe711eba21f57f4b92ae11" exitCode=0 Sep 13 20:13:17.578149 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:17.577790 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vkdmh" event={"ID":"f071dc13-4600-49f7-9396-4ee02d4a2c6f","Type":"ContainerDied","Data":"2d5c8c7ec316ddd6823c65e1115869d7579963137ebe711eba21f57f4b92ae11"} Sep 13 20:13:18.222944 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:18.222546 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9zrsq" Sep 13 20:13:18.222944 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:18.222696 2557 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 13 20:13:18.223206 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:18.223195 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9zrsq" Sep 13 20:13:18.306924 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:18.306893 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:13:18.307077 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:18.307014 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:13:18.307077 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:18.307030 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95zg2" podUID="32f68c96-97d3-4904-a675-45f93df87572" Sep 13 20:13:18.307183 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:18.307108 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zzm8" podUID="ee8bb811-1f06-4ca4-9766-3406ac59ee27" Sep 13 20:13:18.307183 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:18.307168 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:13:18.307276 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:18.307242 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6shg5" podUID="855047d0-780f-4f76-a546-e77902ec5399" Sep 13 20:13:20.306080 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:20.306046 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:13:20.306539 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:20.306115 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:13:20.306539 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:20.306121 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:13:20.306539 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:20.306219 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6shg5" podUID="855047d0-780f-4f76-a546-e77902ec5399" Sep 13 20:13:20.306539 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:20.306314 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zzm8" podUID="ee8bb811-1f06-4ca4-9766-3406ac59ee27" Sep 13 20:13:20.306539 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:20.306409 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95zg2" podUID="32f68c96-97d3-4904-a675-45f93df87572" Sep 13 20:13:21.988746 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:21.988703 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/855047d0-780f-4f76-a546-e77902ec5399-original-pull-secret\") pod \"global-pull-secret-syncer-6shg5\" (UID: \"855047d0-780f-4f76-a546-e77902ec5399\") " pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:13:21.989265 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:21.988766 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs\") pod \"network-metrics-daemon-95zg2\" (UID: \"32f68c96-97d3-4904-a675-45f93df87572\") " pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:13:21.989265 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:21.988882 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 13 20:13:21.989265 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:21.988913 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 20:13:21.989265 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:21.988949 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xldlf\" (UniqueName: \"kubernetes.io/projected/ee8bb811-1f06-4ca4-9766-3406ac59ee27-kube-api-access-xldlf\") pod \"network-check-target-6zzm8\" (UID: \"ee8bb811-1f06-4ca4-9766-3406ac59ee27\") " pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:13:21.989265 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:21.988958 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/855047d0-780f-4f76-a546-e77902ec5399-original-pull-secret podName:855047d0-780f-4f76-a546-e77902ec5399 nodeName:}" failed. No retries permitted until 2026-09-13 20:13:53.988936894 +0000 UTC m=+66.165372918 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/855047d0-780f-4f76-a546-e77902ec5399-original-pull-secret") pod "global-pull-secret-syncer-6shg5" (UID: "855047d0-780f-4f76-a546-e77902ec5399") : object "kube-system"/"original-pull-secret" not registered Sep 13 20:13:21.989265 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:21.989017 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs podName:32f68c96-97d3-4904-a675-45f93df87572 nodeName:}" failed. No retries permitted until 2026-09-13 20:13:53.988989199 +0000 UTC m=+66.165425214 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs") pod "network-metrics-daemon-95zg2" (UID: "32f68c96-97d3-4904-a675-45f93df87572") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 13 20:13:21.989265 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:21.989066 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 13 20:13:21.989265 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:21.989084 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 13 20:13:21.989265 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:21.989098 2557 projected.go:196] Error preparing data for projected volume kube-api-access-xldlf for pod openshift-network-diagnostics/network-check-target-6zzm8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 20:13:21.989265 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:21.989144 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ee8bb811-1f06-4ca4-9766-3406ac59ee27-kube-api-access-xldlf podName:ee8bb811-1f06-4ca4-9766-3406ac59ee27 nodeName:}" failed. No retries permitted until 2026-09-13 20:13:53.989131808 +0000 UTC m=+66.165567825 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xldlf" (UniqueName: "kubernetes.io/projected/ee8bb811-1f06-4ca4-9766-3406ac59ee27-kube-api-access-xldlf") pod "network-check-target-6zzm8" (UID: "ee8bb811-1f06-4ca4-9766-3406ac59ee27") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 13 20:13:22.305717 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:22.305678 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:13:22.305988 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:22.305678 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:13:22.305988 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:22.305788 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6shg5" podUID="855047d0-780f-4f76-a546-e77902ec5399" Sep 13 20:13:22.305988 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:22.305693 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:13:22.305988 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:22.305879 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95zg2" podUID="32f68c96-97d3-4904-a675-45f93df87572" Sep 13 20:13:22.306126 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:22.305985 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zzm8" podUID="ee8bb811-1f06-4ca4-9766-3406ac59ee27" Sep 13 20:13:23.136040 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.135868 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-99.ec2.internal" event="NodeReady" Sep 13 20:13:23.136515 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.136141 2557 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 13 20:13:23.167395 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.167364 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-6lr4c"] Sep 13 20:13:23.188922 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.188894 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-9866c996d-vpm54"] Sep 13 20:13:23.189105 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.189029 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6lr4c" Sep 13 20:13:23.191248 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.191223 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 13 20:13:23.191377 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.191258 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 13 20:13:23.191377 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.191323 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-wzmsm\"" Sep 13 20:13:23.203137 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.203116 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-6lr4c"] Sep 13 20:13:23.203242 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.203145 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fw949"] Sep 13 20:13:23.203242 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.203229 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:23.205513 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.205443 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 13 20:13:23.205513 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.205487 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 13 20:13:23.205748 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.205564 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 13 20:13:23.205748 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.205737 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zlk89\"" Sep 13 20:13:23.210877 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.210861 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 13 20:13:23.221268 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.221247 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-k8c5m"] Sep 13 20:13:23.221361 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.221348 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fw949" Sep 13 20:13:23.223561 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.223529 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 13 20:13:23.223774 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.223755 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 13 20:13:23.223856 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.223761 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9fnqc\"" Sep 13 20:13:23.247384 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.247359 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9866c996d-vpm54"] Sep 13 20:13:23.247384 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.247393 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fw949"] Sep 13 20:13:23.247576 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.247413 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-k8c5m"] Sep 13 20:13:23.247576 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.247476 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-k8c5m" Sep 13 20:13:23.249935 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.249917 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 13 20:13:23.249935 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.249933 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 13 20:13:23.250113 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.250040 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dgck6\"" Sep 13 20:13:23.250113 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.250061 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 13 20:13:23.298697 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.298666 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-tmp-dir\") pod \"dns-default-fw949\" (UID: \"a99d1dc1-43a2-4ada-99dd-a4a05c5faf68\") " pod="openshift-dns/dns-default-fw949" Sep 13 20:13:23.298697 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.298698 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-bound-sa-token\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:23.298892 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.298731 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8dz9\" (UniqueName: \"kubernetes.io/projected/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-kube-api-access-r8dz9\") pod \"dns-default-fw949\" (UID: \"a99d1dc1-43a2-4ada-99dd-a4a05c5faf68\") " pod="openshift-dns/dns-default-fw949" Sep 13 20:13:23.298892 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.298780 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-6lr4c\" (UID: \"ba047ed2-663d-47be-bd5f-08548fdcdec9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6lr4c" Sep 13 20:13:23.298892 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.298799 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-image-registry-private-configuration\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:23.298892 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.298839 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls\") pod \"dns-default-fw949\" (UID: \"a99d1dc1-43a2-4ada-99dd-a4a05c5faf68\") " pod="openshift-dns/dns-default-fw949" Sep 13 20:13:23.298892 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.298861 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-installation-pull-secrets\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:23.298892 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.298880 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-certificates\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:23.299085 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.298925 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdpwn\" (UniqueName: \"kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-kube-api-access-qdpwn\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:23.299085 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.298972 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-config-volume\") pod \"dns-default-fw949\" (UID: \"a99d1dc1-43a2-4ada-99dd-a4a05c5faf68\") " pod="openshift-dns/dns-default-fw949" Sep 13 20:13:23.299085 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.298993 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:23.299085 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.299011 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-trusted-ca\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:23.299085 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.299032 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ba047ed2-663d-47be-bd5f-08548fdcdec9-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-6lr4c\" (UID: \"ba047ed2-663d-47be-bd5f-08548fdcdec9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6lr4c" Sep 13 20:13:23.299085 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.299054 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-ca-trust-extracted\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:23.399875 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.399789 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-certificates\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:23.399875 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.399831 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdpwn\" (UniqueName: \"kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-kube-api-access-qdpwn\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:23.399875 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.399858 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-config-volume\") pod \"dns-default-fw949\" (UID: \"a99d1dc1-43a2-4ada-99dd-a4a05c5faf68\") " pod="openshift-dns/dns-default-fw949" Sep 13 20:13:23.400194 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.399884 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:23.400194 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.399906 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-trusted-ca\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:23.400194 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.399931 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ba047ed2-663d-47be-bd5f-08548fdcdec9-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-6lr4c\" (UID: \"ba047ed2-663d-47be-bd5f-08548fdcdec9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6lr4c" Sep 13 20:13:23.400194 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.399952 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-ca-trust-extracted\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:23.400194 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.399978 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-tmp-dir\") pod \"dns-default-fw949\" (UID: \"a99d1dc1-43a2-4ada-99dd-a4a05c5faf68\") " pod="openshift-dns/dns-default-fw949" Sep 13 20:13:23.400194 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.400002 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b70f512-38d7-4546-be8f-c40233eabd8f-cert\") pod \"ingress-canary-k8c5m\" (UID: \"5b70f512-38d7-4546-be8f-c40233eabd8f\") " pod="openshift-ingress-canary/ingress-canary-k8c5m" Sep 13 20:13:23.400194 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:23.400025 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 13 20:13:23.400194 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:23.400053 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9866c996d-vpm54: secret "image-registry-tls" not found Sep 13 20:13:23.400194 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:23.400127 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls podName:fd0e27d4-cbed-4619-94be-5c9f8eb73fc2 nodeName:}" failed. No retries permitted until 2026-09-13 20:13:23.900104445 +0000 UTC m=+36.076540451 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls") pod "image-registry-9866c996d-vpm54" (UID: "fd0e27d4-cbed-4619-94be-5c9f8eb73fc2") : secret "image-registry-tls" not found Sep 13 20:13:23.400194 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.400033 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-bound-sa-token\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:23.400708 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.400259 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8dz9\" (UniqueName: \"kubernetes.io/projected/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-kube-api-access-r8dz9\") pod \"dns-default-fw949\" (UID: \"a99d1dc1-43a2-4ada-99dd-a4a05c5faf68\") " pod="openshift-dns/dns-default-fw949" Sep 13 20:13:23.400708 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.400321 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-6lr4c\" (UID: \"ba047ed2-663d-47be-bd5f-08548fdcdec9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6lr4c" Sep 13 20:13:23.400708 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.400359 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-image-registry-private-configuration\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:23.400708 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.400414 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls\") pod \"dns-default-fw949\" (UID: \"a99d1dc1-43a2-4ada-99dd-a4a05c5faf68\") " pod="openshift-dns/dns-default-fw949" Sep 13 20:13:23.400708 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.400444 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-installation-pull-secrets\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:23.400708 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.400475 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r764\" (UniqueName: \"kubernetes.io/projected/5b70f512-38d7-4546-be8f-c40233eabd8f-kube-api-access-4r764\") pod \"ingress-canary-k8c5m\" (UID: \"5b70f512-38d7-4546-be8f-c40233eabd8f\") " pod="openshift-ingress-canary/ingress-canary-k8c5m" Sep 13 20:13:23.400708 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:23.400545 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 13 20:13:23.400708 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.400580 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-certificates\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:23.400708 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:23.400623 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert podName:ba047ed2-663d-47be-bd5f-08548fdcdec9 nodeName:}" failed. No retries permitted until 2026-09-13 20:13:23.900604618 +0000 UTC m=+36.077040632 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-6lr4c" (UID: "ba047ed2-663d-47be-bd5f-08548fdcdec9") : secret "networking-console-plugin-cert" not found Sep 13 20:13:23.400708 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.400665 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-tmp-dir\") pod \"dns-default-fw949\" (UID: \"a99d1dc1-43a2-4ada-99dd-a4a05c5faf68\") " pod="openshift-dns/dns-default-fw949" Sep 13 20:13:23.400708 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:23.400691 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 13 20:13:23.400708 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.400700 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-config-volume\") pod \"dns-default-fw949\" (UID: \"a99d1dc1-43a2-4ada-99dd-a4a05c5faf68\") " pod="openshift-dns/dns-default-fw949" Sep 13 20:13:23.401084 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:23.400746 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls podName:a99d1dc1-43a2-4ada-99dd-a4a05c5faf68 nodeName:}" failed. No retries permitted until 2026-09-13 20:13:23.900725833 +0000 UTC m=+36.077161846 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls") pod "dns-default-fw949" (UID: "a99d1dc1-43a2-4ada-99dd-a4a05c5faf68") : secret "dns-default-metrics-tls" not found Sep 13 20:13:23.401084 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.400807 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ba047ed2-663d-47be-bd5f-08548fdcdec9-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-6lr4c\" (UID: \"ba047ed2-663d-47be-bd5f-08548fdcdec9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6lr4c" Sep 13 20:13:23.401084 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.401017 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-ca-trust-extracted\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:23.401398 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.401369 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-trusted-ca\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:23.404529 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.404488 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-installation-pull-secrets\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:23.404602 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.404514 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-image-registry-private-configuration\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:23.407766 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.407738 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8dz9\" (UniqueName: \"kubernetes.io/projected/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-kube-api-access-r8dz9\") pod \"dns-default-fw949\" (UID: \"a99d1dc1-43a2-4ada-99dd-a4a05c5faf68\") " pod="openshift-dns/dns-default-fw949" Sep 13 20:13:23.407881 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.407866 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-bound-sa-token\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:23.408088 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.408070 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdpwn\" (UniqueName: \"kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-kube-api-access-qdpwn\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:23.501124 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.501089 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r764\" (UniqueName: \"kubernetes.io/projected/5b70f512-38d7-4546-be8f-c40233eabd8f-kube-api-access-4r764\") pod \"ingress-canary-k8c5m\" (UID: \"5b70f512-38d7-4546-be8f-c40233eabd8f\") " pod="openshift-ingress-canary/ingress-canary-k8c5m" Sep 13 20:13:23.501308 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.501156 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b70f512-38d7-4546-be8f-c40233eabd8f-cert\") pod \"ingress-canary-k8c5m\" (UID: \"5b70f512-38d7-4546-be8f-c40233eabd8f\") " pod="openshift-ingress-canary/ingress-canary-k8c5m" Sep 13 20:13:23.503427 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.503406 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b70f512-38d7-4546-be8f-c40233eabd8f-cert\") pod \"ingress-canary-k8c5m\" (UID: \"5b70f512-38d7-4546-be8f-c40233eabd8f\") " pod="openshift-ingress-canary/ingress-canary-k8c5m" Sep 13 20:13:23.508289 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.508270 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r764\" (UniqueName: \"kubernetes.io/projected/5b70f512-38d7-4546-be8f-c40233eabd8f-kube-api-access-4r764\") pod \"ingress-canary-k8c5m\" (UID: \"5b70f512-38d7-4546-be8f-c40233eabd8f\") " pod="openshift-ingress-canary/ingress-canary-k8c5m" Sep 13 20:13:23.555474 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.555454 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-k8c5m" Sep 13 20:13:23.602631 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.602606 2557 generic.go:356] "Generic (PLEG): container finished" podID="f071dc13-4600-49f7-9396-4ee02d4a2c6f" containerID="c91e23d098264541a632e594acceb3a109009824e9f976ff9c2e5aaf2ca6a67e" exitCode=0 Sep 13 20:13:23.602731 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.602656 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vkdmh" event={"ID":"f071dc13-4600-49f7-9396-4ee02d4a2c6f","Type":"ContainerDied","Data":"c91e23d098264541a632e594acceb3a109009824e9f976ff9c2e5aaf2ca6a67e"} Sep 13 20:13:23.889772 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.889742 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-k8c5m"] Sep 13 20:13:23.900599 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:13:23.900572 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b70f512_38d7_4546_be8f_c40233eabd8f.slice/crio-583f6188b0d6e75be54599d6696eb349681d59b4a93d3b7c5832bff88e0f7c22 WatchSource:0}: Error finding container 583f6188b0d6e75be54599d6696eb349681d59b4a93d3b7c5832bff88e0f7c22: Status 404 returned error can't find the container with id 583f6188b0d6e75be54599d6696eb349681d59b4a93d3b7c5832bff88e0f7c22 Sep 13 20:13:23.904042 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.904020 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-6lr4c\" (UID: \"ba047ed2-663d-47be-bd5f-08548fdcdec9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6lr4c" Sep 13 20:13:23.904119 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.904069 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls\") pod \"dns-default-fw949\" (UID: \"a99d1dc1-43a2-4ada-99dd-a4a05c5faf68\") " pod="openshift-dns/dns-default-fw949" Sep 13 20:13:23.904119 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:23.904109 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:23.904226 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:23.904159 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 13 20:13:23.904226 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:23.904221 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert podName:ba047ed2-663d-47be-bd5f-08548fdcdec9 nodeName:}" failed. No retries permitted until 2026-09-13 20:13:24.904193924 +0000 UTC m=+37.080629937 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-6lr4c" (UID: "ba047ed2-663d-47be-bd5f-08548fdcdec9") : secret "networking-console-plugin-cert" not found Sep 13 20:13:23.904330 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:23.904225 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 13 20:13:23.904330 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:23.904239 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9866c996d-vpm54: secret "image-registry-tls" not found Sep 13 20:13:23.904330 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:23.904265 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls podName:fd0e27d4-cbed-4619-94be-5c9f8eb73fc2 nodeName:}" failed. No retries permitted until 2026-09-13 20:13:24.904258233 +0000 UTC m=+37.080694232 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls") pod "image-registry-9866c996d-vpm54" (UID: "fd0e27d4-cbed-4619-94be-5c9f8eb73fc2") : secret "image-registry-tls" not found Sep 13 20:13:23.904330 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:23.904276 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 13 20:13:23.904541 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:23.904339 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls podName:a99d1dc1-43a2-4ada-99dd-a4a05c5faf68 nodeName:}" failed. No retries permitted until 2026-09-13 20:13:24.904320977 +0000 UTC m=+37.080756991 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls") pod "dns-default-fw949" (UID: "a99d1dc1-43a2-4ada-99dd-a4a05c5faf68") : secret "dns-default-metrics-tls" not found Sep 13 20:13:24.306515 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:24.306461 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:13:24.307266 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:24.306461 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:13:24.307266 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:24.306601 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:13:24.309098 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:24.308873 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 13 20:13:24.309098 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:24.308893 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ln5vr\"" Sep 13 20:13:24.309306 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:24.308956 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 13 20:13:24.309306 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:24.308960 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 13 20:13:24.309306 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:24.308982 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 13 20:13:24.309306 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:24.308982 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5c5lx\"" Sep 13 20:13:24.609791 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:24.609678 2557 generic.go:356] "Generic (PLEG): container finished" podID="f071dc13-4600-49f7-9396-4ee02d4a2c6f" containerID="5d2d5dc7d34320a167ec23fe9d3d25ad41571527ca92391566b5bdf08fef5e67" exitCode=0 Sep 13 20:13:24.609791 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:24.609763 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vkdmh" event={"ID":"f071dc13-4600-49f7-9396-4ee02d4a2c6f","Type":"ContainerDied","Data":"5d2d5dc7d34320a167ec23fe9d3d25ad41571527ca92391566b5bdf08fef5e67"} Sep 13 20:13:24.611237 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:24.611214 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-k8c5m" event={"ID":"5b70f512-38d7-4546-be8f-c40233eabd8f","Type":"ContainerStarted","Data":"583f6188b0d6e75be54599d6696eb349681d59b4a93d3b7c5832bff88e0f7c22"} Sep 13 20:13:24.910465 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:24.910381 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:24.910647 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:24.910472 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-6lr4c\" (UID: \"ba047ed2-663d-47be-bd5f-08548fdcdec9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6lr4c" Sep 13 20:13:24.910647 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:24.910548 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls\") pod \"dns-default-fw949\" (UID: \"a99d1dc1-43a2-4ada-99dd-a4a05c5faf68\") " pod="openshift-dns/dns-default-fw949" Sep 13 20:13:24.910647 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:24.910582 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 13 20:13:24.910647 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:24.910601 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9866c996d-vpm54: secret "image-registry-tls" not found Sep 13 20:13:24.910647 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:24.910609 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 13 20:13:24.910893 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:24.910654 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 13 20:13:24.910893 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:24.910669 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls podName:fd0e27d4-cbed-4619-94be-5c9f8eb73fc2 nodeName:}" failed. No retries permitted until 2026-09-13 20:13:26.910648673 +0000 UTC m=+39.087084673 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls") pod "image-registry-9866c996d-vpm54" (UID: "fd0e27d4-cbed-4619-94be-5c9f8eb73fc2") : secret "image-registry-tls" not found Sep 13 20:13:24.910893 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:24.910687 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert podName:ba047ed2-663d-47be-bd5f-08548fdcdec9 nodeName:}" failed. No retries permitted until 2026-09-13 20:13:26.91067773 +0000 UTC m=+39.087113731 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-6lr4c" (UID: "ba047ed2-663d-47be-bd5f-08548fdcdec9") : secret "networking-console-plugin-cert" not found Sep 13 20:13:24.910893 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:24.910703 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls podName:a99d1dc1-43a2-4ada-99dd-a4a05c5faf68 nodeName:}" failed. No retries permitted until 2026-09-13 20:13:26.910694957 +0000 UTC m=+39.087130964 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls") pod "dns-default-fw949" (UID: "a99d1dc1-43a2-4ada-99dd-a4a05c5faf68") : secret "dns-default-metrics-tls" not found Sep 13 20:13:25.616630 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:25.616600 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vkdmh" event={"ID":"f071dc13-4600-49f7-9396-4ee02d4a2c6f","Type":"ContainerStarted","Data":"5a04ace303455854fe0494be7f770dcf9c1ba4e1a849e18dabf16c1e5d55daf0"} Sep 13 20:13:25.637813 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:25.637751 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vkdmh" podStartSLOduration=7.8369476890000005 podStartE2EDuration="37.637728109s" totalImagesPullingTime="29.80078042s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-13 20:12:48 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:13:25.635657231 +0000 UTC m=+37.812093255" watchObservedRunningTime="2026-09-13 20:13:25.637728109 +0000 UTC m=+37.814164132" Sep 13 20:13:26.926013 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:26.925985 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-6lr4c\" (UID: \"ba047ed2-663d-47be-bd5f-08548fdcdec9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6lr4c" Sep 13 20:13:26.926387 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:26.926031 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls\") pod \"dns-default-fw949\" (UID: \"a99d1dc1-43a2-4ada-99dd-a4a05c5faf68\") " pod="openshift-dns/dns-default-fw949" Sep 13 20:13:26.926387 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:26.926059 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:26.926387 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:26.926135 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 13 20:13:26.926387 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:26.926151 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 13 20:13:26.926387 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:26.926165 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9866c996d-vpm54: secret "image-registry-tls" not found Sep 13 20:13:26.926387 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:26.926207 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls podName:fd0e27d4-cbed-4619-94be-5c9f8eb73fc2 nodeName:}" failed. No retries permitted until 2026-09-13 20:13:30.926192778 +0000 UTC m=+43.102628792 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls") pod "image-registry-9866c996d-vpm54" (UID: "fd0e27d4-cbed-4619-94be-5c9f8eb73fc2") : secret "image-registry-tls" not found Sep 13 20:13:26.926387 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:26.926218 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert podName:ba047ed2-663d-47be-bd5f-08548fdcdec9 nodeName:}" failed. No retries permitted until 2026-09-13 20:13:30.926212955 +0000 UTC m=+43.102648954 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-6lr4c" (UID: "ba047ed2-663d-47be-bd5f-08548fdcdec9") : secret "networking-console-plugin-cert" not found Sep 13 20:13:26.926387 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:26.926223 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 13 20:13:26.926387 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:26.926284 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls podName:a99d1dc1-43a2-4ada-99dd-a4a05c5faf68 nodeName:}" failed. No retries permitted until 2026-09-13 20:13:30.926266111 +0000 UTC m=+43.102702112 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls") pod "dns-default-fw949" (UID: "a99d1dc1-43a2-4ada-99dd-a4a05c5faf68") : secret "dns-default-metrics-tls" not found Sep 13 20:13:27.624316 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:27.624281 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-k8c5m" event={"ID":"5b70f512-38d7-4546-be8f-c40233eabd8f","Type":"ContainerStarted","Data":"c5ababed2e6d3a7e5a6fb9da4ffecb011d8d92476d93ea4f7a4516769425c3e2"} Sep 13 20:13:27.638621 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:27.638571 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-k8c5m" podStartSLOduration=1.716249057 podStartE2EDuration="4.638558892s" totalImagesPullingTime="2.922309835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:13:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:13:27.637474301 +0000 UTC m=+39.813910323" watchObservedRunningTime="2026-09-13 20:13:27.638558892 +0000 UTC m=+39.814994911" Sep 13 20:13:30.957305 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:30.957267 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls\") pod \"dns-default-fw949\" (UID: \"a99d1dc1-43a2-4ada-99dd-a4a05c5faf68\") " pod="openshift-dns/dns-default-fw949" Sep 13 20:13:30.957750 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:30.957315 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:30.957750 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:30.957388 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-6lr4c\" (UID: \"ba047ed2-663d-47be-bd5f-08548fdcdec9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6lr4c" Sep 13 20:13:30.957750 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:30.957427 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 13 20:13:30.957750 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:30.957468 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 13 20:13:30.957750 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:30.957509 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 13 20:13:30.957750 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:30.957529 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9866c996d-vpm54: secret "image-registry-tls" not found Sep 13 20:13:30.957750 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:30.957515 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls podName:a99d1dc1-43a2-4ada-99dd-a4a05c5faf68 nodeName:}" failed. No retries permitted until 2026-09-13 20:13:38.957479891 +0000 UTC m=+51.133915907 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls") pod "dns-default-fw949" (UID: "a99d1dc1-43a2-4ada-99dd-a4a05c5faf68") : secret "dns-default-metrics-tls" not found Sep 13 20:13:30.957750 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:30.957589 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert podName:ba047ed2-663d-47be-bd5f-08548fdcdec9 nodeName:}" failed. No retries permitted until 2026-09-13 20:13:38.957575631 +0000 UTC m=+51.134011631 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-6lr4c" (UID: "ba047ed2-663d-47be-bd5f-08548fdcdec9") : secret "networking-console-plugin-cert" not found Sep 13 20:13:30.957750 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:30.957600 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls podName:fd0e27d4-cbed-4619-94be-5c9f8eb73fc2 nodeName:}" failed. No retries permitted until 2026-09-13 20:13:38.957593411 +0000 UTC m=+51.134029410 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls") pod "image-registry-9866c996d-vpm54" (UID: "fd0e27d4-cbed-4619-94be-5c9f8eb73fc2") : secret "image-registry-tls" not found Sep 13 20:13:39.016819 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:39.016779 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls\") pod \"dns-default-fw949\" (UID: \"a99d1dc1-43a2-4ada-99dd-a4a05c5faf68\") " pod="openshift-dns/dns-default-fw949" Sep 13 20:13:39.017358 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:39.016839 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:39.017358 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:39.016910 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-6lr4c\" (UID: \"ba047ed2-663d-47be-bd5f-08548fdcdec9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6lr4c" Sep 13 20:13:39.017358 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:39.016944 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 13 20:13:39.017358 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:39.016982 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 13 20:13:39.017358 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:39.016997 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9866c996d-vpm54: secret "image-registry-tls" not found Sep 13 20:13:39.017358 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:39.017023 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls podName:a99d1dc1-43a2-4ada-99dd-a4a05c5faf68 nodeName:}" failed. No retries permitted until 2026-09-13 20:13:55.017000953 +0000 UTC m=+67.193436975 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls") pod "dns-default-fw949" (UID: "a99d1dc1-43a2-4ada-99dd-a4a05c5faf68") : secret "dns-default-metrics-tls" not found Sep 13 20:13:39.017358 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:39.017037 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 13 20:13:39.017358 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:39.017042 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls podName:fd0e27d4-cbed-4619-94be-5c9f8eb73fc2 nodeName:}" failed. No retries permitted until 2026-09-13 20:13:55.017033314 +0000 UTC m=+67.193469314 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls") pod "image-registry-9866c996d-vpm54" (UID: "fd0e27d4-cbed-4619-94be-5c9f8eb73fc2") : secret "image-registry-tls" not found Sep 13 20:13:39.017358 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:39.017105 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert podName:ba047ed2-663d-47be-bd5f-08548fdcdec9 nodeName:}" failed. No retries permitted until 2026-09-13 20:13:55.017088964 +0000 UTC m=+67.193524972 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-6lr4c" (UID: "ba047ed2-663d-47be-bd5f-08548fdcdec9") : secret "networking-console-plugin-cert" not found Sep 13 20:13:48.590922 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:48.590895 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5h748" Sep 13 20:13:54.027579 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:54.027536 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xldlf\" (UniqueName: \"kubernetes.io/projected/ee8bb811-1f06-4ca4-9766-3406ac59ee27-kube-api-access-xldlf\") pod \"network-check-target-6zzm8\" (UID: \"ee8bb811-1f06-4ca4-9766-3406ac59ee27\") " pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:13:54.028013 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:54.027592 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/855047d0-780f-4f76-a546-e77902ec5399-original-pull-secret\") pod \"global-pull-secret-syncer-6shg5\" (UID: \"855047d0-780f-4f76-a546-e77902ec5399\") " pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:13:54.028013 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:54.027636 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs\") pod \"network-metrics-daemon-95zg2\" (UID: \"32f68c96-97d3-4904-a675-45f93df87572\") " pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:13:54.030145 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:54.030126 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 13 20:13:54.030198 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:54.030167 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 13 20:13:54.030198 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:54.030187 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 13 20:13:54.037969 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:54.037948 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 13 20:13:54.038060 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:54.038020 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs podName:32f68c96-97d3-4904-a675-45f93df87572 nodeName:}" failed. No retries permitted until 2026-09-13 20:14:58.038004595 +0000 UTC m=+130.214440599 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs") pod "network-metrics-daemon-95zg2" (UID: "32f68c96-97d3-4904-a675-45f93df87572") : secret "metrics-daemon-secret" not found Sep 13 20:13:54.040121 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:54.040093 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 13 20:13:54.041308 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:54.041289 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/855047d0-780f-4f76-a546-e77902ec5399-original-pull-secret\") pod \"global-pull-secret-syncer-6shg5\" (UID: \"855047d0-780f-4f76-a546-e77902ec5399\") " pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:13:54.050904 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:54.050885 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xldlf\" (UniqueName: \"kubernetes.io/projected/ee8bb811-1f06-4ca4-9766-3406ac59ee27-kube-api-access-xldlf\") pod \"network-check-target-6zzm8\" (UID: \"ee8bb811-1f06-4ca4-9766-3406ac59ee27\") " pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:13:54.318640 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:54.318564 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ln5vr\"" Sep 13 20:13:54.321377 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:54.321358 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6shg5" Sep 13 20:13:54.326943 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:54.326912 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:13:54.597180 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:54.597102 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6shg5"] Sep 13 20:13:54.598151 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:54.598131 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6zzm8"] Sep 13 20:13:54.599842 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:13:54.599809 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod855047d0_780f_4f76_a546_e77902ec5399.slice/crio-227dbe61ff27676a4335a2d2dabfd8e8bfa53088325dbfb4fe9de390d0bff775 WatchSource:0}: Error finding container 227dbe61ff27676a4335a2d2dabfd8e8bfa53088325dbfb4fe9de390d0bff775: Status 404 returned error can't find the container with id 227dbe61ff27676a4335a2d2dabfd8e8bfa53088325dbfb4fe9de390d0bff775 Sep 13 20:13:54.600368 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:13:54.600336 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee8bb811_1f06_4ca4_9766_3406ac59ee27.slice/crio-a5c98014ebe5d3fd62786f13479ad987fea9e310f8bad993ceabb1916a59b7ec WatchSource:0}: Error finding container a5c98014ebe5d3fd62786f13479ad987fea9e310f8bad993ceabb1916a59b7ec: Status 404 returned error can't find the container with id a5c98014ebe5d3fd62786f13479ad987fea9e310f8bad993ceabb1916a59b7ec Sep 13 20:13:54.686487 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:54.686450 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6zzm8" event={"ID":"ee8bb811-1f06-4ca4-9766-3406ac59ee27","Type":"ContainerStarted","Data":"a5c98014ebe5d3fd62786f13479ad987fea9e310f8bad993ceabb1916a59b7ec"} Sep 13 20:13:54.687482 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:54.687457 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6shg5" event={"ID":"855047d0-780f-4f76-a546-e77902ec5399","Type":"ContainerStarted","Data":"227dbe61ff27676a4335a2d2dabfd8e8bfa53088325dbfb4fe9de390d0bff775"} Sep 13 20:13:55.034760 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:55.034729 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls\") pod \"dns-default-fw949\" (UID: \"a99d1dc1-43a2-4ada-99dd-a4a05c5faf68\") " pod="openshift-dns/dns-default-fw949" Sep 13 20:13:55.035115 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:55.034775 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:13:55.035115 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:55.034812 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-6lr4c\" (UID: \"ba047ed2-663d-47be-bd5f-08548fdcdec9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6lr4c" Sep 13 20:13:55.035115 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:55.034885 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 13 20:13:55.035115 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:55.034943 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 13 20:13:55.035115 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:55.034961 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9866c996d-vpm54: secret "image-registry-tls" not found Sep 13 20:13:55.035115 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:55.034945 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls podName:a99d1dc1-43a2-4ada-99dd-a4a05c5faf68 nodeName:}" failed. No retries permitted until 2026-09-13 20:14:27.034930225 +0000 UTC m=+99.211366231 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls") pod "dns-default-fw949" (UID: "a99d1dc1-43a2-4ada-99dd-a4a05c5faf68") : secret "dns-default-metrics-tls" not found Sep 13 20:13:55.035115 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:55.035022 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls podName:fd0e27d4-cbed-4619-94be-5c9f8eb73fc2 nodeName:}" failed. No retries permitted until 2026-09-13 20:14:27.035009449 +0000 UTC m=+99.211445448 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls") pod "image-registry-9866c996d-vpm54" (UID: "fd0e27d4-cbed-4619-94be-5c9f8eb73fc2") : secret "image-registry-tls" not found Sep 13 20:13:55.035115 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:55.034894 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 13 20:13:55.035115 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:13:55.035073 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert podName:ba047ed2-663d-47be-bd5f-08548fdcdec9 nodeName:}" failed. No retries permitted until 2026-09-13 20:14:27.035061749 +0000 UTC m=+99.211497749 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-6lr4c" (UID: "ba047ed2-663d-47be-bd5f-08548fdcdec9") : secret "networking-console-plugin-cert" not found Sep 13 20:13:59.701736 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:59.701654 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6zzm8" event={"ID":"ee8bb811-1f06-4ca4-9766-3406ac59ee27","Type":"ContainerStarted","Data":"fe28a7c488b7585294282d3a1fa28e04b284a70ad0fafc4670012f89a4392fbc"} Sep 13 20:13:59.701736 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:59.701731 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:13:59.703095 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:59.703072 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6shg5" event={"ID":"855047d0-780f-4f76-a546-e77902ec5399","Type":"ContainerStarted","Data":"cd74b7b819c918937a35cb2c4f89472f23f08970e85c8b43e3733853f3da32d4"} Sep 13 20:13:59.716482 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:59.716435 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6zzm8" podStartSLOduration=66.929555392 podStartE2EDuration="1m11.716407337s" totalImagesPullingTime="4.786851945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:12:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:13:59.715972195 +0000 UTC m=+71.892408217" watchObservedRunningTime="2026-09-13 20:13:59.716407337 +0000 UTC m=+71.892843356" Sep 13 20:13:59.729155 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:13:59.729108 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6shg5" podStartSLOduration=66.934406706 podStartE2EDuration="1m11.729093903s" totalImagesPullingTime="4.794687197s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-13 20:12:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:13:59.728617759 +0000 UTC m=+71.905053782" watchObservedRunningTime="2026-09-13 20:13:59.729093903 +0000 UTC m=+71.905529959" Sep 13 20:14:27.066010 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:14:27.065966 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-6lr4c\" (UID: \"ba047ed2-663d-47be-bd5f-08548fdcdec9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6lr4c" Sep 13 20:14:27.066486 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:14:27.066020 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls\") pod \"dns-default-fw949\" (UID: \"a99d1dc1-43a2-4ada-99dd-a4a05c5faf68\") " pod="openshift-dns/dns-default-fw949" Sep 13 20:14:27.066486 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:14:27.066046 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:14:27.066486 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:14:27.066128 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 13 20:14:27.066486 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:14:27.066182 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 13 20:14:27.066486 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:14:27.066206 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert podName:ba047ed2-663d-47be-bd5f-08548fdcdec9 nodeName:}" failed. No retries permitted until 2026-09-13 20:15:31.066191236 +0000 UTC m=+163.242627236 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-6lr4c" (UID: "ba047ed2-663d-47be-bd5f-08548fdcdec9") : secret "networking-console-plugin-cert" not found Sep 13 20:14:27.066486 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:14:27.066136 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 13 20:14:27.066486 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:14:27.066230 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9866c996d-vpm54: secret "image-registry-tls" not found Sep 13 20:14:27.066486 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:14:27.066248 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls podName:a99d1dc1-43a2-4ada-99dd-a4a05c5faf68 nodeName:}" failed. No retries permitted until 2026-09-13 20:15:31.06623118 +0000 UTC m=+163.242667181 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls") pod "dns-default-fw949" (UID: "a99d1dc1-43a2-4ada-99dd-a4a05c5faf68") : secret "dns-default-metrics-tls" not found Sep 13 20:14:27.066486 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:14:27.066268 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls podName:fd0e27d4-cbed-4619-94be-5c9f8eb73fc2 nodeName:}" failed. No retries permitted until 2026-09-13 20:15:31.066258844 +0000 UTC m=+163.242694844 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls") pod "image-registry-9866c996d-vpm54" (UID: "fd0e27d4-cbed-4619-94be-5c9f8eb73fc2") : secret "image-registry-tls" not found Sep 13 20:14:30.709010 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:14:30.708982 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6zzm8" Sep 13 20:14:58.088807 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:14:58.088750 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs\") pod \"network-metrics-daemon-95zg2\" (UID: \"32f68c96-97d3-4904-a675-45f93df87572\") " pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:14:58.089292 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:14:58.088914 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 13 20:14:58.089292 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:14:58.088980 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs podName:32f68c96-97d3-4904-a675-45f93df87572 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:00.08896441 +0000 UTC m=+252.265400440 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs") pod "network-metrics-daemon-95zg2" (UID: "32f68c96-97d3-4904-a675-45f93df87572") : secret "metrics-daemon-secret" not found Sep 13 20:15:26.199511 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:15:26.199454 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6lr4c" podUID="ba047ed2-663d-47be-bd5f-08548fdcdec9" Sep 13 20:15:26.211810 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:15:26.211778 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-9866c996d-vpm54" podUID="fd0e27d4-cbed-4619-94be-5c9f8eb73fc2" Sep 13 20:15:26.229924 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:15:26.229899 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-fw949" podUID="a99d1dc1-43a2-4ada-99dd-a4a05c5faf68" Sep 13 20:15:26.916101 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:15:26.916006 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:15:26.916263 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:15:26.916013 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fw949" Sep 13 20:15:26.916263 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:15:26.916025 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6lr4c" Sep 13 20:15:27.326721 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:15:27.326681 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-95zg2" podUID="32f68c96-97d3-4904-a675-45f93df87572" Sep 13 20:15:31.116330 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:15:31.116288 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-6lr4c\" (UID: \"ba047ed2-663d-47be-bd5f-08548fdcdec9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6lr4c" Sep 13 20:15:31.116757 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:15:31.116354 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls\") pod \"dns-default-fw949\" (UID: \"a99d1dc1-43a2-4ada-99dd-a4a05c5faf68\") " pod="openshift-dns/dns-default-fw949" Sep 13 20:15:31.116757 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:15:31.116379 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:15:31.116757 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:15:31.116447 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 13 20:15:31.116757 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:15:31.116467 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 13 20:15:31.116757 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:15:31.116471 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 13 20:15:31.116757 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:15:31.116532 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9866c996d-vpm54: secret "image-registry-tls" not found Sep 13 20:15:31.116757 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:15:31.116522 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert podName:ba047ed2-663d-47be-bd5f-08548fdcdec9 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:33.116505444 +0000 UTC m=+285.292941457 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-6lr4c" (UID: "ba047ed2-663d-47be-bd5f-08548fdcdec9") : secret "networking-console-plugin-cert" not found Sep 13 20:15:31.116757 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:15:31.116567 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls podName:a99d1dc1-43a2-4ada-99dd-a4a05c5faf68 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:33.11655412 +0000 UTC m=+285.292990124 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls") pod "dns-default-fw949" (UID: "a99d1dc1-43a2-4ada-99dd-a4a05c5faf68") : secret "dns-default-metrics-tls" not found Sep 13 20:15:31.116757 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:15:31.116582 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls podName:fd0e27d4-cbed-4619-94be-5c9f8eb73fc2 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:33.116574635 +0000 UTC m=+285.293010657 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls") pod "image-registry-9866c996d-vpm54" (UID: "fd0e27d4-cbed-4619-94be-5c9f8eb73fc2") : secret "image-registry-tls" not found Sep 13 20:15:39.305897 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:15:39.305848 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:17:00.129008 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:00.128961 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs\") pod \"network-metrics-daemon-95zg2\" (UID: \"32f68c96-97d3-4904-a675-45f93df87572\") " pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:17:00.129477 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:00.129114 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 13 20:17:00.129477 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:00.129180 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs podName:32f68c96-97d3-4904-a675-45f93df87572 nodeName:}" failed. No retries permitted until 2026-09-13 20:19:02.129162616 +0000 UTC m=+374.305598631 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs") pod "network-metrics-daemon-95zg2" (UID: "32f68c96-97d3-4904-a675-45f93df87572") : secret "metrics-daemon-secret" not found Sep 13 20:17:08.293509 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.293470 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-f8hfw"] Sep 13 20:17:08.296292 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.296270 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-g7j4k"] Sep 13 20:17:08.296433 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.296396 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-f8hfw" Sep 13 20:17:08.298725 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.298700 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-xkntg\"" Sep 13 20:17:08.299196 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.299182 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-tjppt"] Sep 13 20:17:08.299296 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.299282 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-g7j4k" Sep 13 20:17:08.301316 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.301292 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 13 20:17:08.301443 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.301367 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 13 20:17:08.301443 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.301417 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 13 20:17:08.301654 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.301640 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 13 20:17:08.301926 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.301898 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-4jdsq\"" Sep 13 20:17:08.302066 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.302043 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-pg8lq"] Sep 13 20:17:08.302159 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.302146 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-tjppt" Sep 13 20:17:08.304293 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.304268 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 13 20:17:08.304393 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.304271 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-wf4fn\"" Sep 13 20:17:08.304684 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.304667 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 13 20:17:08.304746 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.304689 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-655c4b696-bgtsn"] Sep 13 20:17:08.304746 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.304741 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-pg8lq" Sep 13 20:17:08.307293 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.307274 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 13 20:17:08.307583 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.307564 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 13 20:17:08.307659 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.307614 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 13 20:17:08.307795 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.307777 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:08.308716 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.308691 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 13 20:17:08.308836 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.308704 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-rjfgk\"" Sep 13 20:17:08.310092 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.310073 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 13 20:17:08.310234 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.310132 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 13 20:17:08.310695 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.310607 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 13 20:17:08.310891 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.310812 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 13 20:17:08.311462 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.311064 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-lvkhz\"" Sep 13 20:17:08.311462 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.311238 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 13 20:17:08.311462 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.311251 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 13 20:17:08.313761 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.313739 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-f8hfw"] Sep 13 20:17:08.313857 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.313779 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-g7j4k"] Sep 13 20:17:08.313857 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.313802 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-pg8lq"] Sep 13 20:17:08.315793 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.315777 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-655c4b696-bgtsn"] Sep 13 20:17:08.319264 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.319244 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-tjppt"] Sep 13 20:17:08.380290 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.380261 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/28166755-2fd2-4b23-ac8a-55a8121e0b00-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-pg8lq\" (UID: \"28166755-2fd2-4b23-ac8a-55a8121e0b00\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-pg8lq" Sep 13 20:17:08.380290 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.380293 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-metrics-certs\") pod \"router-default-655c4b696-bgtsn\" (UID: \"ff4cd6d7-3b88-4122-9d4e-d1560d389a70\") " pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:08.380551 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.380311 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ee7be3e3-c579-4378-9a12-94b54e80d33e-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-g7j4k\" (UID: \"ee7be3e3-c579-4378-9a12-94b54e80d33e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-g7j4k" Sep 13 20:17:08.380551 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.380332 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ee7be3e3-c579-4378-9a12-94b54e80d33e-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-g7j4k\" (UID: \"ee7be3e3-c579-4378-9a12-94b54e80d33e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-g7j4k" Sep 13 20:17:08.380551 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.380457 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-stats-auth\") pod \"router-default-655c4b696-bgtsn\" (UID: \"ff4cd6d7-3b88-4122-9d4e-d1560d389a70\") " pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:08.380551 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.380521 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6lr6\" (UniqueName: \"kubernetes.io/projected/ee7be3e3-c579-4378-9a12-94b54e80d33e-kube-api-access-z6lr6\") pod \"kube-storage-version-migrator-operator-6c8f84f897-g7j4k\" (UID: \"ee7be3e3-c579-4378-9a12-94b54e80d33e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-g7j4k" Sep 13 20:17:08.380551 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.380549 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ndbz\" (UniqueName: \"kubernetes.io/projected/28166755-2fd2-4b23-ac8a-55a8121e0b00-kube-api-access-8ndbz\") pod \"cluster-monitoring-operator-6cd954b897-pg8lq\" (UID: \"28166755-2fd2-4b23-ac8a-55a8121e0b00\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-pg8lq" Sep 13 20:17:08.380792 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.380580 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsrqg\" (UniqueName: \"kubernetes.io/projected/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-kube-api-access-hsrqg\") pod \"router-default-655c4b696-bgtsn\" (UID: \"ff4cd6d7-3b88-4122-9d4e-d1560d389a70\") " pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:08.380792 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.380620 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kzj9\" (UniqueName: \"kubernetes.io/projected/72b2c696-47fc-4e4a-9d33-0cf58a016928-kube-api-access-7kzj9\") pod \"network-check-source-6f8b768455-f8hfw\" (UID: \"72b2c696-47fc-4e4a-9d33-0cf58a016928\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-f8hfw" Sep 13 20:17:08.380792 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.380667 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/28166755-2fd2-4b23-ac8a-55a8121e0b00-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-pg8lq\" (UID: \"28166755-2fd2-4b23-ac8a-55a8121e0b00\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-pg8lq" Sep 13 20:17:08.380792 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.380718 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfhm8\" (UniqueName: \"kubernetes.io/projected/7c028e5d-3eb9-4423-9201-4aeaefc67d49-kube-api-access-pfhm8\") pod \"volume-data-source-validator-86695b7cc7-tjppt\" (UID: \"7c028e5d-3eb9-4423-9201-4aeaefc67d49\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-tjppt" Sep 13 20:17:08.380792 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.380770 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-service-ca-bundle\") pod \"router-default-655c4b696-bgtsn\" (UID: \"ff4cd6d7-3b88-4122-9d4e-d1560d389a70\") " pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:08.380946 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.380797 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-default-certificate\") pod \"router-default-655c4b696-bgtsn\" (UID: \"ff4cd6d7-3b88-4122-9d4e-d1560d389a70\") " pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:08.481106 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.481077 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-stats-auth\") pod \"router-default-655c4b696-bgtsn\" (UID: \"ff4cd6d7-3b88-4122-9d4e-d1560d389a70\") " pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:08.481295 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.481117 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6lr6\" (UniqueName: \"kubernetes.io/projected/ee7be3e3-c579-4378-9a12-94b54e80d33e-kube-api-access-z6lr6\") pod \"kube-storage-version-migrator-operator-6c8f84f897-g7j4k\" (UID: \"ee7be3e3-c579-4378-9a12-94b54e80d33e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-g7j4k" Sep 13 20:17:08.481295 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.481142 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ndbz\" (UniqueName: \"kubernetes.io/projected/28166755-2fd2-4b23-ac8a-55a8121e0b00-kube-api-access-8ndbz\") pod \"cluster-monitoring-operator-6cd954b897-pg8lq\" (UID: \"28166755-2fd2-4b23-ac8a-55a8121e0b00\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-pg8lq" Sep 13 20:17:08.481295 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.481167 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsrqg\" (UniqueName: \"kubernetes.io/projected/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-kube-api-access-hsrqg\") pod \"router-default-655c4b696-bgtsn\" (UID: \"ff4cd6d7-3b88-4122-9d4e-d1560d389a70\") " pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:08.481295 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.481194 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kzj9\" (UniqueName: \"kubernetes.io/projected/72b2c696-47fc-4e4a-9d33-0cf58a016928-kube-api-access-7kzj9\") pod \"network-check-source-6f8b768455-f8hfw\" (UID: \"72b2c696-47fc-4e4a-9d33-0cf58a016928\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-f8hfw" Sep 13 20:17:08.481295 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.481234 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/28166755-2fd2-4b23-ac8a-55a8121e0b00-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-pg8lq\" (UID: \"28166755-2fd2-4b23-ac8a-55a8121e0b00\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-pg8lq" Sep 13 20:17:08.481295 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.481281 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfhm8\" (UniqueName: \"kubernetes.io/projected/7c028e5d-3eb9-4423-9201-4aeaefc67d49-kube-api-access-pfhm8\") pod \"volume-data-source-validator-86695b7cc7-tjppt\" (UID: \"7c028e5d-3eb9-4423-9201-4aeaefc67d49\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-tjppt" Sep 13 20:17:08.481637 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.481313 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-service-ca-bundle\") pod \"router-default-655c4b696-bgtsn\" (UID: \"ff4cd6d7-3b88-4122-9d4e-d1560d389a70\") " pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:08.481637 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.481340 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-default-certificate\") pod \"router-default-655c4b696-bgtsn\" (UID: \"ff4cd6d7-3b88-4122-9d4e-d1560d389a70\") " pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:08.481637 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.481368 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/28166755-2fd2-4b23-ac8a-55a8121e0b00-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-pg8lq\" (UID: \"28166755-2fd2-4b23-ac8a-55a8121e0b00\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-pg8lq" Sep 13 20:17:08.481637 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.481396 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-metrics-certs\") pod \"router-default-655c4b696-bgtsn\" (UID: \"ff4cd6d7-3b88-4122-9d4e-d1560d389a70\") " pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:08.481637 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.481421 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ee7be3e3-c579-4378-9a12-94b54e80d33e-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-g7j4k\" (UID: \"ee7be3e3-c579-4378-9a12-94b54e80d33e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-g7j4k" Sep 13 20:17:08.481637 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.481460 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ee7be3e3-c579-4378-9a12-94b54e80d33e-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-g7j4k\" (UID: \"ee7be3e3-c579-4378-9a12-94b54e80d33e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-g7j4k" Sep 13 20:17:08.481637 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:08.481476 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 13 20:17:08.481637 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:08.481558 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 13 20:17:08.481637 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:08.481570 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28166755-2fd2-4b23-ac8a-55a8121e0b00-cluster-monitoring-operator-tls podName:28166755-2fd2-4b23-ac8a-55a8121e0b00 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:08.981548676 +0000 UTC m=+261.157984690 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/28166755-2fd2-4b23-ac8a-55a8121e0b00-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-pg8lq" (UID: "28166755-2fd2-4b23-ac8a-55a8121e0b00") : secret "cluster-monitoring-operator-tls" not found Sep 13 20:17:08.481637 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:08.481636 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-metrics-certs podName:ff4cd6d7-3b88-4122-9d4e-d1560d389a70 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:08.981618454 +0000 UTC m=+261.158054454 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-metrics-certs") pod "router-default-655c4b696-bgtsn" (UID: "ff4cd6d7-3b88-4122-9d4e-d1560d389a70") : secret "router-metrics-certs-default" not found Sep 13 20:17:08.482138 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:08.481741 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-service-ca-bundle podName:ff4cd6d7-3b88-4122-9d4e-d1560d389a70 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:08.981717542 +0000 UTC m=+261.158153566 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-service-ca-bundle") pod "router-default-655c4b696-bgtsn" (UID: "ff4cd6d7-3b88-4122-9d4e-d1560d389a70") : configmap references non-existent config key: service-ca.crt Sep 13 20:17:08.482138 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.482048 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/28166755-2fd2-4b23-ac8a-55a8121e0b00-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-pg8lq\" (UID: \"28166755-2fd2-4b23-ac8a-55a8121e0b00\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-pg8lq" Sep 13 20:17:08.482245 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.482157 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ee7be3e3-c579-4378-9a12-94b54e80d33e-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-g7j4k\" (UID: \"ee7be3e3-c579-4378-9a12-94b54e80d33e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-g7j4k" Sep 13 20:17:08.483735 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.483707 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-stats-auth\") pod \"router-default-655c4b696-bgtsn\" (UID: \"ff4cd6d7-3b88-4122-9d4e-d1560d389a70\") " pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:08.483735 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.483714 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ee7be3e3-c579-4378-9a12-94b54e80d33e-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-g7j4k\" (UID: \"ee7be3e3-c579-4378-9a12-94b54e80d33e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-g7j4k" Sep 13 20:17:08.483896 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.483804 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-default-certificate\") pod \"router-default-655c4b696-bgtsn\" (UID: \"ff4cd6d7-3b88-4122-9d4e-d1560d389a70\") " pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:08.489490 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.489463 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsrqg\" (UniqueName: \"kubernetes.io/projected/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-kube-api-access-hsrqg\") pod \"router-default-655c4b696-bgtsn\" (UID: \"ff4cd6d7-3b88-4122-9d4e-d1560d389a70\") " pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:08.489755 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.489732 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6lr6\" (UniqueName: \"kubernetes.io/projected/ee7be3e3-c579-4378-9a12-94b54e80d33e-kube-api-access-z6lr6\") pod \"kube-storage-version-migrator-operator-6c8f84f897-g7j4k\" (UID: \"ee7be3e3-c579-4378-9a12-94b54e80d33e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-g7j4k" Sep 13 20:17:08.490027 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.490001 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ndbz\" (UniqueName: \"kubernetes.io/projected/28166755-2fd2-4b23-ac8a-55a8121e0b00-kube-api-access-8ndbz\") pod \"cluster-monitoring-operator-6cd954b897-pg8lq\" (UID: \"28166755-2fd2-4b23-ac8a-55a8121e0b00\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-pg8lq" Sep 13 20:17:08.490115 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.490009 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kzj9\" (UniqueName: \"kubernetes.io/projected/72b2c696-47fc-4e4a-9d33-0cf58a016928-kube-api-access-7kzj9\") pod \"network-check-source-6f8b768455-f8hfw\" (UID: \"72b2c696-47fc-4e4a-9d33-0cf58a016928\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-f8hfw" Sep 13 20:17:08.490315 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.490299 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfhm8\" (UniqueName: \"kubernetes.io/projected/7c028e5d-3eb9-4423-9201-4aeaefc67d49-kube-api-access-pfhm8\") pod \"volume-data-source-validator-86695b7cc7-tjppt\" (UID: \"7c028e5d-3eb9-4423-9201-4aeaefc67d49\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-tjppt" Sep 13 20:17:08.606726 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.606624 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-f8hfw" Sep 13 20:17:08.614912 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.614784 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-g7j4k" Sep 13 20:17:08.622590 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.622565 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-tjppt" Sep 13 20:17:08.880218 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.880173 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-f8hfw"] Sep 13 20:17:08.882596 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:17:08.882572 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72b2c696_47fc_4e4a_9d33_0cf58a016928.slice/crio-265f4f3d00fdd3a31f2f29e42ca4dd8fe887af1ae51f1851fbdd46b6df174806 WatchSource:0}: Error finding container 265f4f3d00fdd3a31f2f29e42ca4dd8fe887af1ae51f1851fbdd46b6df174806: Status 404 returned error can't find the container with id 265f4f3d00fdd3a31f2f29e42ca4dd8fe887af1ae51f1851fbdd46b6df174806 Sep 13 20:17:08.885174 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.885148 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-g7j4k"] Sep 13 20:17:08.887417 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:17:08.887394 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee7be3e3_c579_4378_9a12_94b54e80d33e.slice/crio-622a7207751f72785d4723c75465f4efbd992a4c108231c11cfde2a815fce5a4 WatchSource:0}: Error finding container 622a7207751f72785d4723c75465f4efbd992a4c108231c11cfde2a815fce5a4: Status 404 returned error can't find the container with id 622a7207751f72785d4723c75465f4efbd992a4c108231c11cfde2a815fce5a4 Sep 13 20:17:08.902795 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.902773 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-tjppt"] Sep 13 20:17:08.920556 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:17:08.920528 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c028e5d_3eb9_4423_9201_4aeaefc67d49.slice/crio-742505c5b946e275743661d585b2aa22ce41aed15322f06b1fc9d44bacbc0b3b WatchSource:0}: Error finding container 742505c5b946e275743661d585b2aa22ce41aed15322f06b1fc9d44bacbc0b3b: Status 404 returned error can't find the container with id 742505c5b946e275743661d585b2aa22ce41aed15322f06b1fc9d44bacbc0b3b Sep 13 20:17:08.985233 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.985196 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-service-ca-bundle\") pod \"router-default-655c4b696-bgtsn\" (UID: \"ff4cd6d7-3b88-4122-9d4e-d1560d389a70\") " pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:08.985387 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.985237 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/28166755-2fd2-4b23-ac8a-55a8121e0b00-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-pg8lq\" (UID: \"28166755-2fd2-4b23-ac8a-55a8121e0b00\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-pg8lq" Sep 13 20:17:08.985387 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:08.985257 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-metrics-certs\") pod \"router-default-655c4b696-bgtsn\" (UID: \"ff4cd6d7-3b88-4122-9d4e-d1560d389a70\") " pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:08.985387 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:08.985353 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 13 20:17:08.985387 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:08.985360 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-service-ca-bundle podName:ff4cd6d7-3b88-4122-9d4e-d1560d389a70 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:09.985339905 +0000 UTC m=+262.161775911 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-service-ca-bundle") pod "router-default-655c4b696-bgtsn" (UID: "ff4cd6d7-3b88-4122-9d4e-d1560d389a70") : configmap references non-existent config key: service-ca.crt Sep 13 20:17:08.985387 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:08.985360 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 13 20:17:08.985677 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:08.985394 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-metrics-certs podName:ff4cd6d7-3b88-4122-9d4e-d1560d389a70 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:09.985382795 +0000 UTC m=+262.161818795 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-metrics-certs") pod "router-default-655c4b696-bgtsn" (UID: "ff4cd6d7-3b88-4122-9d4e-d1560d389a70") : secret "router-metrics-certs-default" not found Sep 13 20:17:08.985677 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:08.985408 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28166755-2fd2-4b23-ac8a-55a8121e0b00-cluster-monitoring-operator-tls podName:28166755-2fd2-4b23-ac8a-55a8121e0b00 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:09.985398662 +0000 UTC m=+262.161834671 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/28166755-2fd2-4b23-ac8a-55a8121e0b00-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-pg8lq" (UID: "28166755-2fd2-4b23-ac8a-55a8121e0b00") : secret "cluster-monitoring-operator-tls" not found Sep 13 20:17:09.154834 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:09.154740 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-g7j4k" event={"ID":"ee7be3e3-c579-4378-9a12-94b54e80d33e","Type":"ContainerStarted","Data":"622a7207751f72785d4723c75465f4efbd992a4c108231c11cfde2a815fce5a4"} Sep 13 20:17:09.155905 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:09.155877 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-tjppt" event={"ID":"7c028e5d-3eb9-4423-9201-4aeaefc67d49","Type":"ContainerStarted","Data":"742505c5b946e275743661d585b2aa22ce41aed15322f06b1fc9d44bacbc0b3b"} Sep 13 20:17:09.157313 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:09.157286 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-f8hfw" event={"ID":"72b2c696-47fc-4e4a-9d33-0cf58a016928","Type":"ContainerStarted","Data":"fc14d81260ca5ef67ae24f49e1a6731d8e8e231feb960a7a86ab87f09cfc0b15"} Sep 13 20:17:09.157404 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:09.157322 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-f8hfw" event={"ID":"72b2c696-47fc-4e4a-9d33-0cf58a016928","Type":"ContainerStarted","Data":"265f4f3d00fdd3a31f2f29e42ca4dd8fe887af1ae51f1851fbdd46b6df174806"} Sep 13 20:17:09.170748 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:09.170708 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-f8hfw" podStartSLOduration=1.170696056 podStartE2EDuration="1.170696056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:17:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:17:09.169760351 +0000 UTC m=+261.346196375" watchObservedRunningTime="2026-09-13 20:17:09.170696056 +0000 UTC m=+261.347132077" Sep 13 20:17:09.993223 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:09.993182 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-service-ca-bundle\") pod \"router-default-655c4b696-bgtsn\" (UID: \"ff4cd6d7-3b88-4122-9d4e-d1560d389a70\") " pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:09.993701 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:09.993228 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/28166755-2fd2-4b23-ac8a-55a8121e0b00-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-pg8lq\" (UID: \"28166755-2fd2-4b23-ac8a-55a8121e0b00\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-pg8lq" Sep 13 20:17:09.993701 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:09.993261 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-metrics-certs\") pod \"router-default-655c4b696-bgtsn\" (UID: \"ff4cd6d7-3b88-4122-9d4e-d1560d389a70\") " pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:09.993701 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:09.993351 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-service-ca-bundle podName:ff4cd6d7-3b88-4122-9d4e-d1560d389a70 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:11.993331842 +0000 UTC m=+264.169767858 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-service-ca-bundle") pod "router-default-655c4b696-bgtsn" (UID: "ff4cd6d7-3b88-4122-9d4e-d1560d389a70") : configmap references non-existent config key: service-ca.crt Sep 13 20:17:09.993701 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:09.993385 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 13 20:17:09.993701 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:09.993422 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-metrics-certs podName:ff4cd6d7-3b88-4122-9d4e-d1560d389a70 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:11.993411687 +0000 UTC m=+264.169847700 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-metrics-certs") pod "router-default-655c4b696-bgtsn" (UID: "ff4cd6d7-3b88-4122-9d4e-d1560d389a70") : secret "router-metrics-certs-default" not found Sep 13 20:17:09.993701 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:09.993475 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 13 20:17:09.993701 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:09.993521 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28166755-2fd2-4b23-ac8a-55a8121e0b00-cluster-monitoring-operator-tls podName:28166755-2fd2-4b23-ac8a-55a8121e0b00 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:11.993510504 +0000 UTC m=+264.169946504 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/28166755-2fd2-4b23-ac8a-55a8121e0b00-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-pg8lq" (UID: "28166755-2fd2-4b23-ac8a-55a8121e0b00") : secret "cluster-monitoring-operator-tls" not found Sep 13 20:17:11.165405 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:11.165315 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-g7j4k" event={"ID":"ee7be3e3-c579-4378-9a12-94b54e80d33e","Type":"ContainerStarted","Data":"dd70cc33718895a60840c751c71458ba3c793a0f4e7d3fc7e100c8485fefd174"} Sep 13 20:17:11.166906 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:11.166880 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-tjppt" event={"ID":"7c028e5d-3eb9-4423-9201-4aeaefc67d49","Type":"ContainerStarted","Data":"2bb6edae4821c0bf62477123296ae3226aa9a731f0440d0f9e8248b8a27074f2"} Sep 13 20:17:11.179739 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:11.179694 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-g7j4k" podStartSLOduration=1.238747342 podStartE2EDuration="3.17967968s" totalImagesPullingTime="1.940932338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:17:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:17:11.17845455 +0000 UTC m=+263.354890572" watchObservedRunningTime="2026-09-13 20:17:11.17967968 +0000 UTC m=+263.356115702" Sep 13 20:17:11.191942 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:11.191901 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-tjppt" podStartSLOduration=1.938254593 podStartE2EDuration="3.191887539s" totalImagesPullingTime="1.253632946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:17:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:17:11.191275084 +0000 UTC m=+263.367711105" watchObservedRunningTime="2026-09-13 20:17:11.191887539 +0000 UTC m=+263.368323561" Sep 13 20:17:11.769042 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:11.769008 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-p72zr"] Sep 13 20:17:11.772361 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:11.772340 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-p72zr" Sep 13 20:17:11.774460 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:11.774433 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Sep 13 20:17:11.774587 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:11.774471 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-qwx6v\"" Sep 13 20:17:11.774587 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:11.774444 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Sep 13 20:17:11.779273 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:11.779252 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-p72zr"] Sep 13 20:17:11.808810 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:11.808779 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g6sq\" (UniqueName: \"kubernetes.io/projected/ec2b9095-1ce0-4b18-8ab0-ef6491e838cb-kube-api-access-7g6sq\") pod \"migrator-8b4d697c5-p72zr\" (UID: \"ec2b9095-1ce0-4b18-8ab0-ef6491e838cb\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-p72zr" Sep 13 20:17:11.909880 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:11.909843 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g6sq\" (UniqueName: \"kubernetes.io/projected/ec2b9095-1ce0-4b18-8ab0-ef6491e838cb-kube-api-access-7g6sq\") pod \"migrator-8b4d697c5-p72zr\" (UID: \"ec2b9095-1ce0-4b18-8ab0-ef6491e838cb\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-p72zr" Sep 13 20:17:11.917017 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:11.916995 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g6sq\" (UniqueName: \"kubernetes.io/projected/ec2b9095-1ce0-4b18-8ab0-ef6491e838cb-kube-api-access-7g6sq\") pod \"migrator-8b4d697c5-p72zr\" (UID: \"ec2b9095-1ce0-4b18-8ab0-ef6491e838cb\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-p72zr" Sep 13 20:17:12.010635 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:12.010602 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-service-ca-bundle\") pod \"router-default-655c4b696-bgtsn\" (UID: \"ff4cd6d7-3b88-4122-9d4e-d1560d389a70\") " pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:12.010635 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:12.010639 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/28166755-2fd2-4b23-ac8a-55a8121e0b00-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-pg8lq\" (UID: \"28166755-2fd2-4b23-ac8a-55a8121e0b00\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-pg8lq" Sep 13 20:17:12.010827 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:12.010661 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-metrics-certs\") pod \"router-default-655c4b696-bgtsn\" (UID: \"ff4cd6d7-3b88-4122-9d4e-d1560d389a70\") " pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:12.010827 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:12.010751 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 13 20:17:12.010827 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:12.010759 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 13 20:17:12.010827 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:12.010796 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-metrics-certs podName:ff4cd6d7-3b88-4122-9d4e-d1560d389a70 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:16.010782062 +0000 UTC m=+268.187218062 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-metrics-certs") pod "router-default-655c4b696-bgtsn" (UID: "ff4cd6d7-3b88-4122-9d4e-d1560d389a70") : secret "router-metrics-certs-default" not found Sep 13 20:17:12.010827 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:12.010816 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28166755-2fd2-4b23-ac8a-55a8121e0b00-cluster-monitoring-operator-tls podName:28166755-2fd2-4b23-ac8a-55a8121e0b00 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:16.010802587 +0000 UTC m=+268.187238591 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/28166755-2fd2-4b23-ac8a-55a8121e0b00-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-pg8lq" (UID: "28166755-2fd2-4b23-ac8a-55a8121e0b00") : secret "cluster-monitoring-operator-tls" not found Sep 13 20:17:12.010980 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:12.010835 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-service-ca-bundle podName:ff4cd6d7-3b88-4122-9d4e-d1560d389a70 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:16.010825042 +0000 UTC m=+268.187261044 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-service-ca-bundle") pod "router-default-655c4b696-bgtsn" (UID: "ff4cd6d7-3b88-4122-9d4e-d1560d389a70") : configmap references non-existent config key: service-ca.crt Sep 13 20:17:12.081476 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:12.081444 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-p72zr" Sep 13 20:17:12.344207 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:12.344091 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-p72zr"] Sep 13 20:17:12.346661 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:17:12.346627 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec2b9095_1ce0_4b18_8ab0_ef6491e838cb.slice/crio-7deede485b9504f7a990c7751c2c1ee95405fd4a64559e99dd3f4dba309ca537 WatchSource:0}: Error finding container 7deede485b9504f7a990c7751c2c1ee95405fd4a64559e99dd3f4dba309ca537: Status 404 returned error can't find the container with id 7deede485b9504f7a990c7751c2c1ee95405fd4a64559e99dd3f4dba309ca537 Sep 13 20:17:13.175523 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:13.175472 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-p72zr" event={"ID":"ec2b9095-1ce0-4b18-8ab0-ef6491e838cb","Type":"ContainerStarted","Data":"7deede485b9504f7a990c7751c2c1ee95405fd4a64559e99dd3f4dba309ca537"} Sep 13 20:17:13.582238 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:13.582203 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lnffs"] Sep 13 20:17:13.585901 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:13.585880 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lnffs" Sep 13 20:17:13.588128 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:13.588110 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 13 20:17:13.588219 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:13.588110 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 13 20:17:13.589033 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:13.589018 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 13 20:17:13.589114 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:13.589026 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wx957\"" Sep 13 20:17:13.589114 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:13.589049 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 13 20:17:13.595849 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:13.595828 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lnffs"] Sep 13 20:17:13.623224 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:13.623199 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/260a45fa-ca24-4f0f-8248-34f44c75e581-crio-socket\") pod \"insights-runtime-extractor-lnffs\" (UID: \"260a45fa-ca24-4f0f-8248-34f44c75e581\") " pod="openshift-insights/insights-runtime-extractor-lnffs" Sep 13 20:17:13.623328 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:13.623274 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/260a45fa-ca24-4f0f-8248-34f44c75e581-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lnffs\" (UID: \"260a45fa-ca24-4f0f-8248-34f44c75e581\") " pod="openshift-insights/insights-runtime-extractor-lnffs" Sep 13 20:17:13.623328 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:13.623313 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/260a45fa-ca24-4f0f-8248-34f44c75e581-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lnffs\" (UID: \"260a45fa-ca24-4f0f-8248-34f44c75e581\") " pod="openshift-insights/insights-runtime-extractor-lnffs" Sep 13 20:17:13.623439 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:13.623399 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k7xh\" (UniqueName: \"kubernetes.io/projected/260a45fa-ca24-4f0f-8248-34f44c75e581-kube-api-access-9k7xh\") pod \"insights-runtime-extractor-lnffs\" (UID: \"260a45fa-ca24-4f0f-8248-34f44c75e581\") " pod="openshift-insights/insights-runtime-extractor-lnffs" Sep 13 20:17:13.623515 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:13.623453 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/260a45fa-ca24-4f0f-8248-34f44c75e581-data-volume\") pod \"insights-runtime-extractor-lnffs\" (UID: \"260a45fa-ca24-4f0f-8248-34f44c75e581\") " pod="openshift-insights/insights-runtime-extractor-lnffs" Sep 13 20:17:13.724421 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:13.724396 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/260a45fa-ca24-4f0f-8248-34f44c75e581-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lnffs\" (UID: \"260a45fa-ca24-4f0f-8248-34f44c75e581\") " pod="openshift-insights/insights-runtime-extractor-lnffs" Sep 13 20:17:13.724565 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:13.724440 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/260a45fa-ca24-4f0f-8248-34f44c75e581-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lnffs\" (UID: \"260a45fa-ca24-4f0f-8248-34f44c75e581\") " pod="openshift-insights/insights-runtime-extractor-lnffs" Sep 13 20:17:13.724565 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:13.724545 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 13 20:17:13.724686 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:13.724564 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k7xh\" (UniqueName: \"kubernetes.io/projected/260a45fa-ca24-4f0f-8248-34f44c75e581-kube-api-access-9k7xh\") pod \"insights-runtime-extractor-lnffs\" (UID: \"260a45fa-ca24-4f0f-8248-34f44c75e581\") " pod="openshift-insights/insights-runtime-extractor-lnffs" Sep 13 20:17:13.724686 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:13.724592 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/260a45fa-ca24-4f0f-8248-34f44c75e581-data-volume\") pod \"insights-runtime-extractor-lnffs\" (UID: \"260a45fa-ca24-4f0f-8248-34f44c75e581\") " pod="openshift-insights/insights-runtime-extractor-lnffs" Sep 13 20:17:13.724686 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:13.724613 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/260a45fa-ca24-4f0f-8248-34f44c75e581-kube-rbac-proxy-cm podName:260a45fa-ca24-4f0f-8248-34f44c75e581 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:14.224590183 +0000 UTC m=+266.401026199 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/260a45fa-ca24-4f0f-8248-34f44c75e581-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lnffs" (UID: "260a45fa-ca24-4f0f-8248-34f44c75e581") : configmap "kube-rbac-proxy" not found Sep 13 20:17:13.724686 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:13.724553 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 13 20:17:13.724859 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:13.724692 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/260a45fa-ca24-4f0f-8248-34f44c75e581-crio-socket\") pod \"insights-runtime-extractor-lnffs\" (UID: \"260a45fa-ca24-4f0f-8248-34f44c75e581\") " pod="openshift-insights/insights-runtime-extractor-lnffs" Sep 13 20:17:13.724859 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:13.724704 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/260a45fa-ca24-4f0f-8248-34f44c75e581-insights-runtime-extractor-tls podName:260a45fa-ca24-4f0f-8248-34f44c75e581 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:14.224686282 +0000 UTC m=+266.401122301 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/260a45fa-ca24-4f0f-8248-34f44c75e581-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lnffs" (UID: "260a45fa-ca24-4f0f-8248-34f44c75e581") : secret "insights-runtime-extractor-tls" not found Sep 13 20:17:13.724859 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:13.724753 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/260a45fa-ca24-4f0f-8248-34f44c75e581-crio-socket\") pod \"insights-runtime-extractor-lnffs\" (UID: \"260a45fa-ca24-4f0f-8248-34f44c75e581\") " pod="openshift-insights/insights-runtime-extractor-lnffs" Sep 13 20:17:13.725414 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:13.725398 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/260a45fa-ca24-4f0f-8248-34f44c75e581-data-volume\") pod \"insights-runtime-extractor-lnffs\" (UID: \"260a45fa-ca24-4f0f-8248-34f44c75e581\") " pod="openshift-insights/insights-runtime-extractor-lnffs" Sep 13 20:17:13.733777 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:13.733752 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k7xh\" (UniqueName: \"kubernetes.io/projected/260a45fa-ca24-4f0f-8248-34f44c75e581-kube-api-access-9k7xh\") pod \"insights-runtime-extractor-lnffs\" (UID: \"260a45fa-ca24-4f0f-8248-34f44c75e581\") " pod="openshift-insights/insights-runtime-extractor-lnffs" Sep 13 20:17:14.180004 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:14.179969 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-p72zr" event={"ID":"ec2b9095-1ce0-4b18-8ab0-ef6491e838cb","Type":"ContainerStarted","Data":"4c75d3ac0af09f5a07623957b971b5e63d46a280ae8b22fcc605c4dedfad29d3"} Sep 13 20:17:14.180168 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:14.180019 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-p72zr" event={"ID":"ec2b9095-1ce0-4b18-8ab0-ef6491e838cb","Type":"ContainerStarted","Data":"476e8bed42788ad3dce2dc27d5a98b102925a87b78f32c082aeb43067ab20171"} Sep 13 20:17:14.205727 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:14.205681 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-p72zr" podStartSLOduration=1.964659392 podStartE2EDuration="3.205667314s" totalImagesPullingTime="1.241007922s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:17:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:17:14.2043081 +0000 UTC m=+266.380744125" watchObservedRunningTime="2026-09-13 20:17:14.205667314 +0000 UTC m=+266.382103336" Sep 13 20:17:14.227649 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:14.227623 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/260a45fa-ca24-4f0f-8248-34f44c75e581-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lnffs\" (UID: \"260a45fa-ca24-4f0f-8248-34f44c75e581\") " pod="openshift-insights/insights-runtime-extractor-lnffs" Sep 13 20:17:14.227768 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:14.227668 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/260a45fa-ca24-4f0f-8248-34f44c75e581-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lnffs\" (UID: \"260a45fa-ca24-4f0f-8248-34f44c75e581\") " pod="openshift-insights/insights-runtime-extractor-lnffs" Sep 13 20:17:14.227768 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:14.227727 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 13 20:17:14.227837 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:14.227786 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/260a45fa-ca24-4f0f-8248-34f44c75e581-kube-rbac-proxy-cm podName:260a45fa-ca24-4f0f-8248-34f44c75e581 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:15.227772681 +0000 UTC m=+267.404208682 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/260a45fa-ca24-4f0f-8248-34f44c75e581-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lnffs" (UID: "260a45fa-ca24-4f0f-8248-34f44c75e581") : configmap "kube-rbac-proxy" not found Sep 13 20:17:14.227837 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:14.227804 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 13 20:17:14.227909 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:14.227845 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/260a45fa-ca24-4f0f-8248-34f44c75e581-insights-runtime-extractor-tls podName:260a45fa-ca24-4f0f-8248-34f44c75e581 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:15.227830554 +0000 UTC m=+267.404266555 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/260a45fa-ca24-4f0f-8248-34f44c75e581-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lnffs" (UID: "260a45fa-ca24-4f0f-8248-34f44c75e581") : secret "insights-runtime-extractor-tls" not found Sep 13 20:17:15.236241 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:15.236203 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/260a45fa-ca24-4f0f-8248-34f44c75e581-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lnffs\" (UID: \"260a45fa-ca24-4f0f-8248-34f44c75e581\") " pod="openshift-insights/insights-runtime-extractor-lnffs" Sep 13 20:17:15.236637 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:15.236254 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/260a45fa-ca24-4f0f-8248-34f44c75e581-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lnffs\" (UID: \"260a45fa-ca24-4f0f-8248-34f44c75e581\") " pod="openshift-insights/insights-runtime-extractor-lnffs" Sep 13 20:17:15.236637 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:15.236310 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 13 20:17:15.236637 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:15.236364 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/260a45fa-ca24-4f0f-8248-34f44c75e581-kube-rbac-proxy-cm podName:260a45fa-ca24-4f0f-8248-34f44c75e581 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:17.236350288 +0000 UTC m=+269.412786287 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/260a45fa-ca24-4f0f-8248-34f44c75e581-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lnffs" (UID: "260a45fa-ca24-4f0f-8248-34f44c75e581") : configmap "kube-rbac-proxy" not found Sep 13 20:17:15.236637 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:15.236426 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 13 20:17:15.236637 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:15.236483 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/260a45fa-ca24-4f0f-8248-34f44c75e581-insights-runtime-extractor-tls podName:260a45fa-ca24-4f0f-8248-34f44c75e581 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:17.236467248 +0000 UTC m=+269.412903248 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/260a45fa-ca24-4f0f-8248-34f44c75e581-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lnffs" (UID: "260a45fa-ca24-4f0f-8248-34f44c75e581") : secret "insights-runtime-extractor-tls" not found Sep 13 20:17:15.777900 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:15.777857 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-mvxgz"] Sep 13 20:17:15.781059 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:15.781043 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-mvxgz" Sep 13 20:17:15.783272 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:15.783249 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 13 20:17:15.783391 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:15.783290 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 13 20:17:15.784251 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:15.784235 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 13 20:17:15.784251 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:15.784244 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-v6qsc\"" Sep 13 20:17:15.784339 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:15.784297 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 13 20:17:15.788011 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:15.787992 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-mvxgz"] Sep 13 20:17:15.841253 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:15.841226 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-542z9\" (UniqueName: \"kubernetes.io/projected/19f17579-8b32-44b3-a917-42baec3e09ac-kube-api-access-542z9\") pod \"service-ca-568869fdc5-mvxgz\" (UID: \"19f17579-8b32-44b3-a917-42baec3e09ac\") " pod="openshift-service-ca/service-ca-568869fdc5-mvxgz" Sep 13 20:17:15.841413 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:15.841341 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/19f17579-8b32-44b3-a917-42baec3e09ac-signing-key\") pod \"service-ca-568869fdc5-mvxgz\" (UID: \"19f17579-8b32-44b3-a917-42baec3e09ac\") " pod="openshift-service-ca/service-ca-568869fdc5-mvxgz" Sep 13 20:17:15.841413 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:15.841379 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/19f17579-8b32-44b3-a917-42baec3e09ac-signing-cabundle\") pod \"service-ca-568869fdc5-mvxgz\" (UID: \"19f17579-8b32-44b3-a917-42baec3e09ac\") " pod="openshift-service-ca/service-ca-568869fdc5-mvxgz" Sep 13 20:17:15.942684 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:15.942650 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-542z9\" (UniqueName: \"kubernetes.io/projected/19f17579-8b32-44b3-a917-42baec3e09ac-kube-api-access-542z9\") pod \"service-ca-568869fdc5-mvxgz\" (UID: \"19f17579-8b32-44b3-a917-42baec3e09ac\") " pod="openshift-service-ca/service-ca-568869fdc5-mvxgz" Sep 13 20:17:15.942827 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:15.942744 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/19f17579-8b32-44b3-a917-42baec3e09ac-signing-key\") pod \"service-ca-568869fdc5-mvxgz\" (UID: \"19f17579-8b32-44b3-a917-42baec3e09ac\") " pod="openshift-service-ca/service-ca-568869fdc5-mvxgz" Sep 13 20:17:15.942827 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:15.942762 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/19f17579-8b32-44b3-a917-42baec3e09ac-signing-cabundle\") pod \"service-ca-568869fdc5-mvxgz\" (UID: \"19f17579-8b32-44b3-a917-42baec3e09ac\") " pod="openshift-service-ca/service-ca-568869fdc5-mvxgz" Sep 13 20:17:15.943366 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:15.943346 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/19f17579-8b32-44b3-a917-42baec3e09ac-signing-cabundle\") pod \"service-ca-568869fdc5-mvxgz\" (UID: \"19f17579-8b32-44b3-a917-42baec3e09ac\") " pod="openshift-service-ca/service-ca-568869fdc5-mvxgz" Sep 13 20:17:15.945216 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:15.945193 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/19f17579-8b32-44b3-a917-42baec3e09ac-signing-key\") pod \"service-ca-568869fdc5-mvxgz\" (UID: \"19f17579-8b32-44b3-a917-42baec3e09ac\") " pod="openshift-service-ca/service-ca-568869fdc5-mvxgz" Sep 13 20:17:15.949543 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:15.949522 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-542z9\" (UniqueName: \"kubernetes.io/projected/19f17579-8b32-44b3-a917-42baec3e09ac-kube-api-access-542z9\") pod \"service-ca-568869fdc5-mvxgz\" (UID: \"19f17579-8b32-44b3-a917-42baec3e09ac\") " pod="openshift-service-ca/service-ca-568869fdc5-mvxgz" Sep 13 20:17:16.043831 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:16.043755 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-service-ca-bundle\") pod \"router-default-655c4b696-bgtsn\" (UID: \"ff4cd6d7-3b88-4122-9d4e-d1560d389a70\") " pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:16.043831 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:16.043799 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/28166755-2fd2-4b23-ac8a-55a8121e0b00-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-pg8lq\" (UID: \"28166755-2fd2-4b23-ac8a-55a8121e0b00\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-pg8lq" Sep 13 20:17:16.043831 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:16.043818 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-metrics-certs\") pod \"router-default-655c4b696-bgtsn\" (UID: \"ff4cd6d7-3b88-4122-9d4e-d1560d389a70\") " pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:16.044065 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:16.043906 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 13 20:17:16.044065 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:16.043904 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 13 20:17:16.044065 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:16.043953 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-service-ca-bundle podName:ff4cd6d7-3b88-4122-9d4e-d1560d389a70 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:24.043930591 +0000 UTC m=+276.220366592 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-service-ca-bundle") pod "router-default-655c4b696-bgtsn" (UID: "ff4cd6d7-3b88-4122-9d4e-d1560d389a70") : configmap references non-existent config key: service-ca.crt Sep 13 20:17:16.044065 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:16.043982 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-metrics-certs podName:ff4cd6d7-3b88-4122-9d4e-d1560d389a70 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:24.043971403 +0000 UTC m=+276.220407411 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-metrics-certs") pod "router-default-655c4b696-bgtsn" (UID: "ff4cd6d7-3b88-4122-9d4e-d1560d389a70") : secret "router-metrics-certs-default" not found Sep 13 20:17:16.044065 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:16.044006 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28166755-2fd2-4b23-ac8a-55a8121e0b00-cluster-monitoring-operator-tls podName:28166755-2fd2-4b23-ac8a-55a8121e0b00 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:24.043997531 +0000 UTC m=+276.220433535 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/28166755-2fd2-4b23-ac8a-55a8121e0b00-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-pg8lq" (UID: "28166755-2fd2-4b23-ac8a-55a8121e0b00") : secret "cluster-monitoring-operator-tls" not found Sep 13 20:17:16.090388 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:16.090361 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-mvxgz" Sep 13 20:17:16.359723 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:16.359613 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-mvxgz"] Sep 13 20:17:16.360390 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:16.360372 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mgvpg_e80af952-0069-4b10-9b75-2bbef4b3a126/dns-node-resolver/0.log" Sep 13 20:17:16.362083 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:17:16.362056 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19f17579_8b32_44b3_a917_42baec3e09ac.slice/crio-772ceddcdce8bf5faf98e3c2ff09d378acffb2f80abbc1ee1384403b1394487a WatchSource:0}: Error finding container 772ceddcdce8bf5faf98e3c2ff09d378acffb2f80abbc1ee1384403b1394487a: Status 404 returned error can't find the container with id 772ceddcdce8bf5faf98e3c2ff09d378acffb2f80abbc1ee1384403b1394487a Sep 13 20:17:17.189814 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:17.189779 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-mvxgz" event={"ID":"19f17579-8b32-44b3-a917-42baec3e09ac","Type":"ContainerStarted","Data":"772ceddcdce8bf5faf98e3c2ff09d378acffb2f80abbc1ee1384403b1394487a"} Sep 13 20:17:17.253630 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:17.253594 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/260a45fa-ca24-4f0f-8248-34f44c75e581-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lnffs\" (UID: \"260a45fa-ca24-4f0f-8248-34f44c75e581\") " pod="openshift-insights/insights-runtime-extractor-lnffs" Sep 13 20:17:17.253771 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:17.253643 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/260a45fa-ca24-4f0f-8248-34f44c75e581-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lnffs\" (UID: \"260a45fa-ca24-4f0f-8248-34f44c75e581\") " pod="openshift-insights/insights-runtime-extractor-lnffs" Sep 13 20:17:17.253771 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:17.253716 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 13 20:17:17.253771 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:17.253733 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 13 20:17:17.253886 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:17.253778 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/260a45fa-ca24-4f0f-8248-34f44c75e581-kube-rbac-proxy-cm podName:260a45fa-ca24-4f0f-8248-34f44c75e581 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:21.253763969 +0000 UTC m=+273.430199969 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/260a45fa-ca24-4f0f-8248-34f44c75e581-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lnffs" (UID: "260a45fa-ca24-4f0f-8248-34f44c75e581") : configmap "kube-rbac-proxy" not found Sep 13 20:17:17.253886 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:17.253794 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/260a45fa-ca24-4f0f-8248-34f44c75e581-insights-runtime-extractor-tls podName:260a45fa-ca24-4f0f-8248-34f44c75e581 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:21.253787713 +0000 UTC m=+273.430223713 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/260a45fa-ca24-4f0f-8248-34f44c75e581-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lnffs" (UID: "260a45fa-ca24-4f0f-8248-34f44c75e581") : secret "insights-runtime-extractor-tls" not found Sep 13 20:17:17.559027 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:17.559000 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xdjrr_2a742e26-18ce-4bcb-8735-a69419ffb81a/node-ca/0.log" Sep 13 20:17:18.159798 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:18.159762 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-k8c5m_5b70f512-38d7-4546-be8f-c40233eabd8f/serve-healthcheck-canary/0.log" Sep 13 20:17:18.194586 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:18.194554 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-mvxgz" event={"ID":"19f17579-8b32-44b3-a917-42baec3e09ac","Type":"ContainerStarted","Data":"45e0985409ec91364524c2e4d1f9e8d619dc645b6cf82bfa7cabc840579cbb39"} Sep 13 20:17:18.210442 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:18.210397 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-mvxgz" podStartSLOduration=1.5446000560000002 podStartE2EDuration="3.210381868s" totalImagesPullingTime="1.665781812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:17:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:17:18.209716529 +0000 UTC m=+270.386152553" watchObservedRunningTime="2026-09-13 20:17:18.210381868 +0000 UTC m=+270.386817912" Sep 13 20:17:18.559803 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:18.559773 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-p72zr_ec2b9095-1ce0-4b18-8ab0-ef6491e838cb/migrator/0.log" Sep 13 20:17:18.758220 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:18.758144 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-p72zr_ec2b9095-1ce0-4b18-8ab0-ef6491e838cb/graceful-termination/0.log" Sep 13 20:17:18.960410 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:18.960379 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-g7j4k_ee7be3e3-c579-4378-9a12-94b54e80d33e/kube-storage-version-migrator-operator/0.log" Sep 13 20:17:21.291056 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:21.291016 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/260a45fa-ca24-4f0f-8248-34f44c75e581-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lnffs\" (UID: \"260a45fa-ca24-4f0f-8248-34f44c75e581\") " pod="openshift-insights/insights-runtime-extractor-lnffs" Sep 13 20:17:21.291523 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:21.291071 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/260a45fa-ca24-4f0f-8248-34f44c75e581-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lnffs\" (UID: \"260a45fa-ca24-4f0f-8248-34f44c75e581\") " pod="openshift-insights/insights-runtime-extractor-lnffs" Sep 13 20:17:21.291523 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:21.291141 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 13 20:17:21.291523 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:21.291213 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/260a45fa-ca24-4f0f-8248-34f44c75e581-kube-rbac-proxy-cm podName:260a45fa-ca24-4f0f-8248-34f44c75e581 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:29.291194256 +0000 UTC m=+281.467630267 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/260a45fa-ca24-4f0f-8248-34f44c75e581-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lnffs" (UID: "260a45fa-ca24-4f0f-8248-34f44c75e581") : configmap "kube-rbac-proxy" not found Sep 13 20:17:21.291523 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:21.291237 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 13 20:17:21.291523 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:21.291291 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/260a45fa-ca24-4f0f-8248-34f44c75e581-insights-runtime-extractor-tls podName:260a45fa-ca24-4f0f-8248-34f44c75e581 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:29.291277371 +0000 UTC m=+281.467713378 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/260a45fa-ca24-4f0f-8248-34f44c75e581-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lnffs" (UID: "260a45fa-ca24-4f0f-8248-34f44c75e581") : secret "insights-runtime-extractor-tls" not found Sep 13 20:17:24.114416 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:24.114383 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-service-ca-bundle\") pod \"router-default-655c4b696-bgtsn\" (UID: \"ff4cd6d7-3b88-4122-9d4e-d1560d389a70\") " pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:24.114900 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:24.114430 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/28166755-2fd2-4b23-ac8a-55a8121e0b00-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-pg8lq\" (UID: \"28166755-2fd2-4b23-ac8a-55a8121e0b00\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-pg8lq" Sep 13 20:17:24.114900 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:24.114448 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-metrics-certs\") pod \"router-default-655c4b696-bgtsn\" (UID: \"ff4cd6d7-3b88-4122-9d4e-d1560d389a70\") " pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:24.114900 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:24.114559 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 13 20:17:24.114900 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:24.114576 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-service-ca-bundle podName:ff4cd6d7-3b88-4122-9d4e-d1560d389a70 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:40.114555423 +0000 UTC m=+292.290991430 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-service-ca-bundle") pod "router-default-655c4b696-bgtsn" (UID: "ff4cd6d7-3b88-4122-9d4e-d1560d389a70") : configmap references non-existent config key: service-ca.crt Sep 13 20:17:24.114900 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:24.114623 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28166755-2fd2-4b23-ac8a-55a8121e0b00-cluster-monitoring-operator-tls podName:28166755-2fd2-4b23-ac8a-55a8121e0b00 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:40.114606602 +0000 UTC m=+292.291042602 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/28166755-2fd2-4b23-ac8a-55a8121e0b00-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-pg8lq" (UID: "28166755-2fd2-4b23-ac8a-55a8121e0b00") : secret "cluster-monitoring-operator-tls" not found Sep 13 20:17:24.116862 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:24.116839 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-metrics-certs\") pod \"router-default-655c4b696-bgtsn\" (UID: \"ff4cd6d7-3b88-4122-9d4e-d1560d389a70\") " pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:29.354903 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:29.354863 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/260a45fa-ca24-4f0f-8248-34f44c75e581-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lnffs\" (UID: \"260a45fa-ca24-4f0f-8248-34f44c75e581\") " pod="openshift-insights/insights-runtime-extractor-lnffs" Sep 13 20:17:29.355369 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:29.354973 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/260a45fa-ca24-4f0f-8248-34f44c75e581-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lnffs\" (UID: \"260a45fa-ca24-4f0f-8248-34f44c75e581\") " pod="openshift-insights/insights-runtime-extractor-lnffs" Sep 13 20:17:29.355439 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:29.355413 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/260a45fa-ca24-4f0f-8248-34f44c75e581-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lnffs\" (UID: \"260a45fa-ca24-4f0f-8248-34f44c75e581\") " pod="openshift-insights/insights-runtime-extractor-lnffs" Sep 13 20:17:29.357374 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:29.357351 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/260a45fa-ca24-4f0f-8248-34f44c75e581-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lnffs\" (UID: \"260a45fa-ca24-4f0f-8248-34f44c75e581\") " pod="openshift-insights/insights-runtime-extractor-lnffs" Sep 13 20:17:29.496284 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:29.496246 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lnffs" Sep 13 20:17:29.766764 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:29.766685 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lnffs"] Sep 13 20:17:29.770265 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:17:29.770238 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod260a45fa_ca24_4f0f_8248_34f44c75e581.slice/crio-c9b4a1a3ebb9f0781fa3193a48997b7820ff9431b930ef23cc286f747c7116c2 WatchSource:0}: Error finding container c9b4a1a3ebb9f0781fa3193a48997b7820ff9431b930ef23cc286f747c7116c2: Status 404 returned error can't find the container with id c9b4a1a3ebb9f0781fa3193a48997b7820ff9431b930ef23cc286f747c7116c2 Sep 13 20:17:29.916805 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:29.916761 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6lr4c" podUID="ba047ed2-663d-47be-bd5f-08548fdcdec9" Sep 13 20:17:29.916805 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:29.916762 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-fw949" podUID="a99d1dc1-43a2-4ada-99dd-a4a05c5faf68" Sep 13 20:17:29.916805 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:29.916761 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-9866c996d-vpm54" podUID="fd0e27d4-cbed-4619-94be-5c9f8eb73fc2" Sep 13 20:17:30.233849 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:30.233816 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:17:30.233849 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:30.233849 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fw949" Sep 13 20:17:30.234140 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:30.233843 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lnffs" event={"ID":"260a45fa-ca24-4f0f-8248-34f44c75e581","Type":"ContainerStarted","Data":"cf942e8a46d24fdfaa03ae4175f20a068221954c8be104211add98cfe0614d13"} Sep 13 20:17:30.234140 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:30.233821 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6lr4c" Sep 13 20:17:30.234140 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:30.233923 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lnffs" event={"ID":"260a45fa-ca24-4f0f-8248-34f44c75e581","Type":"ContainerStarted","Data":"c9b4a1a3ebb9f0781fa3193a48997b7820ff9431b930ef23cc286f747c7116c2"} Sep 13 20:17:31.238708 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:31.238674 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lnffs" event={"ID":"260a45fa-ca24-4f0f-8248-34f44c75e581","Type":"ContainerStarted","Data":"36a9a45cdb2d512fafcc7bc738139d5b7509e8699c311400bc8e99cac8419943"} Sep 13 20:17:32.243909 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:32.243817 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lnffs" event={"ID":"260a45fa-ca24-4f0f-8248-34f44c75e581","Type":"ContainerStarted","Data":"468a34218659f49e1337aae4ee7ba7dc6b80951d0146fb48678c3bd791130e4c"} Sep 13 20:17:32.260602 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:32.260552 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lnffs" podStartSLOduration=17.150643135 podStartE2EDuration="19.260536442s" totalImagesPullingTime="2.109893307s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-13 20:17:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:17:32.259037825 +0000 UTC m=+284.435473847" watchObservedRunningTime="2026-09-13 20:17:32.260536442 +0000 UTC m=+284.436972477" Sep 13 20:17:33.188671 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:33.188617 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:17:33.188671 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:33.188669 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-6lr4c\" (UID: \"ba047ed2-663d-47be-bd5f-08548fdcdec9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6lr4c" Sep 13 20:17:33.188886 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:33.188852 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls\") pod \"dns-default-fw949\" (UID: \"a99d1dc1-43a2-4ada-99dd-a4a05c5faf68\") " pod="openshift-dns/dns-default-fw949" Sep 13 20:17:33.191086 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:33.191058 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls\") pod \"image-registry-9866c996d-vpm54\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:17:33.191086 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:33.191070 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ba047ed2-663d-47be-bd5f-08548fdcdec9-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-6lr4c\" (UID: \"ba047ed2-663d-47be-bd5f-08548fdcdec9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6lr4c" Sep 13 20:17:33.191230 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:33.191061 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a99d1dc1-43a2-4ada-99dd-a4a05c5faf68-metrics-tls\") pod \"dns-default-fw949\" (UID: \"a99d1dc1-43a2-4ada-99dd-a4a05c5faf68\") " pod="openshift-dns/dns-default-fw949" Sep 13 20:17:33.238028 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:33.237999 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-wzmsm\"" Sep 13 20:17:33.238028 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:33.238028 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9fnqc\"" Sep 13 20:17:33.238198 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:33.237999 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zlk89\"" Sep 13 20:17:33.244268 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:33.244237 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:17:33.244627 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:33.244282 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fw949" Sep 13 20:17:33.244627 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:33.244298 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6lr4c" Sep 13 20:17:33.524155 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:33.524133 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fw949"] Sep 13 20:17:33.526737 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:17:33.526708 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda99d1dc1_43a2_4ada_99dd_a4a05c5faf68.slice/crio-0b37bce6e6fca4dc770929fe563b8030a3c00a8f3a191e54f5ba054671bcb0bb WatchSource:0}: Error finding container 0b37bce6e6fca4dc770929fe563b8030a3c00a8f3a191e54f5ba054671bcb0bb: Status 404 returned error can't find the container with id 0b37bce6e6fca4dc770929fe563b8030a3c00a8f3a191e54f5ba054671bcb0bb Sep 13 20:17:33.528085 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:33.527956 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9866c996d-vpm54"] Sep 13 20:17:33.528992 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:33.528972 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-6lr4c"] Sep 13 20:17:33.529870 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:17:33.529833 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba047ed2_663d_47be_bd5f_08548fdcdec9.slice/crio-9a08f4ae304cd43499296008128fe2eb939d6aa5f6704b10ca3c9e4187d64fc0 WatchSource:0}: Error finding container 9a08f4ae304cd43499296008128fe2eb939d6aa5f6704b10ca3c9e4187d64fc0: Status 404 returned error can't find the container with id 9a08f4ae304cd43499296008128fe2eb939d6aa5f6704b10ca3c9e4187d64fc0 Sep 13 20:17:33.530441 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:17:33.530418 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd0e27d4_cbed_4619_94be_5c9f8eb73fc2.slice/crio-c2e65a79c79d3bbf977d3275a7c1b85a4b39b9d612a1c04429d0fc6937de7a1e WatchSource:0}: Error finding container c2e65a79c79d3bbf977d3275a7c1b85a4b39b9d612a1c04429d0fc6937de7a1e: Status 404 returned error can't find the container with id c2e65a79c79d3bbf977d3275a7c1b85a4b39b9d612a1c04429d0fc6937de7a1e Sep 13 20:17:34.252747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:34.252709 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6lr4c" event={"ID":"ba047ed2-663d-47be-bd5f-08548fdcdec9","Type":"ContainerStarted","Data":"9a08f4ae304cd43499296008128fe2eb939d6aa5f6704b10ca3c9e4187d64fc0"} Sep 13 20:17:34.254295 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:34.254251 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fw949" event={"ID":"a99d1dc1-43a2-4ada-99dd-a4a05c5faf68","Type":"ContainerStarted","Data":"0b37bce6e6fca4dc770929fe563b8030a3c00a8f3a191e54f5ba054671bcb0bb"} Sep 13 20:17:34.256367 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:34.256321 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9866c996d-vpm54" event={"ID":"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2","Type":"ContainerStarted","Data":"124cfad7904a8e404e2f6a24682d92dcef2a97cd54da1285d07d673cbed67d21"} Sep 13 20:17:34.256467 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:34.256368 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9866c996d-vpm54" event={"ID":"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2","Type":"ContainerStarted","Data":"c2e65a79c79d3bbf977d3275a7c1b85a4b39b9d612a1c04429d0fc6937de7a1e"} Sep 13 20:17:34.256559 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:34.256474 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:17:34.277363 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:34.277159 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-9866c996d-vpm54" podStartSLOduration=286.277134054 podStartE2EDuration="4m46.277134054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:12:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:17:34.274998118 +0000 UTC m=+286.451434143" watchObservedRunningTime="2026-09-13 20:17:34.277134054 +0000 UTC m=+286.453570079" Sep 13 20:17:35.261343 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:35.261303 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6lr4c" event={"ID":"ba047ed2-663d-47be-bd5f-08548fdcdec9","Type":"ContainerStarted","Data":"71f24cd599573f8fd9d23d58033571e951f0da371dc3918660cb3955628f9b49"} Sep 13 20:17:35.276422 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:35.276379 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6lr4c" podStartSLOduration=277.127290111 podStartE2EDuration="4m38.276364818s" totalImagesPullingTime="1.149074707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:12:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:17:35.274916832 +0000 UTC m=+287.451352853" watchObservedRunningTime="2026-09-13 20:17:35.276364818 +0000 UTC m=+287.452800841" Sep 13 20:17:36.266632 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:36.266591 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fw949" event={"ID":"a99d1dc1-43a2-4ada-99dd-a4a05c5faf68","Type":"ContainerStarted","Data":"96855a911011135327fdeed3bc0c89282aa7184daf1ff721e4633ed999da8039"} Sep 13 20:17:36.267149 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:36.266658 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fw949" event={"ID":"a99d1dc1-43a2-4ada-99dd-a4a05c5faf68","Type":"ContainerStarted","Data":"f47b9c6e1c9a3c2aefd9c41e5c1d009bd0f02b11559e31853c80fcc6673a84a2"} Sep 13 20:17:36.281749 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:36.281703 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-fw949" podStartSLOduration=250.977906171 podStartE2EDuration="4m13.281687838s" totalImagesPullingTime="2.303781667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:13:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:17:36.281045929 +0000 UTC m=+288.457481965" watchObservedRunningTime="2026-09-13 20:17:36.281687838 +0000 UTC m=+288.458123860" Sep 13 20:17:37.269912 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:37.269880 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fw949" Sep 13 20:17:40.148111 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:40.148068 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-service-ca-bundle\") pod \"router-default-655c4b696-bgtsn\" (UID: \"ff4cd6d7-3b88-4122-9d4e-d1560d389a70\") " pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:40.148111 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:40.148115 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/28166755-2fd2-4b23-ac8a-55a8121e0b00-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-pg8lq\" (UID: \"28166755-2fd2-4b23-ac8a-55a8121e0b00\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-pg8lq" Sep 13 20:17:40.148704 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:40.148681 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff4cd6d7-3b88-4122-9d4e-d1560d389a70-service-ca-bundle\") pod \"router-default-655c4b696-bgtsn\" (UID: \"ff4cd6d7-3b88-4122-9d4e-d1560d389a70\") " pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:40.150457 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:40.150436 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/28166755-2fd2-4b23-ac8a-55a8121e0b00-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-pg8lq\" (UID: \"28166755-2fd2-4b23-ac8a-55a8121e0b00\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-pg8lq" Sep 13 20:17:40.429028 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:40.428929 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-pg8lq" Sep 13 20:17:40.434424 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:40.434405 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:40.708249 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:40.708222 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-pg8lq"] Sep 13 20:17:40.710634 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:17:40.710608 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28166755_2fd2_4b23_ac8a_55a8121e0b00.slice/crio-2fd823493ac0ec425be4aa71d88330d2d89414706c1e2a039f4ce9203b98bc4c WatchSource:0}: Error finding container 2fd823493ac0ec425be4aa71d88330d2d89414706c1e2a039f4ce9203b98bc4c: Status 404 returned error can't find the container with id 2fd823493ac0ec425be4aa71d88330d2d89414706c1e2a039f4ce9203b98bc4c Sep 13 20:17:40.719167 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:40.719138 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-655c4b696-bgtsn"] Sep 13 20:17:40.721639 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:17:40.721616 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff4cd6d7_3b88_4122_9d4e_d1560d389a70.slice/crio-81f0e3b2153264dd41d3e253aa4b7c14f8dd6fd027f2f8c166cab9de9bf9e3ed WatchSource:0}: Error finding container 81f0e3b2153264dd41d3e253aa4b7c14f8dd6fd027f2f8c166cab9de9bf9e3ed: Status 404 returned error can't find the container with id 81f0e3b2153264dd41d3e253aa4b7c14f8dd6fd027f2f8c166cab9de9bf9e3ed Sep 13 20:17:41.285762 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:41.285724 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-655c4b696-bgtsn" event={"ID":"ff4cd6d7-3b88-4122-9d4e-d1560d389a70","Type":"ContainerStarted","Data":"1d8c7c9e15b4c9d83a8b9700e5dc71c8a5b891abdcef6fd1ba947ea0293f34cd"} Sep 13 20:17:41.286250 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:41.285778 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-655c4b696-bgtsn" event={"ID":"ff4cd6d7-3b88-4122-9d4e-d1560d389a70","Type":"ContainerStarted","Data":"81f0e3b2153264dd41d3e253aa4b7c14f8dd6fd027f2f8c166cab9de9bf9e3ed"} Sep 13 20:17:41.286932 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:41.286908 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-pg8lq" event={"ID":"28166755-2fd2-4b23-ac8a-55a8121e0b00","Type":"ContainerStarted","Data":"2fd823493ac0ec425be4aa71d88330d2d89414706c1e2a039f4ce9203b98bc4c"} Sep 13 20:17:41.304345 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:41.304300 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-655c4b696-bgtsn" podStartSLOduration=33.304285614 podStartE2EDuration="33.304285614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:17:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:17:41.303239011 +0000 UTC m=+293.479675034" watchObservedRunningTime="2026-09-13 20:17:41.304285614 +0000 UTC m=+293.480721637" Sep 13 20:17:41.434817 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:41.434781 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:41.437415 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:41.437390 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:42.290425 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:42.290398 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:42.291908 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:42.291884 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-655c4b696-bgtsn" Sep 13 20:17:42.306538 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:42.306504 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-95zg2" podUID="32f68c96-97d3-4904-a675-45f93df87572" Sep 13 20:17:43.299081 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:43.299044 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-pg8lq" event={"ID":"28166755-2fd2-4b23-ac8a-55a8121e0b00","Type":"ContainerStarted","Data":"727e128460852a6fb0e4b30c578a02f9d96d6fd11e6a9c1f91f394d198d8f929"} Sep 13 20:17:43.314309 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:43.314265 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-pg8lq" podStartSLOduration=33.380178295 podStartE2EDuration="35.314250281s" totalImagesPullingTime="1.934071986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:17:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:17:43.313446648 +0000 UTC m=+295.489882671" watchObservedRunningTime="2026-09-13 20:17:43.314250281 +0000 UTC m=+295.490686304" Sep 13 20:17:46.157217 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:46.157187 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-qld5g"] Sep 13 20:17:46.160470 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:46.160453 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-qld5g" Sep 13 20:17:46.162684 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:46.162661 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 13 20:17:46.163967 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:46.163944 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 13 20:17:46.164064 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:46.163948 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-kw475\"" Sep 13 20:17:46.164064 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:46.163982 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 13 20:17:46.169691 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:46.169670 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-qld5g"] Sep 13 20:17:46.295477 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:46.295438 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz5cs\" (UniqueName: \"kubernetes.io/projected/271622a3-b60c-4537-b553-6fd0aaf16d0e-kube-api-access-xz5cs\") pod \"prometheus-operator-68bcd7f987-qld5g\" (UID: \"271622a3-b60c-4537-b553-6fd0aaf16d0e\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-qld5g" Sep 13 20:17:46.295477 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:46.295477 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/271622a3-b60c-4537-b553-6fd0aaf16d0e-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-qld5g\" (UID: \"271622a3-b60c-4537-b553-6fd0aaf16d0e\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-qld5g" Sep 13 20:17:46.295683 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:46.295554 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/271622a3-b60c-4537-b553-6fd0aaf16d0e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-qld5g\" (UID: \"271622a3-b60c-4537-b553-6fd0aaf16d0e\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-qld5g" Sep 13 20:17:46.295683 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:46.295593 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/271622a3-b60c-4537-b553-6fd0aaf16d0e-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-qld5g\" (UID: \"271622a3-b60c-4537-b553-6fd0aaf16d0e\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-qld5g" Sep 13 20:17:46.396614 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:46.396577 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/271622a3-b60c-4537-b553-6fd0aaf16d0e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-qld5g\" (UID: \"271622a3-b60c-4537-b553-6fd0aaf16d0e\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-qld5g" Sep 13 20:17:46.396823 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:46.396624 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/271622a3-b60c-4537-b553-6fd0aaf16d0e-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-qld5g\" (UID: \"271622a3-b60c-4537-b553-6fd0aaf16d0e\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-qld5g" Sep 13 20:17:46.396823 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:46.396680 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz5cs\" (UniqueName: \"kubernetes.io/projected/271622a3-b60c-4537-b553-6fd0aaf16d0e-kube-api-access-xz5cs\") pod \"prometheus-operator-68bcd7f987-qld5g\" (UID: \"271622a3-b60c-4537-b553-6fd0aaf16d0e\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-qld5g" Sep 13 20:17:46.396823 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:46.396700 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/271622a3-b60c-4537-b553-6fd0aaf16d0e-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-qld5g\" (UID: \"271622a3-b60c-4537-b553-6fd0aaf16d0e\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-qld5g" Sep 13 20:17:46.397340 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:46.397314 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/271622a3-b60c-4537-b553-6fd0aaf16d0e-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-qld5g\" (UID: \"271622a3-b60c-4537-b553-6fd0aaf16d0e\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-qld5g" Sep 13 20:17:46.399101 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:46.399077 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/271622a3-b60c-4537-b553-6fd0aaf16d0e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-qld5g\" (UID: \"271622a3-b60c-4537-b553-6fd0aaf16d0e\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-qld5g" Sep 13 20:17:46.399185 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:46.399149 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/271622a3-b60c-4537-b553-6fd0aaf16d0e-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-qld5g\" (UID: \"271622a3-b60c-4537-b553-6fd0aaf16d0e\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-qld5g" Sep 13 20:17:46.404104 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:46.404085 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz5cs\" (UniqueName: \"kubernetes.io/projected/271622a3-b60c-4537-b553-6fd0aaf16d0e-kube-api-access-xz5cs\") pod \"prometheus-operator-68bcd7f987-qld5g\" (UID: \"271622a3-b60c-4537-b553-6fd0aaf16d0e\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-qld5g" Sep 13 20:17:46.469892 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:46.469800 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-qld5g" Sep 13 20:17:46.736986 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:46.736962 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-qld5g"] Sep 13 20:17:46.739058 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:17:46.739023 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod271622a3_b60c_4537_b553_6fd0aaf16d0e.slice/crio-52133221d537f9b6178d20c2d7f2ae1525b6509e1a47232ff636239e073a46c9 WatchSource:0}: Error finding container 52133221d537f9b6178d20c2d7f2ae1525b6509e1a47232ff636239e073a46c9: Status 404 returned error can't find the container with id 52133221d537f9b6178d20c2d7f2ae1525b6509e1a47232ff636239e073a46c9 Sep 13 20:17:47.275011 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:47.274981 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fw949" Sep 13 20:17:47.315579 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:47.315543 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-qld5g" event={"ID":"271622a3-b60c-4537-b553-6fd0aaf16d0e","Type":"ContainerStarted","Data":"52133221d537f9b6178d20c2d7f2ae1525b6509e1a47232ff636239e073a46c9"} Sep 13 20:17:48.260287 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:48.260209 2557 kubelet.go:1648] "Image garbage collection succeeded" Sep 13 20:17:48.320741 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:48.320605 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-qld5g" event={"ID":"271622a3-b60c-4537-b553-6fd0aaf16d0e","Type":"ContainerStarted","Data":"0dcda2452f1f1a35fb28cb826f1a6c8630c9769364de71d7ddcef9285c58dba2"} Sep 13 20:17:48.320741 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:48.320656 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-qld5g" event={"ID":"271622a3-b60c-4537-b553-6fd0aaf16d0e","Type":"ContainerStarted","Data":"a6fec221baae4a31bc328f47c4faf4622db9909da37ef0c346e87a2de9112f48"} Sep 13 20:17:48.336637 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:48.335689 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-qld5g" podStartSLOduration=1.081662779 podStartE2EDuration="2.335675974s" totalImagesPullingTime="1.254013195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:17:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:17:48.334305965 +0000 UTC m=+300.510741988" watchObservedRunningTime="2026-09-13 20:17:48.335675974 +0000 UTC m=+300.512112027" Sep 13 20:17:50.491927 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.491893 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-dvjks"] Sep 13 20:17:50.495686 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.495657 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dvjks" Sep 13 20:17:50.498289 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.498071 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 13 20:17:50.498289 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.498113 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 13 20:17:50.498289 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.498178 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-j5d87\"" Sep 13 20:17:50.503728 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.503707 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-89257"] Sep 13 20:17:50.507425 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.507390 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.509180 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.509157 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-dvjks"] Sep 13 20:17:50.513521 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.512420 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 13 20:17:50.519554 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.519405 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hkdjk\"" Sep 13 20:17:50.522366 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.522344 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 13 20:17:50.522524 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.522413 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 13 20:17:50.631836 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.631798 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/727a9bfe-987a-40b3-ba33-53844a2084f3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-89257\" (UID: \"727a9bfe-987a-40b3-ba33-53844a2084f3\") " pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.632014 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.631844 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/727a9bfe-987a-40b3-ba33-53844a2084f3-sys\") pod \"node-exporter-89257\" (UID: \"727a9bfe-987a-40b3-ba33-53844a2084f3\") " pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.632014 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.631876 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/727a9bfe-987a-40b3-ba33-53844a2084f3-node-exporter-tls\") pod \"node-exporter-89257\" (UID: \"727a9bfe-987a-40b3-ba33-53844a2084f3\") " pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.632014 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.631901 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r24dc\" (UniqueName: \"kubernetes.io/projected/9555dc90-eaf9-4bb6-8fe2-03372a4e44c5-kube-api-access-r24dc\") pod \"openshift-state-metrics-67dcf8c979-dvjks\" (UID: \"9555dc90-eaf9-4bb6-8fe2-03372a4e44c5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dvjks" Sep 13 20:17:50.632014 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.631924 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/727a9bfe-987a-40b3-ba33-53844a2084f3-node-exporter-wtmp\") pod \"node-exporter-89257\" (UID: \"727a9bfe-987a-40b3-ba33-53844a2084f3\") " pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.632014 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.631963 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/727a9bfe-987a-40b3-ba33-53844a2084f3-metrics-client-ca\") pod \"node-exporter-89257\" (UID: \"727a9bfe-987a-40b3-ba33-53844a2084f3\") " pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.632014 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.631990 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9555dc90-eaf9-4bb6-8fe2-03372a4e44c5-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-dvjks\" (UID: \"9555dc90-eaf9-4bb6-8fe2-03372a4e44c5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dvjks" Sep 13 20:17:50.632335 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.632066 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/727a9bfe-987a-40b3-ba33-53844a2084f3-root\") pod \"node-exporter-89257\" (UID: \"727a9bfe-987a-40b3-ba33-53844a2084f3\") " pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.632335 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.632102 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2kvb\" (UniqueName: \"kubernetes.io/projected/727a9bfe-987a-40b3-ba33-53844a2084f3-kube-api-access-t2kvb\") pod \"node-exporter-89257\" (UID: \"727a9bfe-987a-40b3-ba33-53844a2084f3\") " pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.632335 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.632167 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9555dc90-eaf9-4bb6-8fe2-03372a4e44c5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-dvjks\" (UID: \"9555dc90-eaf9-4bb6-8fe2-03372a4e44c5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dvjks" Sep 13 20:17:50.632335 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.632235 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/727a9bfe-987a-40b3-ba33-53844a2084f3-node-exporter-textfile\") pod \"node-exporter-89257\" (UID: \"727a9bfe-987a-40b3-ba33-53844a2084f3\") " pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.632335 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.632264 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/727a9bfe-987a-40b3-ba33-53844a2084f3-node-exporter-accelerators-collector-config\") pod \"node-exporter-89257\" (UID: \"727a9bfe-987a-40b3-ba33-53844a2084f3\") " pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.632335 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.632294 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9555dc90-eaf9-4bb6-8fe2-03372a4e44c5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-dvjks\" (UID: \"9555dc90-eaf9-4bb6-8fe2-03372a4e44c5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dvjks" Sep 13 20:17:50.733347 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.733278 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/727a9bfe-987a-40b3-ba33-53844a2084f3-sys\") pod \"node-exporter-89257\" (UID: \"727a9bfe-987a-40b3-ba33-53844a2084f3\") " pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.733577 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.733404 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/727a9bfe-987a-40b3-ba33-53844a2084f3-sys\") pod \"node-exporter-89257\" (UID: \"727a9bfe-987a-40b3-ba33-53844a2084f3\") " pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.733577 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.733525 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/727a9bfe-987a-40b3-ba33-53844a2084f3-node-exporter-tls\") pod \"node-exporter-89257\" (UID: \"727a9bfe-987a-40b3-ba33-53844a2084f3\") " pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.733577 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.733572 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r24dc\" (UniqueName: \"kubernetes.io/projected/9555dc90-eaf9-4bb6-8fe2-03372a4e44c5-kube-api-access-r24dc\") pod \"openshift-state-metrics-67dcf8c979-dvjks\" (UID: \"9555dc90-eaf9-4bb6-8fe2-03372a4e44c5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dvjks" Sep 13 20:17:50.733784 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.733597 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/727a9bfe-987a-40b3-ba33-53844a2084f3-node-exporter-wtmp\") pod \"node-exporter-89257\" (UID: \"727a9bfe-987a-40b3-ba33-53844a2084f3\") " pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.733784 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.733631 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/727a9bfe-987a-40b3-ba33-53844a2084f3-metrics-client-ca\") pod \"node-exporter-89257\" (UID: \"727a9bfe-987a-40b3-ba33-53844a2084f3\") " pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.733784 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.733661 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9555dc90-eaf9-4bb6-8fe2-03372a4e44c5-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-dvjks\" (UID: \"9555dc90-eaf9-4bb6-8fe2-03372a4e44c5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dvjks" Sep 13 20:17:50.733784 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.733704 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/727a9bfe-987a-40b3-ba33-53844a2084f3-root\") pod \"node-exporter-89257\" (UID: \"727a9bfe-987a-40b3-ba33-53844a2084f3\") " pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.733784 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.733725 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2kvb\" (UniqueName: \"kubernetes.io/projected/727a9bfe-987a-40b3-ba33-53844a2084f3-kube-api-access-t2kvb\") pod \"node-exporter-89257\" (UID: \"727a9bfe-987a-40b3-ba33-53844a2084f3\") " pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.733784 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.733774 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9555dc90-eaf9-4bb6-8fe2-03372a4e44c5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-dvjks\" (UID: \"9555dc90-eaf9-4bb6-8fe2-03372a4e44c5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dvjks" Sep 13 20:17:50.734076 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.733816 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/727a9bfe-987a-40b3-ba33-53844a2084f3-node-exporter-textfile\") pod \"node-exporter-89257\" (UID: \"727a9bfe-987a-40b3-ba33-53844a2084f3\") " pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.734076 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.733860 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/727a9bfe-987a-40b3-ba33-53844a2084f3-node-exporter-accelerators-collector-config\") pod \"node-exporter-89257\" (UID: \"727a9bfe-987a-40b3-ba33-53844a2084f3\") " pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.734076 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.733890 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9555dc90-eaf9-4bb6-8fe2-03372a4e44c5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-dvjks\" (UID: \"9555dc90-eaf9-4bb6-8fe2-03372a4e44c5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dvjks" Sep 13 20:17:50.734243 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.734208 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/727a9bfe-987a-40b3-ba33-53844a2084f3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-89257\" (UID: \"727a9bfe-987a-40b3-ba33-53844a2084f3\") " pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.734708 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.734345 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/727a9bfe-987a-40b3-ba33-53844a2084f3-metrics-client-ca\") pod \"node-exporter-89257\" (UID: \"727a9bfe-987a-40b3-ba33-53844a2084f3\") " pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.734708 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.734651 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/727a9bfe-987a-40b3-ba33-53844a2084f3-node-exporter-textfile\") pod \"node-exporter-89257\" (UID: \"727a9bfe-987a-40b3-ba33-53844a2084f3\") " pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.734896 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.734793 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/727a9bfe-987a-40b3-ba33-53844a2084f3-node-exporter-wtmp\") pod \"node-exporter-89257\" (UID: \"727a9bfe-987a-40b3-ba33-53844a2084f3\") " pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.735379 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.735267 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/727a9bfe-987a-40b3-ba33-53844a2084f3-node-exporter-accelerators-collector-config\") pod \"node-exporter-89257\" (UID: \"727a9bfe-987a-40b3-ba33-53844a2084f3\") " pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.735379 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.735333 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/727a9bfe-987a-40b3-ba33-53844a2084f3-root\") pod \"node-exporter-89257\" (UID: \"727a9bfe-987a-40b3-ba33-53844a2084f3\") " pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.735574 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.735476 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9555dc90-eaf9-4bb6-8fe2-03372a4e44c5-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-dvjks\" (UID: \"9555dc90-eaf9-4bb6-8fe2-03372a4e44c5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dvjks" Sep 13 20:17:50.735747 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:50.735727 2557 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Sep 13 20:17:50.735900 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:17:50.735887 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9555dc90-eaf9-4bb6-8fe2-03372a4e44c5-openshift-state-metrics-tls podName:9555dc90-eaf9-4bb6-8fe2-03372a4e44c5 nodeName:}" failed. No retries permitted until 2026-09-13 20:17:51.23586614 +0000 UTC m=+303.412302158 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/9555dc90-eaf9-4bb6-8fe2-03372a4e44c5-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-dvjks" (UID: "9555dc90-eaf9-4bb6-8fe2-03372a4e44c5") : secret "openshift-state-metrics-tls" not found Sep 13 20:17:50.736586 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.736535 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/727a9bfe-987a-40b3-ba33-53844a2084f3-node-exporter-tls\") pod \"node-exporter-89257\" (UID: \"727a9bfe-987a-40b3-ba33-53844a2084f3\") " pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.737231 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.737200 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9555dc90-eaf9-4bb6-8fe2-03372a4e44c5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-dvjks\" (UID: \"9555dc90-eaf9-4bb6-8fe2-03372a4e44c5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dvjks" Sep 13 20:17:50.738095 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.738059 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/727a9bfe-987a-40b3-ba33-53844a2084f3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-89257\" (UID: \"727a9bfe-987a-40b3-ba33-53844a2084f3\") " pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.744922 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.744858 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2kvb\" (UniqueName: \"kubernetes.io/projected/727a9bfe-987a-40b3-ba33-53844a2084f3-kube-api-access-t2kvb\") pod \"node-exporter-89257\" (UID: \"727a9bfe-987a-40b3-ba33-53844a2084f3\") " pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.745045 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.745023 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r24dc\" (UniqueName: \"kubernetes.io/projected/9555dc90-eaf9-4bb6-8fe2-03372a4e44c5-kube-api-access-r24dc\") pod \"openshift-state-metrics-67dcf8c979-dvjks\" (UID: \"9555dc90-eaf9-4bb6-8fe2-03372a4e44c5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dvjks" Sep 13 20:17:50.823998 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.823964 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-89257" Sep 13 20:17:50.835743 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:17:50.835718 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod727a9bfe_987a_40b3_ba33_53844a2084f3.slice/crio-fb6c43f6d8111118f4163860180b6c8b41c6abcf1bce3493df766a1180eeb777 WatchSource:0}: Error finding container fb6c43f6d8111118f4163860180b6c8b41c6abcf1bce3493df766a1180eeb777: Status 404 returned error can't find the container with id fb6c43f6d8111118f4163860180b6c8b41c6abcf1bce3493df766a1180eeb777 Sep 13 20:17:50.837789 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:50.837772 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 13 20:17:51.239188 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:51.239153 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9555dc90-eaf9-4bb6-8fe2-03372a4e44c5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-dvjks\" (UID: \"9555dc90-eaf9-4bb6-8fe2-03372a4e44c5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dvjks" Sep 13 20:17:51.241741 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:51.241713 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9555dc90-eaf9-4bb6-8fe2-03372a4e44c5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-dvjks\" (UID: \"9555dc90-eaf9-4bb6-8fe2-03372a4e44c5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dvjks" Sep 13 20:17:51.334433 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:51.334396 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-89257" event={"ID":"727a9bfe-987a-40b3-ba33-53844a2084f3","Type":"ContainerStarted","Data":"fb6c43f6d8111118f4163860180b6c8b41c6abcf1bce3493df766a1180eeb777"} Sep 13 20:17:51.410990 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:51.410957 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dvjks" Sep 13 20:17:51.694185 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:51.694153 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-dvjks"] Sep 13 20:17:51.900529 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:17:51.900460 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9555dc90_eaf9_4bb6_8fe2_03372a4e44c5.slice/crio-b399d55e25dff9dbd4da3b3465df3900cc9c72b312f621180081ac4acd03b539 WatchSource:0}: Error finding container b399d55e25dff9dbd4da3b3465df3900cc9c72b312f621180081ac4acd03b539: Status 404 returned error can't find the container with id b399d55e25dff9dbd4da3b3465df3900cc9c72b312f621180081ac4acd03b539 Sep 13 20:17:52.339993 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:52.339959 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dvjks" event={"ID":"9555dc90-eaf9-4bb6-8fe2-03372a4e44c5","Type":"ContainerStarted","Data":"36af0390452201018d5ce3db057b7199cc183d66876a7027f76723ed36b5edba"} Sep 13 20:17:52.340167 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:52.340013 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dvjks" event={"ID":"9555dc90-eaf9-4bb6-8fe2-03372a4e44c5","Type":"ContainerStarted","Data":"6868e7589ae7984ddb75cdfd7797ddb9680a5c9b1557766933d297864f6a699e"} Sep 13 20:17:52.340167 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:52.340041 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dvjks" event={"ID":"9555dc90-eaf9-4bb6-8fe2-03372a4e44c5","Type":"ContainerStarted","Data":"b399d55e25dff9dbd4da3b3465df3900cc9c72b312f621180081ac4acd03b539"} Sep 13 20:17:52.341668 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:52.341643 2557 generic.go:356] "Generic (PLEG): container finished" podID="727a9bfe-987a-40b3-ba33-53844a2084f3" containerID="0e4dc01457b48a1a08ea44b8b18409d1f59009bca74500263aad6055b63ba59b" exitCode=0 Sep 13 20:17:52.341777 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:52.341676 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-89257" event={"ID":"727a9bfe-987a-40b3-ba33-53844a2084f3","Type":"ContainerDied","Data":"0e4dc01457b48a1a08ea44b8b18409d1f59009bca74500263aad6055b63ba59b"} Sep 13 20:17:53.249858 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:53.249823 2557 patch_prober.go:28] interesting pod/image-registry-9866c996d-vpm54 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 13 20:17:53.250239 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:53.249878 2557 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-9866c996d-vpm54" podUID="fd0e27d4-cbed-4619-94be-5c9f8eb73fc2" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 13 20:17:53.347752 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:53.347718 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dvjks" event={"ID":"9555dc90-eaf9-4bb6-8fe2-03372a4e44c5","Type":"ContainerStarted","Data":"2aba2a470c2267639fe29ad12562e15459cf76d756d946dcf4edcc40046a7720"} Sep 13 20:17:53.349897 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:53.349872 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-89257" event={"ID":"727a9bfe-987a-40b3-ba33-53844a2084f3","Type":"ContainerStarted","Data":"0d4517512cdb3863c445f56ddf64ba216789203b59b8dabf2f61463cc659f29d"} Sep 13 20:17:53.350031 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:53.349913 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-89257" event={"ID":"727a9bfe-987a-40b3-ba33-53844a2084f3","Type":"ContainerStarted","Data":"89436de6e9ac2559b5a83408a0fed2589e164b0ef6eff898b879c40fbda03811"} Sep 13 20:17:53.367999 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:53.367955 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dvjks" podStartSLOduration=2.429788378 podStartE2EDuration="3.367941673s" totalImagesPullingTime="938.153295ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:17:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:17:53.366240728 +0000 UTC m=+305.542676750" watchObservedRunningTime="2026-09-13 20:17:53.367941673 +0000 UTC m=+305.544377699" Sep 13 20:17:53.386211 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:53.386157 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-89257" podStartSLOduration=2.2127986809999998 podStartE2EDuration="3.386136659s" totalImagesPullingTime="1.108684675s" totalInitContainerRuntime="64.653303ms" isStatefulPod=true podCreationTimestamp="2026-09-13 20:17:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:17:53.385633601 +0000 UTC m=+305.562069625" watchObservedRunningTime="2026-09-13 20:17:53.386136659 +0000 UTC m=+305.562572683" Sep 13 20:17:55.266984 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:55.266950 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:17:57.306346 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:17:57.306314 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:18:00.542338 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:00.542302 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-9866c996d-vpm54"] Sep 13 20:18:23.191222 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.191183 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-86d5d78598-2wq8m"] Sep 13 20:18:23.197009 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.196988 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:18:23.199262 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.199233 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 13 20:18:23.199394 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.199237 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 13 20:18:23.199484 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.199470 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 13 20:18:23.200334 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.200317 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 13 20:18:23.200465 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.200447 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-njzsp\"" Sep 13 20:18:23.200542 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.200461 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 13 20:18:23.200542 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.200489 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 13 20:18:23.200661 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.200565 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 13 20:18:23.204773 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.204752 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 13 20:18:23.205528 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.205490 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86d5d78598-2wq8m"] Sep 13 20:18:23.303624 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.303587 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/687568a6-db54-413f-8894-8f0717c81ee4-console-config\") pod \"console-86d5d78598-2wq8m\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:18:23.303624 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.303629 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c79vp\" (UniqueName: \"kubernetes.io/projected/687568a6-db54-413f-8894-8f0717c81ee4-kube-api-access-c79vp\") pod \"console-86d5d78598-2wq8m\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:18:23.303838 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.303689 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/687568a6-db54-413f-8894-8f0717c81ee4-trusted-ca-bundle\") pod \"console-86d5d78598-2wq8m\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:18:23.303838 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.303710 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/687568a6-db54-413f-8894-8f0717c81ee4-console-oauth-config\") pod \"console-86d5d78598-2wq8m\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:18:23.303838 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.303738 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/687568a6-db54-413f-8894-8f0717c81ee4-console-serving-cert\") pod \"console-86d5d78598-2wq8m\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:18:23.303838 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.303775 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/687568a6-db54-413f-8894-8f0717c81ee4-service-ca\") pod \"console-86d5d78598-2wq8m\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:18:23.303838 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.303793 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/687568a6-db54-413f-8894-8f0717c81ee4-oauth-serving-cert\") pod \"console-86d5d78598-2wq8m\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:18:23.404243 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.404207 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/687568a6-db54-413f-8894-8f0717c81ee4-service-ca\") pod \"console-86d5d78598-2wq8m\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:18:23.404430 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.404249 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/687568a6-db54-413f-8894-8f0717c81ee4-oauth-serving-cert\") pod \"console-86d5d78598-2wq8m\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:18:23.404430 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.404289 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/687568a6-db54-413f-8894-8f0717c81ee4-console-config\") pod \"console-86d5d78598-2wq8m\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:18:23.404430 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.404318 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c79vp\" (UniqueName: \"kubernetes.io/projected/687568a6-db54-413f-8894-8f0717c81ee4-kube-api-access-c79vp\") pod \"console-86d5d78598-2wq8m\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:18:23.404430 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.404391 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/687568a6-db54-413f-8894-8f0717c81ee4-trusted-ca-bundle\") pod \"console-86d5d78598-2wq8m\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:18:23.404430 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.404425 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/687568a6-db54-413f-8894-8f0717c81ee4-console-oauth-config\") pod \"console-86d5d78598-2wq8m\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:18:23.404718 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.404458 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/687568a6-db54-413f-8894-8f0717c81ee4-console-serving-cert\") pod \"console-86d5d78598-2wq8m\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:18:23.405091 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.405062 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/687568a6-db54-413f-8894-8f0717c81ee4-service-ca\") pod \"console-86d5d78598-2wq8m\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:18:23.405216 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.405098 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/687568a6-db54-413f-8894-8f0717c81ee4-oauth-serving-cert\") pod \"console-86d5d78598-2wq8m\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:18:23.405216 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.405071 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/687568a6-db54-413f-8894-8f0717c81ee4-console-config\") pod \"console-86d5d78598-2wq8m\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:18:23.405387 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.405366 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/687568a6-db54-413f-8894-8f0717c81ee4-trusted-ca-bundle\") pod \"console-86d5d78598-2wq8m\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:18:23.406871 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.406840 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/687568a6-db54-413f-8894-8f0717c81ee4-console-oauth-config\") pod \"console-86d5d78598-2wq8m\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:18:23.406989 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.406975 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/687568a6-db54-413f-8894-8f0717c81ee4-console-serving-cert\") pod \"console-86d5d78598-2wq8m\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:18:23.411461 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.411440 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c79vp\" (UniqueName: \"kubernetes.io/projected/687568a6-db54-413f-8894-8f0717c81ee4-kube-api-access-c79vp\") pod \"console-86d5d78598-2wq8m\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:18:23.507394 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.507322 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:18:23.775625 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:23.775591 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86d5d78598-2wq8m"] Sep 13 20:18:23.780782 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:18:23.780753 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod687568a6_db54_413f_8894_8f0717c81ee4.slice/crio-12a75b82a7f25f2bc985e96a3baa32c41193f0565b652fd84a7ba5e7974da74f WatchSource:0}: Error finding container 12a75b82a7f25f2bc985e96a3baa32c41193f0565b652fd84a7ba5e7974da74f: Status 404 returned error can't find the container with id 12a75b82a7f25f2bc985e96a3baa32c41193f0565b652fd84a7ba5e7974da74f Sep 13 20:18:24.462178 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:24.462131 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86d5d78598-2wq8m" event={"ID":"687568a6-db54-413f-8894-8f0717c81ee4","Type":"ContainerStarted","Data":"12a75b82a7f25f2bc985e96a3baa32c41193f0565b652fd84a7ba5e7974da74f"} Sep 13 20:18:24.588128 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:24.588094 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-655c4b696-bgtsn_ff4cd6d7-3b88-4122-9d4e-d1560d389a70/router/0.log" Sep 13 20:18:24.598536 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:24.598490 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-k8c5m_5b70f512-38d7-4546-be8f-c40233eabd8f/serve-healthcheck-canary/0.log" Sep 13 20:18:25.561018 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:25.560974 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-9866c996d-vpm54" podUID="fd0e27d4-cbed-4619-94be-5c9f8eb73fc2" containerName="registry" containerID="cri-o://124cfad7904a8e404e2f6a24682d92dcef2a97cd54da1285d07d673cbed67d21" gracePeriod=30 Sep 13 20:18:26.430349 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.430318 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:18:26.471726 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.471687 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86d5d78598-2wq8m" event={"ID":"687568a6-db54-413f-8894-8f0717c81ee4","Type":"ContainerStarted","Data":"71b23657b4ce1695e762354683a5889585abb4ae64959b17f9120c8ef485c512"} Sep 13 20:18:26.472927 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.472899 2557 generic.go:356] "Generic (PLEG): container finished" podID="fd0e27d4-cbed-4619-94be-5c9f8eb73fc2" containerID="124cfad7904a8e404e2f6a24682d92dcef2a97cd54da1285d07d673cbed67d21" exitCode=0 Sep 13 20:18:26.473047 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.472939 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9866c996d-vpm54" Sep 13 20:18:26.473047 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.472939 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9866c996d-vpm54" event={"ID":"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2","Type":"ContainerDied","Data":"124cfad7904a8e404e2f6a24682d92dcef2a97cd54da1285d07d673cbed67d21"} Sep 13 20:18:26.473164 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.473065 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9866c996d-vpm54" event={"ID":"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2","Type":"ContainerDied","Data":"c2e65a79c79d3bbf977d3275a7c1b85a4b39b9d612a1c04429d0fc6937de7a1e"} Sep 13 20:18:26.473164 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.473096 2557 scope.go:117] "RemoveContainer" containerID="124cfad7904a8e404e2f6a24682d92dcef2a97cd54da1285d07d673cbed67d21" Sep 13 20:18:26.481704 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.481687 2557 scope.go:117] "RemoveContainer" containerID="124cfad7904a8e404e2f6a24682d92dcef2a97cd54da1285d07d673cbed67d21" Sep 13 20:18:26.481962 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:18:26.481941 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"124cfad7904a8e404e2f6a24682d92dcef2a97cd54da1285d07d673cbed67d21\": container with ID starting with 124cfad7904a8e404e2f6a24682d92dcef2a97cd54da1285d07d673cbed67d21 not found: ID does not exist" containerID="124cfad7904a8e404e2f6a24682d92dcef2a97cd54da1285d07d673cbed67d21" Sep 13 20:18:26.482028 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.481973 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"124cfad7904a8e404e2f6a24682d92dcef2a97cd54da1285d07d673cbed67d21"} err="failed to get container status \"124cfad7904a8e404e2f6a24682d92dcef2a97cd54da1285d07d673cbed67d21\": rpc error: code = NotFound desc = could not find container \"124cfad7904a8e404e2f6a24682d92dcef2a97cd54da1285d07d673cbed67d21\": container with ID starting with 124cfad7904a8e404e2f6a24682d92dcef2a97cd54da1285d07d673cbed67d21 not found: ID does not exist" Sep 13 20:18:26.487098 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.487056 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-86d5d78598-2wq8m" podStartSLOduration=0.938740759 podStartE2EDuration="3.487044342s" totalImagesPullingTime="2.548303583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:18:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:18:26.485871631 +0000 UTC m=+338.662307663" watchObservedRunningTime="2026-09-13 20:18:26.487044342 +0000 UTC m=+338.663480354" Sep 13 20:18:26.530800 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.530767 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-trusted-ca\") pod \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " Sep 13 20:18:26.530952 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.530824 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-ca-trust-extracted\") pod \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " Sep 13 20:18:26.530952 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.530866 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-image-registry-private-configuration\") pod \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " Sep 13 20:18:26.530952 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.530890 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-installation-pull-secrets\") pod \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " Sep 13 20:18:26.530952 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.530908 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-bound-sa-token\") pod \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " Sep 13 20:18:26.530952 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.530927 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdpwn\" (UniqueName: \"kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-kube-api-access-qdpwn\") pod \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " Sep 13 20:18:26.530952 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.530949 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-certificates\") pod \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " Sep 13 20:18:26.531255 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.530984 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls\") pod \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\" (UID: \"fd0e27d4-cbed-4619-94be-5c9f8eb73fc2\") " Sep 13 20:18:26.531255 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.531219 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "fd0e27d4-cbed-4619-94be-5c9f8eb73fc2" (UID: "fd0e27d4-cbed-4619-94be-5c9f8eb73fc2"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 13 20:18:26.532687 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.532620 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "fd0e27d4-cbed-4619-94be-5c9f8eb73fc2" (UID: "fd0e27d4-cbed-4619-94be-5c9f8eb73fc2"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 13 20:18:26.533789 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.533761 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "fd0e27d4-cbed-4619-94be-5c9f8eb73fc2" (UID: "fd0e27d4-cbed-4619-94be-5c9f8eb73fc2"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 20:18:26.533789 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.533775 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "fd0e27d4-cbed-4619-94be-5c9f8eb73fc2" (UID: "fd0e27d4-cbed-4619-94be-5c9f8eb73fc2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 20:18:26.533789 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.533774 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "fd0e27d4-cbed-4619-94be-5c9f8eb73fc2" (UID: "fd0e27d4-cbed-4619-94be-5c9f8eb73fc2"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 20:18:26.533977 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.533938 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "fd0e27d4-cbed-4619-94be-5c9f8eb73fc2" (UID: "fd0e27d4-cbed-4619-94be-5c9f8eb73fc2"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 20:18:26.534356 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.534336 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-kube-api-access-qdpwn" (OuterVolumeSpecName: "kube-api-access-qdpwn") pod "fd0e27d4-cbed-4619-94be-5c9f8eb73fc2" (UID: "fd0e27d4-cbed-4619-94be-5c9f8eb73fc2"). InnerVolumeSpecName "kube-api-access-qdpwn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 20:18:26.539983 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.539955 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "fd0e27d4-cbed-4619-94be-5c9f8eb73fc2" (UID: "fd0e27d4-cbed-4619-94be-5c9f8eb73fc2"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 13 20:18:26.632425 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.632375 2557 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-bound-sa-token\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:18:26.632425 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.632416 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdpwn\" (UniqueName: \"kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-kube-api-access-qdpwn\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:18:26.632425 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.632428 2557 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-certificates\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:18:26.632425 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.632437 2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-registry-tls\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:18:26.632902 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.632446 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-trusted-ca\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:18:26.632902 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.632454 2557 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-ca-trust-extracted\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:18:26.632902 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.632464 2557 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-image-registry-private-configuration\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:18:26.632902 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.632473 2557 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2-installation-pull-secrets\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:18:26.794483 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.794418 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-9866c996d-vpm54"] Sep 13 20:18:26.798613 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:26.798588 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-9866c996d-vpm54"] Sep 13 20:18:28.310921 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:28.310879 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd0e27d4-cbed-4619-94be-5c9f8eb73fc2" path="/var/lib/kubelet/pods/fd0e27d4-cbed-4619-94be-5c9f8eb73fc2/volumes" Sep 13 20:18:33.508412 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:33.508380 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:18:33.509105 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:33.508442 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:18:33.513175 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:33.513152 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:18:34.503668 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:34.503631 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:18:41.528982 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:41.528950 2557 generic.go:356] "Generic (PLEG): container finished" podID="ee7be3e3-c579-4378-9a12-94b54e80d33e" containerID="dd70cc33718895a60840c751c71458ba3c793a0f4e7d3fc7e100c8485fefd174" exitCode=0 Sep 13 20:18:41.529346 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:41.529033 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-g7j4k" event={"ID":"ee7be3e3-c579-4378-9a12-94b54e80d33e","Type":"ContainerDied","Data":"dd70cc33718895a60840c751c71458ba3c793a0f4e7d3fc7e100c8485fefd174"} Sep 13 20:18:41.529434 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:41.529370 2557 scope.go:117] "RemoveContainer" containerID="dd70cc33718895a60840c751c71458ba3c793a0f4e7d3fc7e100c8485fefd174" Sep 13 20:18:42.535396 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:18:42.535367 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-g7j4k" event={"ID":"ee7be3e3-c579-4378-9a12-94b54e80d33e","Type":"ContainerStarted","Data":"494d9812d56ecf040e508456ddc3166281da64cc2fc659c34f99e5fcad12926a"} Sep 13 20:19:02.198443 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:02.198409 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs\") pod \"network-metrics-daemon-95zg2\" (UID: \"32f68c96-97d3-4904-a675-45f93df87572\") " pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:19:02.200682 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:02.200663 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32f68c96-97d3-4904-a675-45f93df87572-metrics-certs\") pod \"network-metrics-daemon-95zg2\" (UID: \"32f68c96-97d3-4904-a675-45f93df87572\") " pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:19:02.409408 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:02.409374 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5c5lx\"" Sep 13 20:19:02.417243 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:02.417225 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95zg2" Sep 13 20:19:02.693043 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:02.693011 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-95zg2"] Sep 13 20:19:02.696217 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:19:02.696186 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32f68c96_97d3_4904_a675_45f93df87572.slice/crio-3a1ad695d3e99aa90d257822e756142c0dea65d6a110018c77788ea134497867 WatchSource:0}: Error finding container 3a1ad695d3e99aa90d257822e756142c0dea65d6a110018c77788ea134497867: Status 404 returned error can't find the container with id 3a1ad695d3e99aa90d257822e756142c0dea65d6a110018c77788ea134497867 Sep 13 20:19:03.616140 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:03.616107 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-95zg2" event={"ID":"32f68c96-97d3-4904-a675-45f93df87572","Type":"ContainerStarted","Data":"3a1ad695d3e99aa90d257822e756142c0dea65d6a110018c77788ea134497867"} Sep 13 20:19:04.622034 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:04.621995 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-95zg2" event={"ID":"32f68c96-97d3-4904-a675-45f93df87572","Type":"ContainerStarted","Data":"2da053539825515a0cb13db4f1b5d542e1cf1f968f6d488c451b6ccd0195364e"} Sep 13 20:19:04.622411 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:04.622052 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-95zg2" event={"ID":"32f68c96-97d3-4904-a675-45f93df87572","Type":"ContainerStarted","Data":"647a67e2f2043fed587ad3a7f368da4d503d8e2d12ab6789142c901733c060e4"} Sep 13 20:19:04.636855 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:04.636809 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-95zg2" podStartSLOduration=375.512201982 podStartE2EDuration="6m16.636794352s" totalImagesPullingTime="1.12459237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:12:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:19:04.635402507 +0000 UTC m=+376.811838528" watchObservedRunningTime="2026-09-13 20:19:04.636794352 +0000 UTC m=+376.813230373" Sep 13 20:19:20.030994 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:20.030948 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86d5d78598-2wq8m"] Sep 13 20:19:45.049969 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.049895 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-86d5d78598-2wq8m" podUID="687568a6-db54-413f-8894-8f0717c81ee4" containerName="console" containerID="cri-o://71b23657b4ce1695e762354683a5889585abb4ae64959b17f9120c8ef485c512" gracePeriod=15 Sep 13 20:19:45.291256 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.291234 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:19:45.424764 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.424677 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/687568a6-db54-413f-8894-8f0717c81ee4-trusted-ca-bundle\") pod \"687568a6-db54-413f-8894-8f0717c81ee4\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " Sep 13 20:19:45.424764 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.424732 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/687568a6-db54-413f-8894-8f0717c81ee4-console-oauth-config\") pod \"687568a6-db54-413f-8894-8f0717c81ee4\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " Sep 13 20:19:45.424764 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.424758 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c79vp\" (UniqueName: \"kubernetes.io/projected/687568a6-db54-413f-8894-8f0717c81ee4-kube-api-access-c79vp\") pod \"687568a6-db54-413f-8894-8f0717c81ee4\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " Sep 13 20:19:45.425022 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.424773 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/687568a6-db54-413f-8894-8f0717c81ee4-console-serving-cert\") pod \"687568a6-db54-413f-8894-8f0717c81ee4\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " Sep 13 20:19:45.425022 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.424812 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/687568a6-db54-413f-8894-8f0717c81ee4-console-config\") pod \"687568a6-db54-413f-8894-8f0717c81ee4\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " Sep 13 20:19:45.425022 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.424837 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/687568a6-db54-413f-8894-8f0717c81ee4-service-ca\") pod \"687568a6-db54-413f-8894-8f0717c81ee4\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " Sep 13 20:19:45.425022 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.424871 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/687568a6-db54-413f-8894-8f0717c81ee4-oauth-serving-cert\") pod \"687568a6-db54-413f-8894-8f0717c81ee4\" (UID: \"687568a6-db54-413f-8894-8f0717c81ee4\") " Sep 13 20:19:45.425211 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.425187 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/687568a6-db54-413f-8894-8f0717c81ee4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "687568a6-db54-413f-8894-8f0717c81ee4" (UID: "687568a6-db54-413f-8894-8f0717c81ee4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 13 20:19:45.425428 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.425307 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/687568a6-db54-413f-8894-8f0717c81ee4-service-ca" (OuterVolumeSpecName: "service-ca") pod "687568a6-db54-413f-8894-8f0717c81ee4" (UID: "687568a6-db54-413f-8894-8f0717c81ee4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 13 20:19:45.425428 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.425331 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/687568a6-db54-413f-8894-8f0717c81ee4-console-config" (OuterVolumeSpecName: "console-config") pod "687568a6-db54-413f-8894-8f0717c81ee4" (UID: "687568a6-db54-413f-8894-8f0717c81ee4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 13 20:19:45.425428 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.425395 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/687568a6-db54-413f-8894-8f0717c81ee4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "687568a6-db54-413f-8894-8f0717c81ee4" (UID: "687568a6-db54-413f-8894-8f0717c81ee4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 13 20:19:45.427130 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.427103 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/687568a6-db54-413f-8894-8f0717c81ee4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "687568a6-db54-413f-8894-8f0717c81ee4" (UID: "687568a6-db54-413f-8894-8f0717c81ee4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 20:19:45.427130 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.427109 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/687568a6-db54-413f-8894-8f0717c81ee4-kube-api-access-c79vp" (OuterVolumeSpecName: "kube-api-access-c79vp") pod "687568a6-db54-413f-8894-8f0717c81ee4" (UID: "687568a6-db54-413f-8894-8f0717c81ee4"). InnerVolumeSpecName "kube-api-access-c79vp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 20:19:45.427256 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.427124 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/687568a6-db54-413f-8894-8f0717c81ee4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "687568a6-db54-413f-8894-8f0717c81ee4" (UID: "687568a6-db54-413f-8894-8f0717c81ee4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 20:19:45.525795 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.525759 2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/687568a6-db54-413f-8894-8f0717c81ee4-console-oauth-config\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:19:45.525795 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.525789 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c79vp\" (UniqueName: \"kubernetes.io/projected/687568a6-db54-413f-8894-8f0717c81ee4-kube-api-access-c79vp\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:19:45.525795 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.525800 2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/687568a6-db54-413f-8894-8f0717c81ee4-console-serving-cert\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:19:45.526004 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.525810 2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/687568a6-db54-413f-8894-8f0717c81ee4-console-config\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:19:45.526004 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.525820 2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/687568a6-db54-413f-8894-8f0717c81ee4-service-ca\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:19:45.526004 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.525828 2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/687568a6-db54-413f-8894-8f0717c81ee4-oauth-serving-cert\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:19:45.526004 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.525836 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/687568a6-db54-413f-8894-8f0717c81ee4-trusted-ca-bundle\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:19:45.769286 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.769203 2557 generic.go:356] "Generic (PLEG): container finished" podID="687568a6-db54-413f-8894-8f0717c81ee4" containerID="71b23657b4ce1695e762354683a5889585abb4ae64959b17f9120c8ef485c512" exitCode=0 Sep 13 20:19:45.769286 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.769259 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86d5d78598-2wq8m" Sep 13 20:19:45.769460 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.769293 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86d5d78598-2wq8m" event={"ID":"687568a6-db54-413f-8894-8f0717c81ee4","Type":"ContainerDied","Data":"71b23657b4ce1695e762354683a5889585abb4ae64959b17f9120c8ef485c512"} Sep 13 20:19:45.769460 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.769346 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86d5d78598-2wq8m" event={"ID":"687568a6-db54-413f-8894-8f0717c81ee4","Type":"ContainerDied","Data":"12a75b82a7f25f2bc985e96a3baa32c41193f0565b652fd84a7ba5e7974da74f"} Sep 13 20:19:45.769460 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.769373 2557 scope.go:117] "RemoveContainer" containerID="71b23657b4ce1695e762354683a5889585abb4ae64959b17f9120c8ef485c512" Sep 13 20:19:45.777894 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.777876 2557 scope.go:117] "RemoveContainer" containerID="71b23657b4ce1695e762354683a5889585abb4ae64959b17f9120c8ef485c512" Sep 13 20:19:45.778162 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:19:45.778144 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71b23657b4ce1695e762354683a5889585abb4ae64959b17f9120c8ef485c512\": container with ID starting with 71b23657b4ce1695e762354683a5889585abb4ae64959b17f9120c8ef485c512 not found: ID does not exist" containerID="71b23657b4ce1695e762354683a5889585abb4ae64959b17f9120c8ef485c512" Sep 13 20:19:45.778211 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.778170 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71b23657b4ce1695e762354683a5889585abb4ae64959b17f9120c8ef485c512"} err="failed to get container status \"71b23657b4ce1695e762354683a5889585abb4ae64959b17f9120c8ef485c512\": rpc error: code = NotFound desc = could not find container \"71b23657b4ce1695e762354683a5889585abb4ae64959b17f9120c8ef485c512\": container with ID starting with 71b23657b4ce1695e762354683a5889585abb4ae64959b17f9120c8ef485c512 not found: ID does not exist" Sep 13 20:19:45.788527 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.788488 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86d5d78598-2wq8m"] Sep 13 20:19:45.790784 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:45.790764 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-86d5d78598-2wq8m"] Sep 13 20:19:46.311027 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:46.310990 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="687568a6-db54-413f-8894-8f0717c81ee4" path="/var/lib/kubelet/pods/687568a6-db54-413f-8894-8f0717c81ee4/volumes" Sep 13 20:19:58.694319 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:58.694265 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd"] Sep 13 20:19:58.694785 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:58.694649 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd0e27d4-cbed-4619-94be-5c9f8eb73fc2" containerName="registry" Sep 13 20:19:58.694785 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:58.694667 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd0e27d4-cbed-4619-94be-5c9f8eb73fc2" containerName="registry" Sep 13 20:19:58.694785 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:58.694678 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="687568a6-db54-413f-8894-8f0717c81ee4" containerName="console" Sep 13 20:19:58.694785 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:58.694683 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="687568a6-db54-413f-8894-8f0717c81ee4" containerName="console" Sep 13 20:19:58.694785 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:58.694738 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd0e27d4-cbed-4619-94be-5c9f8eb73fc2" containerName="registry" Sep 13 20:19:58.694785 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:58.694748 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="687568a6-db54-413f-8894-8f0717c81ee4" containerName="console" Sep 13 20:19:58.699132 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:58.699116 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd" Sep 13 20:19:58.701321 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:58.701298 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 13 20:19:58.701433 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:58.701372 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 13 20:19:58.701512 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:58.701467 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-w8j7b\"" Sep 13 20:19:58.706388 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:58.706364 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd"] Sep 13 20:19:58.829825 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:58.829787 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0f8c907-1983-4437-8284-c7cc9ce62e1a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd\" (UID: \"c0f8c907-1983-4437-8284-c7cc9ce62e1a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd" Sep 13 20:19:58.830025 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:58.829831 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flwgm\" (UniqueName: \"kubernetes.io/projected/c0f8c907-1983-4437-8284-c7cc9ce62e1a-kube-api-access-flwgm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd\" (UID: \"c0f8c907-1983-4437-8284-c7cc9ce62e1a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd" Sep 13 20:19:58.830025 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:58.829933 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0f8c907-1983-4437-8284-c7cc9ce62e1a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd\" (UID: \"c0f8c907-1983-4437-8284-c7cc9ce62e1a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd" Sep 13 20:19:58.931108 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:58.931073 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0f8c907-1983-4437-8284-c7cc9ce62e1a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd\" (UID: \"c0f8c907-1983-4437-8284-c7cc9ce62e1a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd" Sep 13 20:19:58.931264 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:58.931127 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0f8c907-1983-4437-8284-c7cc9ce62e1a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd\" (UID: \"c0f8c907-1983-4437-8284-c7cc9ce62e1a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd" Sep 13 20:19:58.931264 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:58.931163 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flwgm\" (UniqueName: \"kubernetes.io/projected/c0f8c907-1983-4437-8284-c7cc9ce62e1a-kube-api-access-flwgm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd\" (UID: \"c0f8c907-1983-4437-8284-c7cc9ce62e1a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd" Sep 13 20:19:58.931479 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:58.931455 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0f8c907-1983-4437-8284-c7cc9ce62e1a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd\" (UID: \"c0f8c907-1983-4437-8284-c7cc9ce62e1a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd" Sep 13 20:19:58.931579 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:58.931488 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0f8c907-1983-4437-8284-c7cc9ce62e1a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd\" (UID: \"c0f8c907-1983-4437-8284-c7cc9ce62e1a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd" Sep 13 20:19:58.938806 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:58.938787 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flwgm\" (UniqueName: \"kubernetes.io/projected/c0f8c907-1983-4437-8284-c7cc9ce62e1a-kube-api-access-flwgm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd\" (UID: \"c0f8c907-1983-4437-8284-c7cc9ce62e1a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd" Sep 13 20:19:59.008608 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:59.008532 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd" Sep 13 20:19:59.276416 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:59.276391 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd"] Sep 13 20:19:59.278771 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:19:59.278742 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0f8c907_1983_4437_8284_c7cc9ce62e1a.slice/crio-292f58299063ff3072c77ba2e2f19683c5b940edf7a1f463e54ddaf02e75b1aa WatchSource:0}: Error finding container 292f58299063ff3072c77ba2e2f19683c5b940edf7a1f463e54ddaf02e75b1aa: Status 404 returned error can't find the container with id 292f58299063ff3072c77ba2e2f19683c5b940edf7a1f463e54ddaf02e75b1aa Sep 13 20:19:59.818638 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:19:59.818603 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd" event={"ID":"c0f8c907-1983-4437-8284-c7cc9ce62e1a","Type":"ContainerStarted","Data":"292f58299063ff3072c77ba2e2f19683c5b940edf7a1f463e54ddaf02e75b1aa"} Sep 13 20:20:05.842167 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:05.842130 2557 generic.go:356] "Generic (PLEG): container finished" podID="c0f8c907-1983-4437-8284-c7cc9ce62e1a" containerID="3e45936b9f1960f4dcbd9d8908eb79e268594605bf05f7391f2ea9bb7e0190f0" exitCode=0 Sep 13 20:20:05.842644 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:05.842215 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd" event={"ID":"c0f8c907-1983-4437-8284-c7cc9ce62e1a","Type":"ContainerDied","Data":"3e45936b9f1960f4dcbd9d8908eb79e268594605bf05f7391f2ea9bb7e0190f0"} Sep 13 20:20:14.877765 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:14.877731 2557 generic.go:356] "Generic (PLEG): container finished" podID="c0f8c907-1983-4437-8284-c7cc9ce62e1a" containerID="bde8a72dabdfca08093f44f0af5ca29abce4254195f579618c945b0f54f66ea5" exitCode=0 Sep 13 20:20:14.878166 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:14.877821 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd" event={"ID":"c0f8c907-1983-4437-8284-c7cc9ce62e1a","Type":"ContainerDied","Data":"bde8a72dabdfca08093f44f0af5ca29abce4254195f579618c945b0f54f66ea5"} Sep 13 20:20:24.918214 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:24.918177 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd" event={"ID":"c0f8c907-1983-4437-8284-c7cc9ce62e1a","Type":"ContainerStarted","Data":"4dd10718cc69c05b3469fbb521328b2c4586e7bf81f8397935167effd3817a68"} Sep 13 20:20:24.934424 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:24.934368 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd" podStartSLOduration=2.058781397 podStartE2EDuration="26.934351927s" totalImagesPullingTime="24.87557053s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:19:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:20:24.934087142 +0000 UTC m=+457.110523174" watchObservedRunningTime="2026-09-13 20:20:24.934351927 +0000 UTC m=+457.110787950" Sep 13 20:20:25.923963 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:25.923930 2557 generic.go:356] "Generic (PLEG): container finished" podID="c0f8c907-1983-4437-8284-c7cc9ce62e1a" containerID="4dd10718cc69c05b3469fbb521328b2c4586e7bf81f8397935167effd3817a68" exitCode=0 Sep 13 20:20:25.924344 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:25.923988 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd" event={"ID":"c0f8c907-1983-4437-8284-c7cc9ce62e1a","Type":"ContainerDied","Data":"4dd10718cc69c05b3469fbb521328b2c4586e7bf81f8397935167effd3817a68"} Sep 13 20:20:27.051222 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:27.051197 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd" Sep 13 20:20:27.145122 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:27.145086 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-flwgm\" (UniqueName: \"kubernetes.io/projected/c0f8c907-1983-4437-8284-c7cc9ce62e1a-kube-api-access-flwgm\") pod \"c0f8c907-1983-4437-8284-c7cc9ce62e1a\" (UID: \"c0f8c907-1983-4437-8284-c7cc9ce62e1a\") " Sep 13 20:20:27.145288 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:27.145136 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0f8c907-1983-4437-8284-c7cc9ce62e1a-bundle\") pod \"c0f8c907-1983-4437-8284-c7cc9ce62e1a\" (UID: \"c0f8c907-1983-4437-8284-c7cc9ce62e1a\") " Sep 13 20:20:27.145288 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:27.145161 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0f8c907-1983-4437-8284-c7cc9ce62e1a-util\") pod \"c0f8c907-1983-4437-8284-c7cc9ce62e1a\" (UID: \"c0f8c907-1983-4437-8284-c7cc9ce62e1a\") " Sep 13 20:20:27.145808 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:27.145785 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0f8c907-1983-4437-8284-c7cc9ce62e1a-bundle" (OuterVolumeSpecName: "bundle") pod "c0f8c907-1983-4437-8284-c7cc9ce62e1a" (UID: "c0f8c907-1983-4437-8284-c7cc9ce62e1a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 13 20:20:27.147419 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:27.147393 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0f8c907-1983-4437-8284-c7cc9ce62e1a-kube-api-access-flwgm" (OuterVolumeSpecName: "kube-api-access-flwgm") pod "c0f8c907-1983-4437-8284-c7cc9ce62e1a" (UID: "c0f8c907-1983-4437-8284-c7cc9ce62e1a"). InnerVolumeSpecName "kube-api-access-flwgm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 20:20:27.149840 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:27.149818 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0f8c907-1983-4437-8284-c7cc9ce62e1a-util" (OuterVolumeSpecName: "util") pod "c0f8c907-1983-4437-8284-c7cc9ce62e1a" (UID: "c0f8c907-1983-4437-8284-c7cc9ce62e1a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 13 20:20:27.246157 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:27.246077 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flwgm\" (UniqueName: \"kubernetes.io/projected/c0f8c907-1983-4437-8284-c7cc9ce62e1a-kube-api-access-flwgm\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:20:27.246157 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:27.246106 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0f8c907-1983-4437-8284-c7cc9ce62e1a-bundle\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:20:27.246157 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:27.246115 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0f8c907-1983-4437-8284-c7cc9ce62e1a-util\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:20:27.933820 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:27.933793 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd" Sep 13 20:20:27.933820 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:27.933808 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m9cd" event={"ID":"c0f8c907-1983-4437-8284-c7cc9ce62e1a","Type":"ContainerDied","Data":"292f58299063ff3072c77ba2e2f19683c5b940edf7a1f463e54ddaf02e75b1aa"} Sep 13 20:20:27.934023 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:27.933858 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="292f58299063ff3072c77ba2e2f19683c5b940edf7a1f463e54ddaf02e75b1aa" Sep 13 20:20:31.678873 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:31.678836 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bchvv"] Sep 13 20:20:31.679237 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:31.679103 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0f8c907-1983-4437-8284-c7cc9ce62e1a" containerName="util" Sep 13 20:20:31.679237 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:31.679114 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0f8c907-1983-4437-8284-c7cc9ce62e1a" containerName="util" Sep 13 20:20:31.679237 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:31.679129 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0f8c907-1983-4437-8284-c7cc9ce62e1a" containerName="extract" Sep 13 20:20:31.679237 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:31.679134 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0f8c907-1983-4437-8284-c7cc9ce62e1a" containerName="extract" Sep 13 20:20:31.679237 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:31.679146 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0f8c907-1983-4437-8284-c7cc9ce62e1a" containerName="pull" Sep 13 20:20:31.679237 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:31.679152 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0f8c907-1983-4437-8284-c7cc9ce62e1a" containerName="pull" Sep 13 20:20:31.679237 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:31.679193 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0f8c907-1983-4437-8284-c7cc9ce62e1a" containerName="extract" Sep 13 20:20:31.682196 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:31.682178 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bchvv" Sep 13 20:20:31.685352 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:31.685323 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Sep 13 20:20:31.685352 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:31.685347 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Sep 13 20:20:31.685529 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:31.685347 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-29bw9\"" Sep 13 20:20:31.695211 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:31.695190 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bchvv"] Sep 13 20:20:31.779306 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:31.779268 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23069766-0f98-47f0-8ac0-b5de83413386-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bchvv\" (UID: \"23069766-0f98-47f0-8ac0-b5de83413386\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bchvv" Sep 13 20:20:31.779306 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:31.779309 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb49b\" (UniqueName: \"kubernetes.io/projected/23069766-0f98-47f0-8ac0-b5de83413386-kube-api-access-vb49b\") pod \"cert-manager-operator-controller-manager-55c444bf86-bchvv\" (UID: \"23069766-0f98-47f0-8ac0-b5de83413386\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bchvv" Sep 13 20:20:31.879754 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:31.879726 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23069766-0f98-47f0-8ac0-b5de83413386-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bchvv\" (UID: \"23069766-0f98-47f0-8ac0-b5de83413386\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bchvv" Sep 13 20:20:31.879848 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:31.879766 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb49b\" (UniqueName: \"kubernetes.io/projected/23069766-0f98-47f0-8ac0-b5de83413386-kube-api-access-vb49b\") pod \"cert-manager-operator-controller-manager-55c444bf86-bchvv\" (UID: \"23069766-0f98-47f0-8ac0-b5de83413386\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bchvv" Sep 13 20:20:31.880115 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:31.880095 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23069766-0f98-47f0-8ac0-b5de83413386-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bchvv\" (UID: \"23069766-0f98-47f0-8ac0-b5de83413386\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bchvv" Sep 13 20:20:31.892872 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:31.892842 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb49b\" (UniqueName: \"kubernetes.io/projected/23069766-0f98-47f0-8ac0-b5de83413386-kube-api-access-vb49b\") pod \"cert-manager-operator-controller-manager-55c444bf86-bchvv\" (UID: \"23069766-0f98-47f0-8ac0-b5de83413386\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bchvv" Sep 13 20:20:31.991282 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:31.991206 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bchvv" Sep 13 20:20:32.261682 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:32.261554 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bchvv"] Sep 13 20:20:32.266126 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:20:32.266089 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23069766_0f98_47f0_8ac0_b5de83413386.slice/crio-d3ab36c88578f8c5ff2ff96e9f928a080bf9f7e36365ecc1b3e69f3b1a62aeba WatchSource:0}: Error finding container d3ab36c88578f8c5ff2ff96e9f928a080bf9f7e36365ecc1b3e69f3b1a62aeba: Status 404 returned error can't find the container with id d3ab36c88578f8c5ff2ff96e9f928a080bf9f7e36365ecc1b3e69f3b1a62aeba Sep 13 20:20:32.954591 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:32.954553 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bchvv" event={"ID":"23069766-0f98-47f0-8ac0-b5de83413386","Type":"ContainerStarted","Data":"d3ab36c88578f8c5ff2ff96e9f928a080bf9f7e36365ecc1b3e69f3b1a62aeba"} Sep 13 20:20:33.960071 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:33.960035 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bchvv" event={"ID":"23069766-0f98-47f0-8ac0-b5de83413386","Type":"ContainerStarted","Data":"6d40a131e4bb6dfbc7d91df7aa613d33d4dee46a816fa3a38a1c3ac8ecab2daf"} Sep 13 20:20:33.985824 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:33.985768 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bchvv" podStartSLOduration=1.430093207 podStartE2EDuration="2.985753246s" totalImagesPullingTime="1.555660039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:20:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:20:33.98302828 +0000 UTC m=+466.159464303" watchObservedRunningTime="2026-09-13 20:20:33.985753246 +0000 UTC m=+466.162189268" Sep 13 20:20:35.172165 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.172123 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz"] Sep 13 20:20:35.175659 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.175642 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz" Sep 13 20:20:35.177917 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.177895 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-w8j7b\"" Sep 13 20:20:35.177917 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.177895 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 13 20:20:35.178649 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.178631 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 13 20:20:35.186939 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.186918 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz"] Sep 13 20:20:35.206791 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.206770 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1fe5216f-32d9-4768-82ca-e8d0de034a6b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz\" (UID: \"1fe5216f-32d9-4768-82ca-e8d0de034a6b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz" Sep 13 20:20:35.206919 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.206810 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1fe5216f-32d9-4768-82ca-e8d0de034a6b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz\" (UID: \"1fe5216f-32d9-4768-82ca-e8d0de034a6b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz" Sep 13 20:20:35.206919 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.206843 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2kz4\" (UniqueName: \"kubernetes.io/projected/1fe5216f-32d9-4768-82ca-e8d0de034a6b-kube-api-access-s2kz4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz\" (UID: \"1fe5216f-32d9-4768-82ca-e8d0de034a6b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz" Sep 13 20:20:35.308087 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.308059 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1fe5216f-32d9-4768-82ca-e8d0de034a6b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz\" (UID: \"1fe5216f-32d9-4768-82ca-e8d0de034a6b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz" Sep 13 20:20:35.308237 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.308113 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1fe5216f-32d9-4768-82ca-e8d0de034a6b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz\" (UID: \"1fe5216f-32d9-4768-82ca-e8d0de034a6b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz" Sep 13 20:20:35.308237 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.308150 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2kz4\" (UniqueName: \"kubernetes.io/projected/1fe5216f-32d9-4768-82ca-e8d0de034a6b-kube-api-access-s2kz4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz\" (UID: \"1fe5216f-32d9-4768-82ca-e8d0de034a6b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz" Sep 13 20:20:35.308457 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.308436 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1fe5216f-32d9-4768-82ca-e8d0de034a6b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz\" (UID: \"1fe5216f-32d9-4768-82ca-e8d0de034a6b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz" Sep 13 20:20:35.308549 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.308521 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1fe5216f-32d9-4768-82ca-e8d0de034a6b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz\" (UID: \"1fe5216f-32d9-4768-82ca-e8d0de034a6b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz" Sep 13 20:20:35.316054 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.316030 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2kz4\" (UniqueName: \"kubernetes.io/projected/1fe5216f-32d9-4768-82ca-e8d0de034a6b-kube-api-access-s2kz4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz\" (UID: \"1fe5216f-32d9-4768-82ca-e8d0de034a6b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz" Sep 13 20:20:35.484239 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.484148 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz" Sep 13 20:20:35.656007 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.655977 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-2c574"] Sep 13 20:20:35.659347 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.659330 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-2c574" Sep 13 20:20:35.661897 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.661869 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 13 20:20:35.661897 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.661877 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 13 20:20:35.662041 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.661879 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-b4758\"" Sep 13 20:20:35.667212 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.667190 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-2c574"] Sep 13 20:20:35.711948 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.711921 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcmt2\" (UniqueName: \"kubernetes.io/projected/e1afd69a-1e13-4f77-b6e1-f59e6adc909a-kube-api-access-dcmt2\") pod \"cert-manager-webhook-6486ffd997-2c574\" (UID: \"e1afd69a-1e13-4f77-b6e1-f59e6adc909a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2c574" Sep 13 20:20:35.712084 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.711973 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e1afd69a-1e13-4f77-b6e1-f59e6adc909a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-2c574\" (UID: \"e1afd69a-1e13-4f77-b6e1-f59e6adc909a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2c574" Sep 13 20:20:35.754693 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.754668 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz"] Sep 13 20:20:35.756638 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:20:35.756615 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fe5216f_32d9_4768_82ca_e8d0de034a6b.slice/crio-8e6f5a95bb928ed5cb06814936816134ab0d0bb1a037d63d6cd4dcf62aba6714 WatchSource:0}: Error finding container 8e6f5a95bb928ed5cb06814936816134ab0d0bb1a037d63d6cd4dcf62aba6714: Status 404 returned error can't find the container with id 8e6f5a95bb928ed5cb06814936816134ab0d0bb1a037d63d6cd4dcf62aba6714 Sep 13 20:20:35.813294 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.813268 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcmt2\" (UniqueName: \"kubernetes.io/projected/e1afd69a-1e13-4f77-b6e1-f59e6adc909a-kube-api-access-dcmt2\") pod \"cert-manager-webhook-6486ffd997-2c574\" (UID: \"e1afd69a-1e13-4f77-b6e1-f59e6adc909a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2c574" Sep 13 20:20:35.813387 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.813320 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e1afd69a-1e13-4f77-b6e1-f59e6adc909a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-2c574\" (UID: \"e1afd69a-1e13-4f77-b6e1-f59e6adc909a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2c574" Sep 13 20:20:35.821164 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.821138 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcmt2\" (UniqueName: \"kubernetes.io/projected/e1afd69a-1e13-4f77-b6e1-f59e6adc909a-kube-api-access-dcmt2\") pod \"cert-manager-webhook-6486ffd997-2c574\" (UID: \"e1afd69a-1e13-4f77-b6e1-f59e6adc909a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2c574" Sep 13 20:20:35.821249 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.821169 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e1afd69a-1e13-4f77-b6e1-f59e6adc909a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-2c574\" (UID: \"e1afd69a-1e13-4f77-b6e1-f59e6adc909a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2c574" Sep 13 20:20:35.969577 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.969544 2557 generic.go:356] "Generic (PLEG): container finished" podID="1fe5216f-32d9-4768-82ca-e8d0de034a6b" containerID="224088b023c652358e649dc51801cbe00339d9416c227d46384fa957c5034222" exitCode=0 Sep 13 20:20:35.969723 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.969633 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz" event={"ID":"1fe5216f-32d9-4768-82ca-e8d0de034a6b","Type":"ContainerDied","Data":"224088b023c652358e649dc51801cbe00339d9416c227d46384fa957c5034222"} Sep 13 20:20:35.969723 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.969684 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz" event={"ID":"1fe5216f-32d9-4768-82ca-e8d0de034a6b","Type":"ContainerStarted","Data":"8e6f5a95bb928ed5cb06814936816134ab0d0bb1a037d63d6cd4dcf62aba6714"} Sep 13 20:20:35.976984 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:35.976965 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-2c574" Sep 13 20:20:36.243440 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:36.243414 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-2c574"] Sep 13 20:20:36.245511 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:20:36.245468 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1afd69a_1e13_4f77_b6e1_f59e6adc909a.slice/crio-d135085bce6983187dfb4ae03e687affdf8ba2a927100c03c28072b4db5a81d0 WatchSource:0}: Error finding container d135085bce6983187dfb4ae03e687affdf8ba2a927100c03c28072b4db5a81d0: Status 404 returned error can't find the container with id d135085bce6983187dfb4ae03e687affdf8ba2a927100c03c28072b4db5a81d0 Sep 13 20:20:36.976039 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:36.976005 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-2c574" event={"ID":"e1afd69a-1e13-4f77-b6e1-f59e6adc909a","Type":"ContainerStarted","Data":"d135085bce6983187dfb4ae03e687affdf8ba2a927100c03c28072b4db5a81d0"} Sep 13 20:20:38.988489 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:38.988450 2557 generic.go:356] "Generic (PLEG): container finished" podID="1fe5216f-32d9-4768-82ca-e8d0de034a6b" containerID="93c646475140049f1f5c054216ae3b21b5e40d9e08917c88e59d13a2b7d81f4d" exitCode=0 Sep 13 20:20:38.988890 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:38.988528 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz" event={"ID":"1fe5216f-32d9-4768-82ca-e8d0de034a6b","Type":"ContainerDied","Data":"93c646475140049f1f5c054216ae3b21b5e40d9e08917c88e59d13a2b7d81f4d"} Sep 13 20:20:39.996967 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:39.996933 2557 generic.go:356] "Generic (PLEG): container finished" podID="1fe5216f-32d9-4768-82ca-e8d0de034a6b" containerID="8f8153a938001e0f13a80851673c1ce0692898384a5c7a8483bd284340ef7493" exitCode=0 Sep 13 20:20:39.997396 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:39.997011 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz" event={"ID":"1fe5216f-32d9-4768-82ca-e8d0de034a6b","Type":"ContainerDied","Data":"8f8153a938001e0f13a80851673c1ce0692898384a5c7a8483bd284340ef7493"} Sep 13 20:20:41.002578 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:41.002538 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-2c574" event={"ID":"e1afd69a-1e13-4f77-b6e1-f59e6adc909a","Type":"ContainerStarted","Data":"4bc5648ebf568c87c7c64a5d406a234fcf618932dce6bfbe5f6b6ba6c1a4b6e0"} Sep 13 20:20:41.019618 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:41.019561 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-2c574" podStartSLOduration=1.784444395 podStartE2EDuration="6.019543233s" totalImagesPullingTime="4.235098838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:20:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:20:41.018695202 +0000 UTC m=+473.195131235" watchObservedRunningTime="2026-09-13 20:20:41.019543233 +0000 UTC m=+473.195979257" Sep 13 20:20:41.126567 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:41.126542 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz" Sep 13 20:20:41.159059 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:41.159029 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2kz4\" (UniqueName: \"kubernetes.io/projected/1fe5216f-32d9-4768-82ca-e8d0de034a6b-kube-api-access-s2kz4\") pod \"1fe5216f-32d9-4768-82ca-e8d0de034a6b\" (UID: \"1fe5216f-32d9-4768-82ca-e8d0de034a6b\") " Sep 13 20:20:41.159194 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:41.159083 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1fe5216f-32d9-4768-82ca-e8d0de034a6b-util\") pod \"1fe5216f-32d9-4768-82ca-e8d0de034a6b\" (UID: \"1fe5216f-32d9-4768-82ca-e8d0de034a6b\") " Sep 13 20:20:41.159194 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:41.159112 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1fe5216f-32d9-4768-82ca-e8d0de034a6b-bundle\") pod \"1fe5216f-32d9-4768-82ca-e8d0de034a6b\" (UID: \"1fe5216f-32d9-4768-82ca-e8d0de034a6b\") " Sep 13 20:20:41.159490 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:41.159466 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fe5216f-32d9-4768-82ca-e8d0de034a6b-bundle" (OuterVolumeSpecName: "bundle") pod "1fe5216f-32d9-4768-82ca-e8d0de034a6b" (UID: "1fe5216f-32d9-4768-82ca-e8d0de034a6b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 13 20:20:41.161165 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:41.161136 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fe5216f-32d9-4768-82ca-e8d0de034a6b-kube-api-access-s2kz4" (OuterVolumeSpecName: "kube-api-access-s2kz4") pod "1fe5216f-32d9-4768-82ca-e8d0de034a6b" (UID: "1fe5216f-32d9-4768-82ca-e8d0de034a6b"). InnerVolumeSpecName "kube-api-access-s2kz4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 20:20:41.163601 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:41.163575 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fe5216f-32d9-4768-82ca-e8d0de034a6b-util" (OuterVolumeSpecName: "util") pod "1fe5216f-32d9-4768-82ca-e8d0de034a6b" (UID: "1fe5216f-32d9-4768-82ca-e8d0de034a6b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 13 20:20:41.260606 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:41.260524 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2kz4\" (UniqueName: \"kubernetes.io/projected/1fe5216f-32d9-4768-82ca-e8d0de034a6b-kube-api-access-s2kz4\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:20:41.260606 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:41.260552 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1fe5216f-32d9-4768-82ca-e8d0de034a6b-util\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:20:41.260606 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:41.260561 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1fe5216f-32d9-4768-82ca-e8d0de034a6b-bundle\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:20:42.008189 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:42.008158 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz" Sep 13 20:20:42.008594 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:42.008187 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb9brz" event={"ID":"1fe5216f-32d9-4768-82ca-e8d0de034a6b","Type":"ContainerDied","Data":"8e6f5a95bb928ed5cb06814936816134ab0d0bb1a037d63d6cd4dcf62aba6714"} Sep 13 20:20:42.008594 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:42.008238 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8e6f5a95bb928ed5cb06814936816134ab0d0bb1a037d63d6cd4dcf62aba6714" Sep 13 20:20:42.008594 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:42.008532 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-2c574" Sep 13 20:20:43.848040 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:43.848003 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-vg2sv"] Sep 13 20:20:43.848413 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:43.848306 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fe5216f-32d9-4768-82ca-e8d0de034a6b" containerName="util" Sep 13 20:20:43.848413 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:43.848322 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fe5216f-32d9-4768-82ca-e8d0de034a6b" containerName="util" Sep 13 20:20:43.848413 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:43.848333 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fe5216f-32d9-4768-82ca-e8d0de034a6b" containerName="pull" Sep 13 20:20:43.848413 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:43.848338 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fe5216f-32d9-4768-82ca-e8d0de034a6b" containerName="pull" Sep 13 20:20:43.848413 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:43.848357 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fe5216f-32d9-4768-82ca-e8d0de034a6b" containerName="extract" Sep 13 20:20:43.848413 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:43.848362 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fe5216f-32d9-4768-82ca-e8d0de034a6b" containerName="extract" Sep 13 20:20:43.848623 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:43.848421 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fe5216f-32d9-4768-82ca-e8d0de034a6b" containerName="extract" Sep 13 20:20:43.851372 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:43.851356 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vg2sv" Sep 13 20:20:43.853823 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:43.853803 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-l9tnk\"" Sep 13 20:20:43.858255 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:43.858234 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-vg2sv"] Sep 13 20:20:43.882687 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:43.882657 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3dbcff7c-b94f-4887-8911-91206cde685f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-vg2sv\" (UID: \"3dbcff7c-b94f-4887-8911-91206cde685f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vg2sv" Sep 13 20:20:43.882831 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:43.882697 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xttp8\" (UniqueName: \"kubernetes.io/projected/3dbcff7c-b94f-4887-8911-91206cde685f-kube-api-access-xttp8\") pod \"cert-manager-cainjector-559cdd8c87-vg2sv\" (UID: \"3dbcff7c-b94f-4887-8911-91206cde685f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vg2sv" Sep 13 20:20:43.984056 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:43.984017 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3dbcff7c-b94f-4887-8911-91206cde685f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-vg2sv\" (UID: \"3dbcff7c-b94f-4887-8911-91206cde685f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vg2sv" Sep 13 20:20:43.984229 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:43.984068 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xttp8\" (UniqueName: \"kubernetes.io/projected/3dbcff7c-b94f-4887-8911-91206cde685f-kube-api-access-xttp8\") pod \"cert-manager-cainjector-559cdd8c87-vg2sv\" (UID: \"3dbcff7c-b94f-4887-8911-91206cde685f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vg2sv" Sep 13 20:20:43.991681 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:43.991646 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3dbcff7c-b94f-4887-8911-91206cde685f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-vg2sv\" (UID: \"3dbcff7c-b94f-4887-8911-91206cde685f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vg2sv" Sep 13 20:20:43.991912 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:43.991890 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xttp8\" (UniqueName: \"kubernetes.io/projected/3dbcff7c-b94f-4887-8911-91206cde685f-kube-api-access-xttp8\") pod \"cert-manager-cainjector-559cdd8c87-vg2sv\" (UID: \"3dbcff7c-b94f-4887-8911-91206cde685f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vg2sv" Sep 13 20:20:44.161542 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:44.161438 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vg2sv" Sep 13 20:20:44.430963 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:44.430789 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-vg2sv"] Sep 13 20:20:44.433443 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:20:44.433419 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3dbcff7c_b94f_4887_8911_91206cde685f.slice/crio-0c5d9969b8f323c58d6a360cc1552fe2f0c1ca2e9b52a6074ce4c7cf22490081 WatchSource:0}: Error finding container 0c5d9969b8f323c58d6a360cc1552fe2f0c1ca2e9b52a6074ce4c7cf22490081: Status 404 returned error can't find the container with id 0c5d9969b8f323c58d6a360cc1552fe2f0c1ca2e9b52a6074ce4c7cf22490081 Sep 13 20:20:45.021640 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:45.021606 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vg2sv" event={"ID":"3dbcff7c-b94f-4887-8911-91206cde685f","Type":"ContainerStarted","Data":"5bd5781e117d24e7a0c4dbb2c57080f0638ef2574da55413b8fb4b1a665b62d9"} Sep 13 20:20:45.021640 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:45.021645 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vg2sv" event={"ID":"3dbcff7c-b94f-4887-8911-91206cde685f","Type":"ContainerStarted","Data":"0c5d9969b8f323c58d6a360cc1552fe2f0c1ca2e9b52a6074ce4c7cf22490081"} Sep 13 20:20:45.036223 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:45.036158 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vg2sv" podStartSLOduration=2.036139782 podStartE2EDuration="2.036139782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:20:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:20:45.035770727 +0000 UTC m=+477.212206763" watchObservedRunningTime="2026-09-13 20:20:45.036139782 +0000 UTC m=+477.212575820" Sep 13 20:20:48.015124 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:48.015074 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-2c574" Sep 13 20:20:48.364289 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:48.364253 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8rbp7"] Sep 13 20:20:48.371462 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:48.371440 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8rbp7" Sep 13 20:20:48.373674 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:48.373639 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 13 20:20:48.373811 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:48.373684 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 13 20:20:48.373811 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:48.373693 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-7ftjd\"" Sep 13 20:20:48.374294 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:48.374262 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8rbp7"] Sep 13 20:20:48.417823 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:48.417790 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2d0c129-2513-4406-af34-77b1769be0d0-tmp\") pod \"openshift-lws-operator-fd8ccff4c-8rbp7\" (UID: \"e2d0c129-2513-4406-af34-77b1769be0d0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8rbp7" Sep 13 20:20:48.417979 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:48.417828 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csq24\" (UniqueName: \"kubernetes.io/projected/e2d0c129-2513-4406-af34-77b1769be0d0-kube-api-access-csq24\") pod \"openshift-lws-operator-fd8ccff4c-8rbp7\" (UID: \"e2d0c129-2513-4406-af34-77b1769be0d0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8rbp7" Sep 13 20:20:48.519246 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:48.519210 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csq24\" (UniqueName: \"kubernetes.io/projected/e2d0c129-2513-4406-af34-77b1769be0d0-kube-api-access-csq24\") pod \"openshift-lws-operator-fd8ccff4c-8rbp7\" (UID: \"e2d0c129-2513-4406-af34-77b1769be0d0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8rbp7" Sep 13 20:20:48.519405 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:48.519303 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2d0c129-2513-4406-af34-77b1769be0d0-tmp\") pod \"openshift-lws-operator-fd8ccff4c-8rbp7\" (UID: \"e2d0c129-2513-4406-af34-77b1769be0d0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8rbp7" Sep 13 20:20:48.519689 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:48.519668 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2d0c129-2513-4406-af34-77b1769be0d0-tmp\") pod \"openshift-lws-operator-fd8ccff4c-8rbp7\" (UID: \"e2d0c129-2513-4406-af34-77b1769be0d0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8rbp7" Sep 13 20:20:48.526927 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:48.526896 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csq24\" (UniqueName: \"kubernetes.io/projected/e2d0c129-2513-4406-af34-77b1769be0d0-kube-api-access-csq24\") pod \"openshift-lws-operator-fd8ccff4c-8rbp7\" (UID: \"e2d0c129-2513-4406-af34-77b1769be0d0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8rbp7" Sep 13 20:20:48.682064 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:48.681968 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8rbp7" Sep 13 20:20:48.951177 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:48.951151 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8rbp7"] Sep 13 20:20:48.953779 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:20:48.953749 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode2d0c129_2513_4406_af34_77b1769be0d0.slice/crio-80e4e7c66c677f10ff30ebfd2b8fbd12fa6abde3e0428e8eba0bbc321643960b WatchSource:0}: Error finding container 80e4e7c66c677f10ff30ebfd2b8fbd12fa6abde3e0428e8eba0bbc321643960b: Status 404 returned error can't find the container with id 80e4e7c66c677f10ff30ebfd2b8fbd12fa6abde3e0428e8eba0bbc321643960b Sep 13 20:20:49.038967 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:49.038925 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8rbp7" event={"ID":"e2d0c129-2513-4406-af34-77b1769be0d0","Type":"ContainerStarted","Data":"80e4e7c66c677f10ff30ebfd2b8fbd12fa6abde3e0428e8eba0bbc321643960b"} Sep 13 20:20:51.051327 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:51.051242 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8rbp7" event={"ID":"e2d0c129-2513-4406-af34-77b1769be0d0","Type":"ContainerStarted","Data":"fed41424d644caadec93b68d96b4ab97507be6af8792f1d4ecb431d77ee0fe06"} Sep 13 20:20:51.067117 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:51.067068 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8rbp7" podStartSLOduration=1.234747433 podStartE2EDuration="3.067052719s" totalImagesPullingTime="1.832305286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:20:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:20:51.065838676 +0000 UTC m=+483.242274701" watchObservedRunningTime="2026-09-13 20:20:51.067052719 +0000 UTC m=+483.243488741" Sep 13 20:20:55.952367 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:55.952325 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h"] Sep 13 20:20:55.956290 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:55.956268 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h" Sep 13 20:20:55.958410 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:55.958384 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-w8j7b\"" Sep 13 20:20:55.958539 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:55.958447 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 13 20:20:55.959236 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:55.959222 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 13 20:20:55.962301 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:55.962276 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h"] Sep 13 20:20:56.074190 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:56.074159 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c878bd50-1eca-4422-b53e-90649025d8cb-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h\" (UID: \"c878bd50-1eca-4422-b53e-90649025d8cb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h" Sep 13 20:20:56.074356 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:56.074207 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9chs\" (UniqueName: \"kubernetes.io/projected/c878bd50-1eca-4422-b53e-90649025d8cb-kube-api-access-l9chs\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h\" (UID: \"c878bd50-1eca-4422-b53e-90649025d8cb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h" Sep 13 20:20:56.074356 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:56.074237 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c878bd50-1eca-4422-b53e-90649025d8cb-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h\" (UID: \"c878bd50-1eca-4422-b53e-90649025d8cb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h" Sep 13 20:20:56.174710 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:56.174670 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c878bd50-1eca-4422-b53e-90649025d8cb-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h\" (UID: \"c878bd50-1eca-4422-b53e-90649025d8cb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h" Sep 13 20:20:56.174874 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:56.174718 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9chs\" (UniqueName: \"kubernetes.io/projected/c878bd50-1eca-4422-b53e-90649025d8cb-kube-api-access-l9chs\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h\" (UID: \"c878bd50-1eca-4422-b53e-90649025d8cb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h" Sep 13 20:20:56.174874 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:56.174747 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c878bd50-1eca-4422-b53e-90649025d8cb-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h\" (UID: \"c878bd50-1eca-4422-b53e-90649025d8cb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h" Sep 13 20:20:56.175128 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:56.175113 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c878bd50-1eca-4422-b53e-90649025d8cb-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h\" (UID: \"c878bd50-1eca-4422-b53e-90649025d8cb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h" Sep 13 20:20:56.175176 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:56.175114 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c878bd50-1eca-4422-b53e-90649025d8cb-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h\" (UID: \"c878bd50-1eca-4422-b53e-90649025d8cb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h" Sep 13 20:20:56.182491 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:56.182458 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9chs\" (UniqueName: \"kubernetes.io/projected/c878bd50-1eca-4422-b53e-90649025d8cb-kube-api-access-l9chs\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h\" (UID: \"c878bd50-1eca-4422-b53e-90649025d8cb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h" Sep 13 20:20:56.266604 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:56.266518 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h" Sep 13 20:20:56.553253 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:56.553226 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h"] Sep 13 20:20:56.555478 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:20:56.555451 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc878bd50_1eca_4422_b53e_90649025d8cb.slice/crio-f210d43b562c98e4f793a221e61c26484d80fe4db0170e616a55af72c23418c3 WatchSource:0}: Error finding container f210d43b562c98e4f793a221e61c26484d80fe4db0170e616a55af72c23418c3: Status 404 returned error can't find the container with id f210d43b562c98e4f793a221e61c26484d80fe4db0170e616a55af72c23418c3 Sep 13 20:20:57.077259 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:57.077226 2557 generic.go:356] "Generic (PLEG): container finished" podID="c878bd50-1eca-4422-b53e-90649025d8cb" containerID="d55626acd566b681a2c4c887f3d469289bf6844be3ccd823acfde69f371b4b6d" exitCode=0 Sep 13 20:20:57.077672 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:57.077320 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h" event={"ID":"c878bd50-1eca-4422-b53e-90649025d8cb","Type":"ContainerDied","Data":"d55626acd566b681a2c4c887f3d469289bf6844be3ccd823acfde69f371b4b6d"} Sep 13 20:20:57.077672 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:57.077365 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h" event={"ID":"c878bd50-1eca-4422-b53e-90649025d8cb","Type":"ContainerStarted","Data":"f210d43b562c98e4f793a221e61c26484d80fe4db0170e616a55af72c23418c3"} Sep 13 20:20:59.087767 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:59.087731 2557 generic.go:356] "Generic (PLEG): container finished" podID="c878bd50-1eca-4422-b53e-90649025d8cb" containerID="0ff41f9789c07bd7a2f56c6db220ee4d746c8d773612d590e4e1124e118fc9fb" exitCode=0 Sep 13 20:20:59.088207 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:20:59.087784 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h" event={"ID":"c878bd50-1eca-4422-b53e-90649025d8cb","Type":"ContainerDied","Data":"0ff41f9789c07bd7a2f56c6db220ee4d746c8d773612d590e4e1124e118fc9fb"} Sep 13 20:21:00.094452 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:00.094419 2557 generic.go:356] "Generic (PLEG): container finished" podID="c878bd50-1eca-4422-b53e-90649025d8cb" containerID="a51dc9a191172a78b6e43ad543ffca9d694d3eaf5d645a94f43d9645a793711e" exitCode=0 Sep 13 20:21:00.094840 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:00.094475 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h" event={"ID":"c878bd50-1eca-4422-b53e-90649025d8cb","Type":"ContainerDied","Data":"a51dc9a191172a78b6e43ad543ffca9d694d3eaf5d645a94f43d9645a793711e"} Sep 13 20:21:01.221692 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:01.221670 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h" Sep 13 20:21:01.316488 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:01.316451 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9chs\" (UniqueName: \"kubernetes.io/projected/c878bd50-1eca-4422-b53e-90649025d8cb-kube-api-access-l9chs\") pod \"c878bd50-1eca-4422-b53e-90649025d8cb\" (UID: \"c878bd50-1eca-4422-b53e-90649025d8cb\") " Sep 13 20:21:01.316664 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:01.316490 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c878bd50-1eca-4422-b53e-90649025d8cb-util\") pod \"c878bd50-1eca-4422-b53e-90649025d8cb\" (UID: \"c878bd50-1eca-4422-b53e-90649025d8cb\") " Sep 13 20:21:01.316664 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:01.316549 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c878bd50-1eca-4422-b53e-90649025d8cb-bundle\") pod \"c878bd50-1eca-4422-b53e-90649025d8cb\" (UID: \"c878bd50-1eca-4422-b53e-90649025d8cb\") " Sep 13 20:21:01.317304 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:01.317273 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c878bd50-1eca-4422-b53e-90649025d8cb-bundle" (OuterVolumeSpecName: "bundle") pod "c878bd50-1eca-4422-b53e-90649025d8cb" (UID: "c878bd50-1eca-4422-b53e-90649025d8cb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 13 20:21:01.318641 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:01.318619 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c878bd50-1eca-4422-b53e-90649025d8cb-kube-api-access-l9chs" (OuterVolumeSpecName: "kube-api-access-l9chs") pod "c878bd50-1eca-4422-b53e-90649025d8cb" (UID: "c878bd50-1eca-4422-b53e-90649025d8cb"). InnerVolumeSpecName "kube-api-access-l9chs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 20:21:01.324004 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:01.323981 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c878bd50-1eca-4422-b53e-90649025d8cb-util" (OuterVolumeSpecName: "util") pod "c878bd50-1eca-4422-b53e-90649025d8cb" (UID: "c878bd50-1eca-4422-b53e-90649025d8cb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 13 20:21:01.417740 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:01.417634 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9chs\" (UniqueName: \"kubernetes.io/projected/c878bd50-1eca-4422-b53e-90649025d8cb-kube-api-access-l9chs\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:21:01.417740 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:01.417670 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c878bd50-1eca-4422-b53e-90649025d8cb-util\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:21:01.417740 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:01.417679 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c878bd50-1eca-4422-b53e-90649025d8cb-bundle\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:21:02.107400 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:02.107366 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h" event={"ID":"c878bd50-1eca-4422-b53e-90649025d8cb","Type":"ContainerDied","Data":"f210d43b562c98e4f793a221e61c26484d80fe4db0170e616a55af72c23418c3"} Sep 13 20:21:02.107576 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:02.107418 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f210d43b562c98e4f793a221e61c26484d80fe4db0170e616a55af72c23418c3" Sep 13 20:21:02.107576 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:02.107377 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff477h" Sep 13 20:21:11.007722 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:11.007686 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf"] Sep 13 20:21:11.008096 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:11.007981 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c878bd50-1eca-4422-b53e-90649025d8cb" containerName="extract" Sep 13 20:21:11.008096 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:11.007992 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c878bd50-1eca-4422-b53e-90649025d8cb" containerName="extract" Sep 13 20:21:11.008096 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:11.008003 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c878bd50-1eca-4422-b53e-90649025d8cb" containerName="util" Sep 13 20:21:11.008096 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:11.008009 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c878bd50-1eca-4422-b53e-90649025d8cb" containerName="util" Sep 13 20:21:11.008096 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:11.008017 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c878bd50-1eca-4422-b53e-90649025d8cb" containerName="pull" Sep 13 20:21:11.008096 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:11.008025 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c878bd50-1eca-4422-b53e-90649025d8cb" containerName="pull" Sep 13 20:21:11.008300 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:11.008102 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c878bd50-1eca-4422-b53e-90649025d8cb" containerName="extract" Sep 13 20:21:11.054378 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:11.054340 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf"] Sep 13 20:21:11.054559 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:11.054446 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf" Sep 13 20:21:11.056856 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:11.056831 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-w8j7b\"" Sep 13 20:21:11.057008 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:11.056831 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 13 20:21:11.057008 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:11.056882 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 13 20:21:11.200282 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:11.200248 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6tg4\" (UniqueName: \"kubernetes.io/projected/c64563d6-4c05-49c9-b040-0f33d75e14a1-kube-api-access-t6tg4\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf\" (UID: \"c64563d6-4c05-49c9-b040-0f33d75e14a1\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf" Sep 13 20:21:11.200282 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:11.200291 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c64563d6-4c05-49c9-b040-0f33d75e14a1-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf\" (UID: \"c64563d6-4c05-49c9-b040-0f33d75e14a1\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf" Sep 13 20:21:11.200512 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:11.200309 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c64563d6-4c05-49c9-b040-0f33d75e14a1-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf\" (UID: \"c64563d6-4c05-49c9-b040-0f33d75e14a1\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf" Sep 13 20:21:11.301173 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:11.301134 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6tg4\" (UniqueName: \"kubernetes.io/projected/c64563d6-4c05-49c9-b040-0f33d75e14a1-kube-api-access-t6tg4\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf\" (UID: \"c64563d6-4c05-49c9-b040-0f33d75e14a1\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf" Sep 13 20:21:11.301337 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:11.301179 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c64563d6-4c05-49c9-b040-0f33d75e14a1-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf\" (UID: \"c64563d6-4c05-49c9-b040-0f33d75e14a1\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf" Sep 13 20:21:11.301337 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:11.301210 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c64563d6-4c05-49c9-b040-0f33d75e14a1-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf\" (UID: \"c64563d6-4c05-49c9-b040-0f33d75e14a1\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf" Sep 13 20:21:11.301704 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:11.301636 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c64563d6-4c05-49c9-b040-0f33d75e14a1-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf\" (UID: \"c64563d6-4c05-49c9-b040-0f33d75e14a1\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf" Sep 13 20:21:11.301704 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:11.301671 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c64563d6-4c05-49c9-b040-0f33d75e14a1-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf\" (UID: \"c64563d6-4c05-49c9-b040-0f33d75e14a1\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf" Sep 13 20:21:11.310055 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:11.310023 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6tg4\" (UniqueName: \"kubernetes.io/projected/c64563d6-4c05-49c9-b040-0f33d75e14a1-kube-api-access-t6tg4\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf\" (UID: \"c64563d6-4c05-49c9-b040-0f33d75e14a1\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf" Sep 13 20:21:11.366016 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:11.365983 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf" Sep 13 20:21:11.667042 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:11.667016 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf"] Sep 13 20:21:11.669047 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:21:11.669002 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc64563d6_4c05_49c9_b040_0f33d75e14a1.slice/crio-434d9852fb61ac8d94e02476c5760a9ec310f874e0badb17500d1dde0eaab355 WatchSource:0}: Error finding container 434d9852fb61ac8d94e02476c5760a9ec310f874e0badb17500d1dde0eaab355: Status 404 returned error can't find the container with id 434d9852fb61ac8d94e02476c5760a9ec310f874e0badb17500d1dde0eaab355 Sep 13 20:21:12.154919 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:12.154883 2557 generic.go:356] "Generic (PLEG): container finished" podID="c64563d6-4c05-49c9-b040-0f33d75e14a1" containerID="10827fc2d4c70ba4684311e2b3b7798fe64062f2339cf8b31b10065ee6d32af3" exitCode=0 Sep 13 20:21:12.155299 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:12.154932 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf" event={"ID":"c64563d6-4c05-49c9-b040-0f33d75e14a1","Type":"ContainerDied","Data":"10827fc2d4c70ba4684311e2b3b7798fe64062f2339cf8b31b10065ee6d32af3"} Sep 13 20:21:12.155299 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:12.154966 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf" event={"ID":"c64563d6-4c05-49c9-b040-0f33d75e14a1","Type":"ContainerStarted","Data":"434d9852fb61ac8d94e02476c5760a9ec310f874e0badb17500d1dde0eaab355"} Sep 13 20:21:12.692413 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:12.692381 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84f9f8b966-n6gq9"] Sep 13 20:21:12.695870 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:12.695849 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84f9f8b966-n6gq9" Sep 13 20:21:12.698207 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:12.698179 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 13 20:21:12.698207 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:12.698209 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 13 20:21:12.698411 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:12.698265 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 13 20:21:12.698411 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:12.698265 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-q5p9w\"" Sep 13 20:21:12.698411 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:12.698265 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 13 20:21:12.710792 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:12.710767 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84f9f8b966-n6gq9"] Sep 13 20:21:12.813877 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:12.813854 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ca47fa53-4db9-4770-9c33-101323783338-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84f9f8b966-n6gq9\" (UID: \"ca47fa53-4db9-4770-9c33-101323783338\") " pod="opendatahub/opendatahub-operator-controller-manager-84f9f8b966-n6gq9" Sep 13 20:21:12.813973 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:12.813907 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca47fa53-4db9-4770-9c33-101323783338-tmp\") pod \"opendatahub-operator-controller-manager-84f9f8b966-n6gq9\" (UID: \"ca47fa53-4db9-4770-9c33-101323783338\") " pod="opendatahub/opendatahub-operator-controller-manager-84f9f8b966-n6gq9" Sep 13 20:21:12.814023 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:12.813979 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ca47fa53-4db9-4770-9c33-101323783338-webhook-cert\") pod \"opendatahub-operator-controller-manager-84f9f8b966-n6gq9\" (UID: \"ca47fa53-4db9-4770-9c33-101323783338\") " pod="opendatahub/opendatahub-operator-controller-manager-84f9f8b966-n6gq9" Sep 13 20:21:12.814023 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:12.814012 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wf84\" (UniqueName: \"kubernetes.io/projected/ca47fa53-4db9-4770-9c33-101323783338-kube-api-access-5wf84\") pod \"opendatahub-operator-controller-manager-84f9f8b966-n6gq9\" (UID: \"ca47fa53-4db9-4770-9c33-101323783338\") " pod="opendatahub/opendatahub-operator-controller-manager-84f9f8b966-n6gq9" Sep 13 20:21:12.814092 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:12.814036 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ca47fa53-4db9-4770-9c33-101323783338-manifests\") pod \"opendatahub-operator-controller-manager-84f9f8b966-n6gq9\" (UID: \"ca47fa53-4db9-4770-9c33-101323783338\") " pod="opendatahub/opendatahub-operator-controller-manager-84f9f8b966-n6gq9" Sep 13 20:21:12.915161 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:12.915120 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ca47fa53-4db9-4770-9c33-101323783338-webhook-cert\") pod \"opendatahub-operator-controller-manager-84f9f8b966-n6gq9\" (UID: \"ca47fa53-4db9-4770-9c33-101323783338\") " pod="opendatahub/opendatahub-operator-controller-manager-84f9f8b966-n6gq9" Sep 13 20:21:12.915161 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:12.915159 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wf84\" (UniqueName: \"kubernetes.io/projected/ca47fa53-4db9-4770-9c33-101323783338-kube-api-access-5wf84\") pod \"opendatahub-operator-controller-manager-84f9f8b966-n6gq9\" (UID: \"ca47fa53-4db9-4770-9c33-101323783338\") " pod="opendatahub/opendatahub-operator-controller-manager-84f9f8b966-n6gq9" Sep 13 20:21:12.915434 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:12.915186 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ca47fa53-4db9-4770-9c33-101323783338-manifests\") pod \"opendatahub-operator-controller-manager-84f9f8b966-n6gq9\" (UID: \"ca47fa53-4db9-4770-9c33-101323783338\") " pod="opendatahub/opendatahub-operator-controller-manager-84f9f8b966-n6gq9" Sep 13 20:21:12.915434 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:12.915231 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ca47fa53-4db9-4770-9c33-101323783338-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84f9f8b966-n6gq9\" (UID: \"ca47fa53-4db9-4770-9c33-101323783338\") " pod="opendatahub/opendatahub-operator-controller-manager-84f9f8b966-n6gq9" Sep 13 20:21:12.915434 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:12.915274 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca47fa53-4db9-4770-9c33-101323783338-tmp\") pod \"opendatahub-operator-controller-manager-84f9f8b966-n6gq9\" (UID: \"ca47fa53-4db9-4770-9c33-101323783338\") " pod="opendatahub/opendatahub-operator-controller-manager-84f9f8b966-n6gq9" Sep 13 20:21:12.915696 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:12.915666 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ca47fa53-4db9-4770-9c33-101323783338-manifests\") pod \"opendatahub-operator-controller-manager-84f9f8b966-n6gq9\" (UID: \"ca47fa53-4db9-4770-9c33-101323783338\") " pod="opendatahub/opendatahub-operator-controller-manager-84f9f8b966-n6gq9" Sep 13 20:21:12.917593 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:12.917571 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca47fa53-4db9-4770-9c33-101323783338-tmp\") pod \"opendatahub-operator-controller-manager-84f9f8b966-n6gq9\" (UID: \"ca47fa53-4db9-4770-9c33-101323783338\") " pod="opendatahub/opendatahub-operator-controller-manager-84f9f8b966-n6gq9" Sep 13 20:21:12.917915 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:12.917892 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ca47fa53-4db9-4770-9c33-101323783338-webhook-cert\") pod \"opendatahub-operator-controller-manager-84f9f8b966-n6gq9\" (UID: \"ca47fa53-4db9-4770-9c33-101323783338\") " pod="opendatahub/opendatahub-operator-controller-manager-84f9f8b966-n6gq9" Sep 13 20:21:12.917979 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:12.917892 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ca47fa53-4db9-4770-9c33-101323783338-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84f9f8b966-n6gq9\" (UID: \"ca47fa53-4db9-4770-9c33-101323783338\") " pod="opendatahub/opendatahub-operator-controller-manager-84f9f8b966-n6gq9" Sep 13 20:21:12.922232 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:12.922209 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wf84\" (UniqueName: \"kubernetes.io/projected/ca47fa53-4db9-4770-9c33-101323783338-kube-api-access-5wf84\") pod \"opendatahub-operator-controller-manager-84f9f8b966-n6gq9\" (UID: \"ca47fa53-4db9-4770-9c33-101323783338\") " pod="opendatahub/opendatahub-operator-controller-manager-84f9f8b966-n6gq9" Sep 13 20:21:13.006515 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:13.006410 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84f9f8b966-n6gq9" Sep 13 20:21:13.161561 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:13.161523 2557 generic.go:356] "Generic (PLEG): container finished" podID="c64563d6-4c05-49c9-b040-0f33d75e14a1" containerID="eee6bb9948cc947b648f56ce13dbb2033f3d21f968d23e376729f8fa811481b0" exitCode=0 Sep 13 20:21:13.162017 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:13.161576 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf" event={"ID":"c64563d6-4c05-49c9-b040-0f33d75e14a1","Type":"ContainerDied","Data":"eee6bb9948cc947b648f56ce13dbb2033f3d21f968d23e376729f8fa811481b0"} Sep 13 20:21:13.285707 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:13.285626 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84f9f8b966-n6gq9"] Sep 13 20:21:13.288250 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:21:13.288209 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca47fa53_4db9_4770_9c33_101323783338.slice/crio-5ab0c2f62105c23a8760e08a9d68c8c4961b5539fa8bfb71ba5176819b538d1a WatchSource:0}: Error finding container 5ab0c2f62105c23a8760e08a9d68c8c4961b5539fa8bfb71ba5176819b538d1a: Status 404 returned error can't find the container with id 5ab0c2f62105c23a8760e08a9d68c8c4961b5539fa8bfb71ba5176819b538d1a Sep 13 20:21:14.170160 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:14.170082 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84f9f8b966-n6gq9" event={"ID":"ca47fa53-4db9-4770-9c33-101323783338","Type":"ContainerStarted","Data":"5ab0c2f62105c23a8760e08a9d68c8c4961b5539fa8bfb71ba5176819b538d1a"} Sep 13 20:21:14.173261 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:14.173226 2557 generic.go:356] "Generic (PLEG): container finished" podID="c64563d6-4c05-49c9-b040-0f33d75e14a1" containerID="2a83252c46db1a1d80e44960c32020bded464206a26480c8cf064dd461059510" exitCode=0 Sep 13 20:21:14.173400 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:14.173299 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf" event={"ID":"c64563d6-4c05-49c9-b040-0f33d75e14a1","Type":"ContainerDied","Data":"2a83252c46db1a1d80e44960c32020bded464206a26480c8cf064dd461059510"} Sep 13 20:21:15.462407 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:15.462376 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf" Sep 13 20:21:15.536260 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:15.536230 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c64563d6-4c05-49c9-b040-0f33d75e14a1-util\") pod \"c64563d6-4c05-49c9-b040-0f33d75e14a1\" (UID: \"c64563d6-4c05-49c9-b040-0f33d75e14a1\") " Sep 13 20:21:15.536260 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:15.536268 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c64563d6-4c05-49c9-b040-0f33d75e14a1-bundle\") pod \"c64563d6-4c05-49c9-b040-0f33d75e14a1\" (UID: \"c64563d6-4c05-49c9-b040-0f33d75e14a1\") " Sep 13 20:21:15.536517 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:15.536299 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6tg4\" (UniqueName: \"kubernetes.io/projected/c64563d6-4c05-49c9-b040-0f33d75e14a1-kube-api-access-t6tg4\") pod \"c64563d6-4c05-49c9-b040-0f33d75e14a1\" (UID: \"c64563d6-4c05-49c9-b040-0f33d75e14a1\") " Sep 13 20:21:15.537176 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:15.537143 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c64563d6-4c05-49c9-b040-0f33d75e14a1-bundle" (OuterVolumeSpecName: "bundle") pod "c64563d6-4c05-49c9-b040-0f33d75e14a1" (UID: "c64563d6-4c05-49c9-b040-0f33d75e14a1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 13 20:21:15.538849 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:15.538818 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c64563d6-4c05-49c9-b040-0f33d75e14a1-kube-api-access-t6tg4" (OuterVolumeSpecName: "kube-api-access-t6tg4") pod "c64563d6-4c05-49c9-b040-0f33d75e14a1" (UID: "c64563d6-4c05-49c9-b040-0f33d75e14a1"). InnerVolumeSpecName "kube-api-access-t6tg4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 20:21:15.542955 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:15.542921 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c64563d6-4c05-49c9-b040-0f33d75e14a1-util" (OuterVolumeSpecName: "util") pod "c64563d6-4c05-49c9-b040-0f33d75e14a1" (UID: "c64563d6-4c05-49c9-b040-0f33d75e14a1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 13 20:21:15.636979 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:15.636935 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c64563d6-4c05-49c9-b040-0f33d75e14a1-util\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:21:15.636979 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:15.636974 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c64563d6-4c05-49c9-b040-0f33d75e14a1-bundle\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:21:15.636979 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:15.636986 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6tg4\" (UniqueName: \"kubernetes.io/projected/c64563d6-4c05-49c9-b040-0f33d75e14a1-kube-api-access-t6tg4\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:21:16.185670 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:16.185640 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf" event={"ID":"c64563d6-4c05-49c9-b040-0f33d75e14a1","Type":"ContainerDied","Data":"434d9852fb61ac8d94e02476c5760a9ec310f874e0badb17500d1dde0eaab355"} Sep 13 20:21:16.185848 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:16.185678 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nfcqf" Sep 13 20:21:16.185848 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:16.185693 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="434d9852fb61ac8d94e02476c5760a9ec310f874e0badb17500d1dde0eaab355" Sep 13 20:21:17.192326 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:17.192283 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84f9f8b966-n6gq9" event={"ID":"ca47fa53-4db9-4770-9c33-101323783338","Type":"ContainerStarted","Data":"efea707e8976512567e38234d0c0cd5610a81614bfa123fbe7a2a4ed97593ec0"} Sep 13 20:21:18.198341 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:18.198305 2557 generic.go:356] "Generic (PLEG): container finished" podID="ca47fa53-4db9-4770-9c33-101323783338" containerID="efea707e8976512567e38234d0c0cd5610a81614bfa123fbe7a2a4ed97593ec0" exitCode=0 Sep 13 20:21:18.198745 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:18.198366 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84f9f8b966-n6gq9" event={"ID":"ca47fa53-4db9-4770-9c33-101323783338","Type":"ContainerDied","Data":"efea707e8976512567e38234d0c0cd5610a81614bfa123fbe7a2a4ed97593ec0"} Sep 13 20:21:19.205142 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:19.205099 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84f9f8b966-n6gq9" event={"ID":"ca47fa53-4db9-4770-9c33-101323783338","Type":"ContainerStarted","Data":"94d8294555b65a749c34339d4fd130f3220ef39ab80b887ac8ef97b9f0631e81"} Sep 13 20:21:19.205577 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:19.205236 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-84f9f8b966-n6gq9" Sep 13 20:21:19.226740 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:19.226692 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-84f9f8b966-n6gq9" podStartSLOduration=2.646314856 podStartE2EDuration="7.226676387s" totalImagesPullingTime="3.507007522s" totalInitContainerRuntime="1.073354009s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:21:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:21:19.224896646 +0000 UTC m=+511.401332667" watchObservedRunningTime="2026-09-13 20:21:19.226676387 +0000 UTC m=+511.403112408" Sep 13 20:21:30.212609 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:30.212579 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-84f9f8b966-n6gq9" Sep 13 20:21:32.528531 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:32.528488 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-54976cff48-8rdkx"] Sep 13 20:21:32.528881 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:32.528807 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c64563d6-4c05-49c9-b040-0f33d75e14a1" containerName="util" Sep 13 20:21:32.528881 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:32.528819 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c64563d6-4c05-49c9-b040-0f33d75e14a1" containerName="util" Sep 13 20:21:32.528881 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:32.528841 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c64563d6-4c05-49c9-b040-0f33d75e14a1" containerName="pull" Sep 13 20:21:32.528881 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:32.528849 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c64563d6-4c05-49c9-b040-0f33d75e14a1" containerName="pull" Sep 13 20:21:32.528881 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:32.528857 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c64563d6-4c05-49c9-b040-0f33d75e14a1" containerName="extract" Sep 13 20:21:32.528881 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:32.528862 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c64563d6-4c05-49c9-b040-0f33d75e14a1" containerName="extract" Sep 13 20:21:32.529064 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:32.528918 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c64563d6-4c05-49c9-b040-0f33d75e14a1" containerName="extract" Sep 13 20:21:32.536478 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:32.536456 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-54976cff48-8rdkx" Sep 13 20:21:32.539197 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:32.539170 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 13 20:21:32.539427 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:32.539289 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 13 20:21:32.541859 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:32.541834 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-54976cff48-8rdkx"] Sep 13 20:21:32.575556 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:32.575524 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0720a4d9-96d6-48eb-839a-ec1f25d7bc56-tmp\") pod \"kube-auth-proxy-54976cff48-8rdkx\" (UID: \"0720a4d9-96d6-48eb-839a-ec1f25d7bc56\") " pod="openshift-ingress/kube-auth-proxy-54976cff48-8rdkx" Sep 13 20:21:32.575710 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:32.575561 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slskf\" (UniqueName: \"kubernetes.io/projected/0720a4d9-96d6-48eb-839a-ec1f25d7bc56-kube-api-access-slskf\") pod \"kube-auth-proxy-54976cff48-8rdkx\" (UID: \"0720a4d9-96d6-48eb-839a-ec1f25d7bc56\") " pod="openshift-ingress/kube-auth-proxy-54976cff48-8rdkx" Sep 13 20:21:32.575710 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:32.575584 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0720a4d9-96d6-48eb-839a-ec1f25d7bc56-tls-certs\") pod \"kube-auth-proxy-54976cff48-8rdkx\" (UID: \"0720a4d9-96d6-48eb-839a-ec1f25d7bc56\") " pod="openshift-ingress/kube-auth-proxy-54976cff48-8rdkx" Sep 13 20:21:32.676632 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:32.676595 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0720a4d9-96d6-48eb-839a-ec1f25d7bc56-tmp\") pod \"kube-auth-proxy-54976cff48-8rdkx\" (UID: \"0720a4d9-96d6-48eb-839a-ec1f25d7bc56\") " pod="openshift-ingress/kube-auth-proxy-54976cff48-8rdkx" Sep 13 20:21:32.676632 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:32.676637 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slskf\" (UniqueName: \"kubernetes.io/projected/0720a4d9-96d6-48eb-839a-ec1f25d7bc56-kube-api-access-slskf\") pod \"kube-auth-proxy-54976cff48-8rdkx\" (UID: \"0720a4d9-96d6-48eb-839a-ec1f25d7bc56\") " pod="openshift-ingress/kube-auth-proxy-54976cff48-8rdkx" Sep 13 20:21:32.676836 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:32.676664 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0720a4d9-96d6-48eb-839a-ec1f25d7bc56-tls-certs\") pod \"kube-auth-proxy-54976cff48-8rdkx\" (UID: \"0720a4d9-96d6-48eb-839a-ec1f25d7bc56\") " pod="openshift-ingress/kube-auth-proxy-54976cff48-8rdkx" Sep 13 20:21:32.676836 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:21:32.676795 2557 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Sep 13 20:21:32.676896 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:21:32.676859 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0720a4d9-96d6-48eb-839a-ec1f25d7bc56-tls-certs podName:0720a4d9-96d6-48eb-839a-ec1f25d7bc56 nodeName:}" failed. No retries permitted until 2026-09-13 20:21:33.176841501 +0000 UTC m=+525.353277507 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/0720a4d9-96d6-48eb-839a-ec1f25d7bc56-tls-certs") pod "kube-auth-proxy-54976cff48-8rdkx" (UID: "0720a4d9-96d6-48eb-839a-ec1f25d7bc56") : secret "kube-auth-proxy-tls" not found Sep 13 20:21:32.678802 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:32.678778 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0720a4d9-96d6-48eb-839a-ec1f25d7bc56-tmp\") pod \"kube-auth-proxy-54976cff48-8rdkx\" (UID: \"0720a4d9-96d6-48eb-839a-ec1f25d7bc56\") " pod="openshift-ingress/kube-auth-proxy-54976cff48-8rdkx" Sep 13 20:21:32.684955 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:32.684933 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slskf\" (UniqueName: \"kubernetes.io/projected/0720a4d9-96d6-48eb-839a-ec1f25d7bc56-kube-api-access-slskf\") pod \"kube-auth-proxy-54976cff48-8rdkx\" (UID: \"0720a4d9-96d6-48eb-839a-ec1f25d7bc56\") " pod="openshift-ingress/kube-auth-proxy-54976cff48-8rdkx" Sep 13 20:21:33.180165 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.180123 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0720a4d9-96d6-48eb-839a-ec1f25d7bc56-tls-certs\") pod \"kube-auth-proxy-54976cff48-8rdkx\" (UID: \"0720a4d9-96d6-48eb-839a-ec1f25d7bc56\") " pod="openshift-ingress/kube-auth-proxy-54976cff48-8rdkx" Sep 13 20:21:33.182655 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.182630 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0720a4d9-96d6-48eb-839a-ec1f25d7bc56-tls-certs\") pod \"kube-auth-proxy-54976cff48-8rdkx\" (UID: \"0720a4d9-96d6-48eb-839a-ec1f25d7bc56\") " pod="openshift-ingress/kube-auth-proxy-54976cff48-8rdkx" Sep 13 20:21:33.447954 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.447873 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-54976cff48-8rdkx" Sep 13 20:21:33.577518 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.577474 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7"] Sep 13 20:21:33.581916 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.581899 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" Sep 13 20:21:33.584660 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.584637 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 13 20:21:33.585157 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.584671 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 13 20:21:33.585157 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.584933 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 13 20:21:33.585157 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.584969 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-sjpzs\"" Sep 13 20:21:33.586880 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.586862 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 13 20:21:33.597912 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.597889 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7"] Sep 13 20:21:33.683815 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.683781 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/411832cf-3cef-4c27-858b-b5461c6fe6b8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-kv4t7\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" Sep 13 20:21:33.683815 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.683819 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94rls\" (UniqueName: \"kubernetes.io/projected/411832cf-3cef-4c27-858b-b5461c6fe6b8-kube-api-access-94rls\") pod \"istiod-openshift-gateway-7b7995d5cd-kv4t7\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" Sep 13 20:21:33.684037 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.683919 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/411832cf-3cef-4c27-858b-b5461c6fe6b8-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-kv4t7\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" Sep 13 20:21:33.684037 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.683954 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/411832cf-3cef-4c27-858b-b5461c6fe6b8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-kv4t7\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" Sep 13 20:21:33.684037 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.683986 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/411832cf-3cef-4c27-858b-b5461c6fe6b8-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-kv4t7\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" Sep 13 20:21:33.684037 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.684017 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/411832cf-3cef-4c27-858b-b5461c6fe6b8-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-kv4t7\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" Sep 13 20:21:33.684252 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.684043 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/411832cf-3cef-4c27-858b-b5461c6fe6b8-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-kv4t7\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" Sep 13 20:21:33.720222 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.719206 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-54976cff48-8rdkx"] Sep 13 20:21:33.785338 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.785300 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/411832cf-3cef-4c27-858b-b5461c6fe6b8-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-kv4t7\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" Sep 13 20:21:33.785338 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.785338 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/411832cf-3cef-4c27-858b-b5461c6fe6b8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-kv4t7\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" Sep 13 20:21:33.785571 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.785358 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/411832cf-3cef-4c27-858b-b5461c6fe6b8-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-kv4t7\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" Sep 13 20:21:33.785571 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.785379 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/411832cf-3cef-4c27-858b-b5461c6fe6b8-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-kv4t7\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" Sep 13 20:21:33.785571 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.785396 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/411832cf-3cef-4c27-858b-b5461c6fe6b8-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-kv4t7\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" Sep 13 20:21:33.785571 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.785421 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/411832cf-3cef-4c27-858b-b5461c6fe6b8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-kv4t7\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" Sep 13 20:21:33.785571 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.785441 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94rls\" (UniqueName: \"kubernetes.io/projected/411832cf-3cef-4c27-858b-b5461c6fe6b8-kube-api-access-94rls\") pod \"istiod-openshift-gateway-7b7995d5cd-kv4t7\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" Sep 13 20:21:33.786080 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.786049 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/411832cf-3cef-4c27-858b-b5461c6fe6b8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-kv4t7\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" Sep 13 20:21:33.787799 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.787776 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/411832cf-3cef-4c27-858b-b5461c6fe6b8-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-kv4t7\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" Sep 13 20:21:33.787970 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.787948 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/411832cf-3cef-4c27-858b-b5461c6fe6b8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-kv4t7\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" Sep 13 20:21:33.788024 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.787998 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/411832cf-3cef-4c27-858b-b5461c6fe6b8-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-kv4t7\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" Sep 13 20:21:33.788120 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.788100 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/411832cf-3cef-4c27-858b-b5461c6fe6b8-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-kv4t7\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" Sep 13 20:21:33.795688 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.795666 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/411832cf-3cef-4c27-858b-b5461c6fe6b8-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-kv4t7\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" Sep 13 20:21:33.796044 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.796019 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94rls\" (UniqueName: \"kubernetes.io/projected/411832cf-3cef-4c27-858b-b5461c6fe6b8-kube-api-access-94rls\") pod \"istiod-openshift-gateway-7b7995d5cd-kv4t7\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" Sep 13 20:21:33.891990 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:33.891960 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" Sep 13 20:21:34.168200 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:34.168177 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7"] Sep 13 20:21:34.170046 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:21:34.170017 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod411832cf_3cef_4c27_858b_b5461c6fe6b8.slice/crio-da6c8c68b733015aa2343552a19a3f0a0c1f7a2cad039fefca1a4b9e413979f7 WatchSource:0}: Error finding container da6c8c68b733015aa2343552a19a3f0a0c1f7a2cad039fefca1a4b9e413979f7: Status 404 returned error can't find the container with id da6c8c68b733015aa2343552a19a3f0a0c1f7a2cad039fefca1a4b9e413979f7 Sep 13 20:21:34.275925 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:34.275892 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-54976cff48-8rdkx" event={"ID":"0720a4d9-96d6-48eb-839a-ec1f25d7bc56","Type":"ContainerStarted","Data":"be6ce396c36f511c437c5531af9d2b7d395c55d2735d98f9ff99a3a2a167c89e"} Sep 13 20:21:34.277063 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:34.277036 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" event={"ID":"411832cf-3cef-4c27-858b-b5461c6fe6b8","Type":"ContainerStarted","Data":"da6c8c68b733015aa2343552a19a3f0a0c1f7a2cad039fefca1a4b9e413979f7"} Sep 13 20:21:37.667355 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:37.667307 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 13 20:21:37.667705 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:37.667403 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 13 20:21:38.300868 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:38.300835 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-54976cff48-8rdkx" event={"ID":"0720a4d9-96d6-48eb-839a-ec1f25d7bc56","Type":"ContainerStarted","Data":"eefd38ff529db6d0184b46f9ec44c13cb0419a217c33c875261543ee5f3e4cd7"} Sep 13 20:21:38.302477 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:38.302446 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" event={"ID":"411832cf-3cef-4c27-858b-b5461c6fe6b8","Type":"ContainerStarted","Data":"cc8ed272b5ebaff26b3c3448845b863bc4cd808eea2916342a89b63f91544092"} Sep 13 20:21:38.302641 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:38.302630 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" Sep 13 20:21:38.304441 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:38.304414 2557 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-kv4t7 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 13 20:21:38.304585 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:38.304469 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" podUID="411832cf-3cef-4c27-858b-b5461c6fe6b8" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 13 20:21:38.316822 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:38.316780 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-54976cff48-8rdkx" podStartSLOduration=2.424376034 podStartE2EDuration="6.316766221s" totalImagesPullingTime="3.892390187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:21:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:21:38.316400694 +0000 UTC m=+530.492836717" watchObservedRunningTime="2026-09-13 20:21:38.316766221 +0000 UTC m=+530.493202327" Sep 13 20:21:38.333441 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:38.333386 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" podStartSLOduration=1.8384090199999998 podStartE2EDuration="5.33336848s" totalImagesPullingTime="3.49495946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:21:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:21:38.332474638 +0000 UTC m=+530.508910670" watchObservedRunningTime="2026-09-13 20:21:38.33336848 +0000 UTC m=+530.509804503" Sep 13 20:21:39.309756 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:39.309722 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" Sep 13 20:21:43.326602 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:43.326518 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-pjpgr"] Sep 13 20:21:43.330101 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:43.330081 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-pjpgr" Sep 13 20:21:43.332153 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:43.332131 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Sep 13 20:21:43.332269 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:43.332135 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-8g8bn\"" Sep 13 20:21:43.343081 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:43.343050 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-pjpgr"] Sep 13 20:21:43.372026 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:43.371997 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e74cb73-9d45-47fc-9ce4-6b88d88fbdae-tls-certs\") pod \"model-serving-api-54d89bd5fc-pjpgr\" (UID: \"0e74cb73-9d45-47fc-9ce4-6b88d88fbdae\") " pod="opendatahub/model-serving-api-54d89bd5fc-pjpgr" Sep 13 20:21:43.372154 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:43.372036 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4xhq\" (UniqueName: \"kubernetes.io/projected/0e74cb73-9d45-47fc-9ce4-6b88d88fbdae-kube-api-access-k4xhq\") pod \"model-serving-api-54d89bd5fc-pjpgr\" (UID: \"0e74cb73-9d45-47fc-9ce4-6b88d88fbdae\") " pod="opendatahub/model-serving-api-54d89bd5fc-pjpgr" Sep 13 20:21:43.473391 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:43.473357 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e74cb73-9d45-47fc-9ce4-6b88d88fbdae-tls-certs\") pod \"model-serving-api-54d89bd5fc-pjpgr\" (UID: \"0e74cb73-9d45-47fc-9ce4-6b88d88fbdae\") " pod="opendatahub/model-serving-api-54d89bd5fc-pjpgr" Sep 13 20:21:43.473391 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:43.473400 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4xhq\" (UniqueName: \"kubernetes.io/projected/0e74cb73-9d45-47fc-9ce4-6b88d88fbdae-kube-api-access-k4xhq\") pod \"model-serving-api-54d89bd5fc-pjpgr\" (UID: \"0e74cb73-9d45-47fc-9ce4-6b88d88fbdae\") " pod="opendatahub/model-serving-api-54d89bd5fc-pjpgr" Sep 13 20:21:43.475935 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:43.475912 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e74cb73-9d45-47fc-9ce4-6b88d88fbdae-tls-certs\") pod \"model-serving-api-54d89bd5fc-pjpgr\" (UID: \"0e74cb73-9d45-47fc-9ce4-6b88d88fbdae\") " pod="opendatahub/model-serving-api-54d89bd5fc-pjpgr" Sep 13 20:21:43.480601 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:43.480574 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4xhq\" (UniqueName: \"kubernetes.io/projected/0e74cb73-9d45-47fc-9ce4-6b88d88fbdae-kube-api-access-k4xhq\") pod \"model-serving-api-54d89bd5fc-pjpgr\" (UID: \"0e74cb73-9d45-47fc-9ce4-6b88d88fbdae\") " pod="opendatahub/model-serving-api-54d89bd5fc-pjpgr" Sep 13 20:21:43.641976 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:43.641880 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-pjpgr" Sep 13 20:21:43.909797 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:43.909767 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-pjpgr"] Sep 13 20:21:43.911871 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:21:43.911845 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e74cb73_9d45_47fc_9ce4_6b88d88fbdae.slice/crio-eea1fa2f2b58d505cddbf577da00abf7bad522ddf2216b7178dc65407fc0733e WatchSource:0}: Error finding container eea1fa2f2b58d505cddbf577da00abf7bad522ddf2216b7178dc65407fc0733e: Status 404 returned error can't find the container with id eea1fa2f2b58d505cddbf577da00abf7bad522ddf2216b7178dc65407fc0733e Sep 13 20:21:44.332842 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:44.332806 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-pjpgr" event={"ID":"0e74cb73-9d45-47fc-9ce4-6b88d88fbdae","Type":"ContainerStarted","Data":"eea1fa2f2b58d505cddbf577da00abf7bad522ddf2216b7178dc65407fc0733e"} Sep 13 20:21:46.343837 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:46.343801 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-pjpgr" event={"ID":"0e74cb73-9d45-47fc-9ce4-6b88d88fbdae","Type":"ContainerStarted","Data":"d4c3f3b54ba0e5380f2f30eabf46802904569f484469bc6e0d3aeaaa6790fa1a"} Sep 13 20:21:46.344262 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:46.343859 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-pjpgr" Sep 13 20:21:46.359549 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:46.359486 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-pjpgr" podStartSLOduration=1.690635859 podStartE2EDuration="3.359470175s" totalImagesPullingTime="1.668834316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:21:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:21:46.358467135 +0000 UTC m=+538.534903157" watchObservedRunningTime="2026-09-13 20:21:46.359470175 +0000 UTC m=+538.535906251" Sep 13 20:21:51.394995 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:51.394962 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-q2h9r"] Sep 13 20:21:51.403965 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:51.403922 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-q2h9r" Sep 13 20:21:51.406777 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:51.406480 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Sep 13 20:21:51.406777 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:51.406570 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-pjl74\"" Sep 13 20:21:51.413275 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:51.412688 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-q2h9r"] Sep 13 20:21:51.422167 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:51.421519 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-n7zns"] Sep 13 20:21:51.425351 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:51.425329 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-n7zns" Sep 13 20:21:51.427855 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:51.427826 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Sep 13 20:21:51.428668 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:51.428645 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-5df6m\"" Sep 13 20:21:51.428837 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:51.428820 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-n7zns"] Sep 13 20:21:51.552399 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:51.552352 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5lbx\" (UniqueName: \"kubernetes.io/projected/39c26102-f478-43d5-aaf2-9d1cbe372f8d-kube-api-access-k5lbx\") pod \"kserve-controller-manager-d6f4b65b8-q2h9r\" (UID: \"39c26102-f478-43d5-aaf2-9d1cbe372f8d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-q2h9r" Sep 13 20:21:51.552617 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:51.552520 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqk6n\" (UniqueName: \"kubernetes.io/projected/b047f42d-3486-452a-8daf-9f367be63881-kube-api-access-jqk6n\") pod \"llmisvc-controller-manager-777c6bd7d5-n7zns\" (UID: \"b047f42d-3486-452a-8daf-9f367be63881\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-n7zns" Sep 13 20:21:51.552617 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:51.552569 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39c26102-f478-43d5-aaf2-9d1cbe372f8d-cert\") pod \"kserve-controller-manager-d6f4b65b8-q2h9r\" (UID: \"39c26102-f478-43d5-aaf2-9d1cbe372f8d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-q2h9r" Sep 13 20:21:51.552617 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:51.552593 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b047f42d-3486-452a-8daf-9f367be63881-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-n7zns\" (UID: \"b047f42d-3486-452a-8daf-9f367be63881\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-n7zns" Sep 13 20:21:51.653796 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:51.653698 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5lbx\" (UniqueName: \"kubernetes.io/projected/39c26102-f478-43d5-aaf2-9d1cbe372f8d-kube-api-access-k5lbx\") pod \"kserve-controller-manager-d6f4b65b8-q2h9r\" (UID: \"39c26102-f478-43d5-aaf2-9d1cbe372f8d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-q2h9r" Sep 13 20:21:51.653796 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:51.653791 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqk6n\" (UniqueName: \"kubernetes.io/projected/b047f42d-3486-452a-8daf-9f367be63881-kube-api-access-jqk6n\") pod \"llmisvc-controller-manager-777c6bd7d5-n7zns\" (UID: \"b047f42d-3486-452a-8daf-9f367be63881\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-n7zns" Sep 13 20:21:51.654028 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:51.653823 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39c26102-f478-43d5-aaf2-9d1cbe372f8d-cert\") pod \"kserve-controller-manager-d6f4b65b8-q2h9r\" (UID: \"39c26102-f478-43d5-aaf2-9d1cbe372f8d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-q2h9r" Sep 13 20:21:51.654028 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:51.653844 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b047f42d-3486-452a-8daf-9f367be63881-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-n7zns\" (UID: \"b047f42d-3486-452a-8daf-9f367be63881\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-n7zns" Sep 13 20:21:51.654028 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:21:51.653976 2557 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Sep 13 20:21:51.654184 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:21:51.654055 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b047f42d-3486-452a-8daf-9f367be63881-cert podName:b047f42d-3486-452a-8daf-9f367be63881 nodeName:}" failed. No retries permitted until 2026-09-13 20:21:52.154039582 +0000 UTC m=+544.330475585 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b047f42d-3486-452a-8daf-9f367be63881-cert") pod "llmisvc-controller-manager-777c6bd7d5-n7zns" (UID: "b047f42d-3486-452a-8daf-9f367be63881") : secret "llmisvc-webhook-server-cert" not found Sep 13 20:21:51.656466 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:51.656443 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39c26102-f478-43d5-aaf2-9d1cbe372f8d-cert\") pod \"kserve-controller-manager-d6f4b65b8-q2h9r\" (UID: \"39c26102-f478-43d5-aaf2-9d1cbe372f8d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-q2h9r" Sep 13 20:21:51.668387 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:51.668365 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5lbx\" (UniqueName: \"kubernetes.io/projected/39c26102-f478-43d5-aaf2-9d1cbe372f8d-kube-api-access-k5lbx\") pod \"kserve-controller-manager-d6f4b65b8-q2h9r\" (UID: \"39c26102-f478-43d5-aaf2-9d1cbe372f8d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-q2h9r" Sep 13 20:21:51.675821 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:51.675795 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqk6n\" (UniqueName: \"kubernetes.io/projected/b047f42d-3486-452a-8daf-9f367be63881-kube-api-access-jqk6n\") pod \"llmisvc-controller-manager-777c6bd7d5-n7zns\" (UID: \"b047f42d-3486-452a-8daf-9f367be63881\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-n7zns" Sep 13 20:21:51.721876 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:51.721841 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-q2h9r" Sep 13 20:21:51.991812 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:51.991724 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-q2h9r"] Sep 13 20:21:51.996117 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:21:51.996087 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39c26102_f478_43d5_aaf2_9d1cbe372f8d.slice/crio-eeb58b135076635fa36bff5a85e6615c593587df350f429d1e65ed03025eb370 WatchSource:0}: Error finding container eeb58b135076635fa36bff5a85e6615c593587df350f429d1e65ed03025eb370: Status 404 returned error can't find the container with id eeb58b135076635fa36bff5a85e6615c593587df350f429d1e65ed03025eb370 Sep 13 20:21:52.158350 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:52.158312 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b047f42d-3486-452a-8daf-9f367be63881-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-n7zns\" (UID: \"b047f42d-3486-452a-8daf-9f367be63881\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-n7zns" Sep 13 20:21:52.160753 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:52.160724 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b047f42d-3486-452a-8daf-9f367be63881-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-n7zns\" (UID: \"b047f42d-3486-452a-8daf-9f367be63881\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-n7zns" Sep 13 20:21:52.338584 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:52.338552 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-n7zns" Sep 13 20:21:52.377505 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:52.377470 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-q2h9r" event={"ID":"39c26102-f478-43d5-aaf2-9d1cbe372f8d","Type":"ContainerStarted","Data":"eeb58b135076635fa36bff5a85e6615c593587df350f429d1e65ed03025eb370"} Sep 13 20:21:52.630746 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:52.630723 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-n7zns"] Sep 13 20:21:52.632594 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:21:52.632565 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb047f42d_3486_452a_8daf_9f367be63881.slice/crio-38d2ea7c5d9dacb279b3b13b82ce111a76e8d10d0fb55511a321e131c6e9aa8e WatchSource:0}: Error finding container 38d2ea7c5d9dacb279b3b13b82ce111a76e8d10d0fb55511a321e131c6e9aa8e: Status 404 returned error can't find the container with id 38d2ea7c5d9dacb279b3b13b82ce111a76e8d10d0fb55511a321e131c6e9aa8e Sep 13 20:21:53.383789 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:53.383757 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-n7zns" event={"ID":"b047f42d-3486-452a-8daf-9f367be63881","Type":"ContainerStarted","Data":"38d2ea7c5d9dacb279b3b13b82ce111a76e8d10d0fb55511a321e131c6e9aa8e"} Sep 13 20:21:53.619513 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:53.619471 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7"] Sep 13 20:21:53.619729 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:53.619705 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" podUID="411832cf-3cef-4c27-858b-b5461c6fe6b8" containerName="discovery" containerID="cri-o://cc8ed272b5ebaff26b3c3448845b863bc4cd808eea2916342a89b63f91544092" gracePeriod=30 Sep 13 20:21:54.392585 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:54.392553 2557 generic.go:356] "Generic (PLEG): container finished" podID="411832cf-3cef-4c27-858b-b5461c6fe6b8" containerID="cc8ed272b5ebaff26b3c3448845b863bc4cd808eea2916342a89b63f91544092" exitCode=0 Sep 13 20:21:54.392992 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:54.392647 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" event={"ID":"411832cf-3cef-4c27-858b-b5461c6fe6b8","Type":"ContainerDied","Data":"cc8ed272b5ebaff26b3c3448845b863bc4cd808eea2916342a89b63f91544092"} Sep 13 20:21:54.560746 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:54.560723 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" Sep 13 20:21:54.685206 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:54.685135 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/411832cf-3cef-4c27-858b-b5461c6fe6b8-istio-kubeconfig\") pod \"411832cf-3cef-4c27-858b-b5461c6fe6b8\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " Sep 13 20:21:54.685206 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:54.685177 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/411832cf-3cef-4c27-858b-b5461c6fe6b8-istio-token\") pod \"411832cf-3cef-4c27-858b-b5461c6fe6b8\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " Sep 13 20:21:54.685206 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:54.685200 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94rls\" (UniqueName: \"kubernetes.io/projected/411832cf-3cef-4c27-858b-b5461c6fe6b8-kube-api-access-94rls\") pod \"411832cf-3cef-4c27-858b-b5461c6fe6b8\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " Sep 13 20:21:54.685461 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:54.685258 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/411832cf-3cef-4c27-858b-b5461c6fe6b8-istio-csr-dns-cert\") pod \"411832cf-3cef-4c27-858b-b5461c6fe6b8\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " Sep 13 20:21:54.685461 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:54.685284 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/411832cf-3cef-4c27-858b-b5461c6fe6b8-istio-csr-ca-configmap\") pod \"411832cf-3cef-4c27-858b-b5461c6fe6b8\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " Sep 13 20:21:54.685461 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:54.685369 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/411832cf-3cef-4c27-858b-b5461c6fe6b8-cacerts\") pod \"411832cf-3cef-4c27-858b-b5461c6fe6b8\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " Sep 13 20:21:54.685461 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:54.685415 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/411832cf-3cef-4c27-858b-b5461c6fe6b8-local-certs\") pod \"411832cf-3cef-4c27-858b-b5461c6fe6b8\" (UID: \"411832cf-3cef-4c27-858b-b5461c6fe6b8\") " Sep 13 20:21:54.685723 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:54.685682 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/411832cf-3cef-4c27-858b-b5461c6fe6b8-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "411832cf-3cef-4c27-858b-b5461c6fe6b8" (UID: "411832cf-3cef-4c27-858b-b5461c6fe6b8"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 13 20:21:54.685870 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:54.685781 2557 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/411832cf-3cef-4c27-858b-b5461c6fe6b8-istio-csr-ca-configmap\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:21:54.687892 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:54.687829 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/411832cf-3cef-4c27-858b-b5461c6fe6b8-cacerts" (OuterVolumeSpecName: "cacerts") pod "411832cf-3cef-4c27-858b-b5461c6fe6b8" (UID: "411832cf-3cef-4c27-858b-b5461c6fe6b8"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 20:21:54.687997 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:54.687893 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/411832cf-3cef-4c27-858b-b5461c6fe6b8-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "411832cf-3cef-4c27-858b-b5461c6fe6b8" (UID: "411832cf-3cef-4c27-858b-b5461c6fe6b8"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 20:21:54.687997 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:54.687912 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/411832cf-3cef-4c27-858b-b5461c6fe6b8-kube-api-access-94rls" (OuterVolumeSpecName: "kube-api-access-94rls") pod "411832cf-3cef-4c27-858b-b5461c6fe6b8" (UID: "411832cf-3cef-4c27-858b-b5461c6fe6b8"). InnerVolumeSpecName "kube-api-access-94rls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 20:21:54.688329 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:54.688298 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/411832cf-3cef-4c27-858b-b5461c6fe6b8-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "411832cf-3cef-4c27-858b-b5461c6fe6b8" (UID: "411832cf-3cef-4c27-858b-b5461c6fe6b8"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 20:21:54.688413 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:54.688375 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/411832cf-3cef-4c27-858b-b5461c6fe6b8-istio-token" (OuterVolumeSpecName: "istio-token") pod "411832cf-3cef-4c27-858b-b5461c6fe6b8" (UID: "411832cf-3cef-4c27-858b-b5461c6fe6b8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 20:21:54.688413 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:54.688370 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/411832cf-3cef-4c27-858b-b5461c6fe6b8-local-certs" (OuterVolumeSpecName: "local-certs") pod "411832cf-3cef-4c27-858b-b5461c6fe6b8" (UID: "411832cf-3cef-4c27-858b-b5461c6fe6b8"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 13 20:21:54.787149 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:54.787110 2557 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/411832cf-3cef-4c27-858b-b5461c6fe6b8-istio-csr-dns-cert\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:21:54.787149 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:54.787144 2557 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/411832cf-3cef-4c27-858b-b5461c6fe6b8-cacerts\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:21:54.787149 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:54.787155 2557 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/411832cf-3cef-4c27-858b-b5461c6fe6b8-local-certs\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:21:54.787406 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:54.787168 2557 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/411832cf-3cef-4c27-858b-b5461c6fe6b8-istio-kubeconfig\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:21:54.787406 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:54.787178 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/411832cf-3cef-4c27-858b-b5461c6fe6b8-istio-token\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:21:54.787406 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:54.787189 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94rls\" (UniqueName: \"kubernetes.io/projected/411832cf-3cef-4c27-858b-b5461c6fe6b8-kube-api-access-94rls\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:21:55.399756 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:55.399732 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" Sep 13 20:21:55.400102 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:55.399749 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" event={"ID":"411832cf-3cef-4c27-858b-b5461c6fe6b8","Type":"ContainerDied","Data":"da6c8c68b733015aa2343552a19a3f0a0c1f7a2cad039fefca1a4b9e413979f7"} Sep 13 20:21:55.400102 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:55.399808 2557 scope.go:117] "RemoveContainer" containerID="cc8ed272b5ebaff26b3c3448845b863bc4cd808eea2916342a89b63f91544092" Sep 13 20:21:55.430292 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:55.430264 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7"] Sep 13 20:21:55.436506 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:55.436469 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7"] Sep 13 20:21:56.312086 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:56.312054 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="411832cf-3cef-4c27-858b-b5461c6fe6b8" path="/var/lib/kubelet/pods/411832cf-3cef-4c27-858b-b5461c6fe6b8/volumes" Sep 13 20:21:56.406976 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:56.406948 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-q2h9r" event={"ID":"39c26102-f478-43d5-aaf2-9d1cbe372f8d","Type":"ContainerStarted","Data":"ff479021253c0a0cac3bf0b7629dad5c7fb1091abd348cbfd785f5dba1280373"} Sep 13 20:21:56.407370 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:56.407115 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-q2h9r" Sep 13 20:21:56.408579 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:56.408553 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-n7zns" event={"ID":"b047f42d-3486-452a-8daf-9f367be63881","Type":"ContainerStarted","Data":"23445f0a7760b6de04cd0dfdeeabfdbf697d9464ff530c94f470c0eca30772af"} Sep 13 20:21:56.408702 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:56.408608 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-n7zns" Sep 13 20:21:56.422742 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:56.422704 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-q2h9r" podStartSLOduration=2.665322901 podStartE2EDuration="5.422691622s" totalImagesPullingTime="2.757368721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:21:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:21:56.421401712 +0000 UTC m=+548.597837733" watchObservedRunningTime="2026-09-13 20:21:56.422691622 +0000 UTC m=+548.599127643" Sep 13 20:21:56.437175 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:56.437139 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-n7zns" podStartSLOduration=2.653475807 podStartE2EDuration="5.437125684s" totalImagesPullingTime="2.783649877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:21:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:21:56.435786062 +0000 UTC m=+548.612222085" watchObservedRunningTime="2026-09-13 20:21:56.437125684 +0000 UTC m=+548.613561705" Sep 13 20:21:57.351759 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:57.351730 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-pjpgr" Sep 13 20:21:59.318278 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:59.318235 2557 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-kv4t7 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.31:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 13 20:21:59.318684 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:21:59.318312 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kv4t7" podUID="411832cf-3cef-4c27-858b-b5461c6fe6b8" containerName="discovery" probeResult="failure" output="Get \"http://10.132.0.31:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 13 20:22:27.416020 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:27.415982 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-n7zns" Sep 13 20:22:27.419085 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:27.419062 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-q2h9r" Sep 13 20:22:47.353320 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.353289 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s"] Sep 13 20:22:47.353790 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.353618 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="411832cf-3cef-4c27-858b-b5461c6fe6b8" containerName="discovery" Sep 13 20:22:47.353790 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.353629 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="411832cf-3cef-4c27-858b-b5461c6fe6b8" containerName="discovery" Sep 13 20:22:47.353790 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.353694 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="411832cf-3cef-4c27-858b-b5461c6fe6b8" containerName="discovery" Sep 13 20:22:47.356723 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.356701 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s" Sep 13 20:22:47.358853 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.358825 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 13 20:22:47.358977 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.358883 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 13 20:22:47.359710 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.359691 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-w8j7b\"" Sep 13 20:22:47.363284 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.363263 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s"] Sep 13 20:22:47.452711 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.452678 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb"] Sep 13 20:22:47.456373 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.456353 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb" Sep 13 20:22:47.471433 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.471394 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb"] Sep 13 20:22:47.502600 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.502572 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a4c4548-0ea8-4699-a1b4-1497348470e8-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s\" (UID: \"9a4c4548-0ea8-4699-a1b4-1497348470e8\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s" Sep 13 20:22:47.502714 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.502629 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a4c4548-0ea8-4699-a1b4-1497348470e8-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s\" (UID: \"9a4c4548-0ea8-4699-a1b4-1497348470e8\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s" Sep 13 20:22:47.502714 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.502670 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lphpf\" (UniqueName: \"kubernetes.io/projected/9a4c4548-0ea8-4699-a1b4-1497348470e8-kube-api-access-lphpf\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s\" (UID: \"9a4c4548-0ea8-4699-a1b4-1497348470e8\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s" Sep 13 20:22:47.549951 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.549921 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24"] Sep 13 20:22:47.553366 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.553351 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24" Sep 13 20:22:47.560672 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.560643 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24"] Sep 13 20:22:47.603563 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.603465 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24bc8b9f-7482-49a4-acc0-38a451eb0bce-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24\" (UID: \"24bc8b9f-7482-49a4-acc0-38a451eb0bce\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24" Sep 13 20:22:47.603563 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.603529 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a4c4548-0ea8-4699-a1b4-1497348470e8-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s\" (UID: \"9a4c4548-0ea8-4699-a1b4-1497348470e8\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s" Sep 13 20:22:47.603784 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.603581 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfr9c\" (UniqueName: \"kubernetes.io/projected/24bc8b9f-7482-49a4-acc0-38a451eb0bce-kube-api-access-hfr9c\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24\" (UID: \"24bc8b9f-7482-49a4-acc0-38a451eb0bce\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24" Sep 13 20:22:47.603784 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.603599 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a4c4548-0ea8-4699-a1b4-1497348470e8-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s\" (UID: \"9a4c4548-0ea8-4699-a1b4-1497348470e8\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s" Sep 13 20:22:47.603784 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.603620 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79ce4c0b-e994-43db-9788-4f37b57f19b7-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb\" (UID: \"79ce4c0b-e994-43db-9788-4f37b57f19b7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb" Sep 13 20:22:47.603784 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.603730 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lphpf\" (UniqueName: \"kubernetes.io/projected/9a4c4548-0ea8-4699-a1b4-1497348470e8-kube-api-access-lphpf\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s\" (UID: \"9a4c4548-0ea8-4699-a1b4-1497348470e8\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s" Sep 13 20:22:47.603784 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.603763 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79ce4c0b-e994-43db-9788-4f37b57f19b7-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb\" (UID: \"79ce4c0b-e994-43db-9788-4f37b57f19b7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb" Sep 13 20:22:47.604052 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.603797 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bdnx\" (UniqueName: \"kubernetes.io/projected/79ce4c0b-e994-43db-9788-4f37b57f19b7-kube-api-access-5bdnx\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb\" (UID: \"79ce4c0b-e994-43db-9788-4f37b57f19b7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb" Sep 13 20:22:47.604052 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.603835 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24bc8b9f-7482-49a4-acc0-38a451eb0bce-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24\" (UID: \"24bc8b9f-7482-49a4-acc0-38a451eb0bce\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24" Sep 13 20:22:47.604052 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.603990 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a4c4548-0ea8-4699-a1b4-1497348470e8-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s\" (UID: \"9a4c4548-0ea8-4699-a1b4-1497348470e8\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s" Sep 13 20:22:47.604052 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.603995 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a4c4548-0ea8-4699-a1b4-1497348470e8-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s\" (UID: \"9a4c4548-0ea8-4699-a1b4-1497348470e8\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s" Sep 13 20:22:47.611188 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.611168 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lphpf\" (UniqueName: \"kubernetes.io/projected/9a4c4548-0ea8-4699-a1b4-1497348470e8-kube-api-access-lphpf\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s\" (UID: \"9a4c4548-0ea8-4699-a1b4-1497348470e8\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s" Sep 13 20:22:47.652040 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.652010 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz"] Sep 13 20:22:47.655843 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.655825 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz" Sep 13 20:22:47.663460 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.663437 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz"] Sep 13 20:22:47.667141 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.667120 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s" Sep 13 20:22:47.704698 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.704665 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79ce4c0b-e994-43db-9788-4f37b57f19b7-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb\" (UID: \"79ce4c0b-e994-43db-9788-4f37b57f19b7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb" Sep 13 20:22:47.704832 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.704705 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79ce4c0b-e994-43db-9788-4f37b57f19b7-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb\" (UID: \"79ce4c0b-e994-43db-9788-4f37b57f19b7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb" Sep 13 20:22:47.704881 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.704839 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bdnx\" (UniqueName: \"kubernetes.io/projected/79ce4c0b-e994-43db-9788-4f37b57f19b7-kube-api-access-5bdnx\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb\" (UID: \"79ce4c0b-e994-43db-9788-4f37b57f19b7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb" Sep 13 20:22:47.704986 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.704910 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24bc8b9f-7482-49a4-acc0-38a451eb0bce-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24\" (UID: \"24bc8b9f-7482-49a4-acc0-38a451eb0bce\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24" Sep 13 20:22:47.704986 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.704945 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24bc8b9f-7482-49a4-acc0-38a451eb0bce-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24\" (UID: \"24bc8b9f-7482-49a4-acc0-38a451eb0bce\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24" Sep 13 20:22:47.705086 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.705046 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79ce4c0b-e994-43db-9788-4f37b57f19b7-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb\" (UID: \"79ce4c0b-e994-43db-9788-4f37b57f19b7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb" Sep 13 20:22:47.705086 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.705046 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfr9c\" (UniqueName: \"kubernetes.io/projected/24bc8b9f-7482-49a4-acc0-38a451eb0bce-kube-api-access-hfr9c\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24\" (UID: \"24bc8b9f-7482-49a4-acc0-38a451eb0bce\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24" Sep 13 20:22:47.705178 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.705128 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79ce4c0b-e994-43db-9788-4f37b57f19b7-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb\" (UID: \"79ce4c0b-e994-43db-9788-4f37b57f19b7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb" Sep 13 20:22:47.705302 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.705281 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24bc8b9f-7482-49a4-acc0-38a451eb0bce-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24\" (UID: \"24bc8b9f-7482-49a4-acc0-38a451eb0bce\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24" Sep 13 20:22:47.705374 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.705355 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24bc8b9f-7482-49a4-acc0-38a451eb0bce-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24\" (UID: \"24bc8b9f-7482-49a4-acc0-38a451eb0bce\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24" Sep 13 20:22:47.713485 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.713460 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfr9c\" (UniqueName: \"kubernetes.io/projected/24bc8b9f-7482-49a4-acc0-38a451eb0bce-kube-api-access-hfr9c\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24\" (UID: \"24bc8b9f-7482-49a4-acc0-38a451eb0bce\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24" Sep 13 20:22:47.713692 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.713565 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bdnx\" (UniqueName: \"kubernetes.io/projected/79ce4c0b-e994-43db-9788-4f37b57f19b7-kube-api-access-5bdnx\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb\" (UID: \"79ce4c0b-e994-43db-9788-4f37b57f19b7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb" Sep 13 20:22:47.772427 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.772397 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb" Sep 13 20:22:47.806241 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.806207 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2758e8d2-83b1-4168-9087-bfbe13d231a9-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz\" (UID: \"2758e8d2-83b1-4168-9087-bfbe13d231a9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz" Sep 13 20:22:47.806419 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.806259 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2758e8d2-83b1-4168-9087-bfbe13d231a9-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz\" (UID: \"2758e8d2-83b1-4168-9087-bfbe13d231a9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz" Sep 13 20:22:47.806419 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.806340 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prgxd\" (UniqueName: \"kubernetes.io/projected/2758e8d2-83b1-4168-9087-bfbe13d231a9-kube-api-access-prgxd\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz\" (UID: \"2758e8d2-83b1-4168-9087-bfbe13d231a9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz" Sep 13 20:22:47.862604 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.862536 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24" Sep 13 20:22:47.907014 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.906919 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prgxd\" (UniqueName: \"kubernetes.io/projected/2758e8d2-83b1-4168-9087-bfbe13d231a9-kube-api-access-prgxd\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz\" (UID: \"2758e8d2-83b1-4168-9087-bfbe13d231a9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz" Sep 13 20:22:47.907014 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.906997 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2758e8d2-83b1-4168-9087-bfbe13d231a9-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz\" (UID: \"2758e8d2-83b1-4168-9087-bfbe13d231a9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz" Sep 13 20:22:47.907226 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.907051 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2758e8d2-83b1-4168-9087-bfbe13d231a9-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz\" (UID: \"2758e8d2-83b1-4168-9087-bfbe13d231a9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz" Sep 13 20:22:47.907541 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.907519 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2758e8d2-83b1-4168-9087-bfbe13d231a9-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz\" (UID: \"2758e8d2-83b1-4168-9087-bfbe13d231a9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz" Sep 13 20:22:47.907649 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.907545 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2758e8d2-83b1-4168-9087-bfbe13d231a9-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz\" (UID: \"2758e8d2-83b1-4168-9087-bfbe13d231a9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz" Sep 13 20:22:47.918754 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.918690 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prgxd\" (UniqueName: \"kubernetes.io/projected/2758e8d2-83b1-4168-9087-bfbe13d231a9-kube-api-access-prgxd\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz\" (UID: \"2758e8d2-83b1-4168-9087-bfbe13d231a9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz" Sep 13 20:22:47.941380 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.941354 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s"] Sep 13 20:22:47.943475 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:22:47.943452 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a4c4548_0ea8_4699_a1b4_1497348470e8.slice/crio-36416e3b683534ad909615f1b60e401211d722992187ad7672f48562cfcf865c WatchSource:0}: Error finding container 36416e3b683534ad909615f1b60e401211d722992187ad7672f48562cfcf865c: Status 404 returned error can't find the container with id 36416e3b683534ad909615f1b60e401211d722992187ad7672f48562cfcf865c Sep 13 20:22:47.965600 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:47.965578 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz" Sep 13 20:22:48.043248 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:48.043220 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb"] Sep 13 20:22:48.046054 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:22:48.046014 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79ce4c0b_e994_43db_9788_4f37b57f19b7.slice/crio-bf6123bcb6ca0abcca33a527fdd0180cc552a9ba1cd4c9a4156889906ef68500 WatchSource:0}: Error finding container bf6123bcb6ca0abcca33a527fdd0180cc552a9ba1cd4c9a4156889906ef68500: Status 404 returned error can't find the container with id bf6123bcb6ca0abcca33a527fdd0180cc552a9ba1cd4c9a4156889906ef68500 Sep 13 20:22:48.132486 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:48.132459 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24"] Sep 13 20:22:48.133705 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:22:48.133673 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24bc8b9f_7482_49a4_acc0_38a451eb0bce.slice/crio-0a49b18387dc0c1873ff0c7d3212c97e1c9d02e76554c5385e638b6ebc1df788 WatchSource:0}: Error finding container 0a49b18387dc0c1873ff0c7d3212c97e1c9d02e76554c5385e638b6ebc1df788: Status 404 returned error can't find the container with id 0a49b18387dc0c1873ff0c7d3212c97e1c9d02e76554c5385e638b6ebc1df788 Sep 13 20:22:48.236806 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:48.236695 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz"] Sep 13 20:22:48.270245 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:22:48.270213 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2758e8d2_83b1_4168_9087_bfbe13d231a9.slice/crio-4cb2ff22c433c1d71268f49ed82897778971c7e409ddb72d5ed2b189b4f478b9 WatchSource:0}: Error finding container 4cb2ff22c433c1d71268f49ed82897778971c7e409ddb72d5ed2b189b4f478b9: Status 404 returned error can't find the container with id 4cb2ff22c433c1d71268f49ed82897778971c7e409ddb72d5ed2b189b4f478b9 Sep 13 20:22:48.662942 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:48.662852 2557 generic.go:356] "Generic (PLEG): container finished" podID="9a4c4548-0ea8-4699-a1b4-1497348470e8" containerID="d8c8446564996b49ab91c0dc31da2e5a1e90421e67ecf3becd61aeb54f0313c2" exitCode=0 Sep 13 20:22:48.662942 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:48.662896 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s" event={"ID":"9a4c4548-0ea8-4699-a1b4-1497348470e8","Type":"ContainerDied","Data":"d8c8446564996b49ab91c0dc31da2e5a1e90421e67ecf3becd61aeb54f0313c2"} Sep 13 20:22:48.663409 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:48.662950 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s" event={"ID":"9a4c4548-0ea8-4699-a1b4-1497348470e8","Type":"ContainerStarted","Data":"36416e3b683534ad909615f1b60e401211d722992187ad7672f48562cfcf865c"} Sep 13 20:22:48.664698 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:48.664679 2557 generic.go:356] "Generic (PLEG): container finished" podID="2758e8d2-83b1-4168-9087-bfbe13d231a9" containerID="ff43bced7f25b11cfb4f055729e033962d46ead8b0bc966467006c0c687bdff0" exitCode=0 Sep 13 20:22:48.664779 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:48.664758 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz" event={"ID":"2758e8d2-83b1-4168-9087-bfbe13d231a9","Type":"ContainerDied","Data":"ff43bced7f25b11cfb4f055729e033962d46ead8b0bc966467006c0c687bdff0"} Sep 13 20:22:48.664840 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:48.664804 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz" event={"ID":"2758e8d2-83b1-4168-9087-bfbe13d231a9","Type":"ContainerStarted","Data":"4cb2ff22c433c1d71268f49ed82897778971c7e409ddb72d5ed2b189b4f478b9"} Sep 13 20:22:48.666436 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:48.666413 2557 generic.go:356] "Generic (PLEG): container finished" podID="24bc8b9f-7482-49a4-acc0-38a451eb0bce" containerID="b4a75c0cf5d05a95d76d68d684e9e55fd5bf22e9ff306eec565c8ac2e9f81a22" exitCode=0 Sep 13 20:22:48.666548 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:48.666527 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24" event={"ID":"24bc8b9f-7482-49a4-acc0-38a451eb0bce","Type":"ContainerDied","Data":"b4a75c0cf5d05a95d76d68d684e9e55fd5bf22e9ff306eec565c8ac2e9f81a22"} Sep 13 20:22:48.666628 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:48.666576 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24" event={"ID":"24bc8b9f-7482-49a4-acc0-38a451eb0bce","Type":"ContainerStarted","Data":"0a49b18387dc0c1873ff0c7d3212c97e1c9d02e76554c5385e638b6ebc1df788"} Sep 13 20:22:48.668236 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:48.668208 2557 generic.go:356] "Generic (PLEG): container finished" podID="79ce4c0b-e994-43db-9788-4f37b57f19b7" containerID="24820e9f2f68357eb7ad1484f70d7985f325556786303b92fffcd594f6e885bd" exitCode=0 Sep 13 20:22:48.668302 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:48.668258 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb" event={"ID":"79ce4c0b-e994-43db-9788-4f37b57f19b7","Type":"ContainerDied","Data":"24820e9f2f68357eb7ad1484f70d7985f325556786303b92fffcd594f6e885bd"} Sep 13 20:22:48.668302 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:48.668288 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb" event={"ID":"79ce4c0b-e994-43db-9788-4f37b57f19b7","Type":"ContainerStarted","Data":"bf6123bcb6ca0abcca33a527fdd0180cc552a9ba1cd4c9a4156889906ef68500"} Sep 13 20:22:50.680299 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:50.680263 2557 generic.go:356] "Generic (PLEG): container finished" podID="9a4c4548-0ea8-4699-a1b4-1497348470e8" containerID="9a293649330c09702d8e5eeea9239fc710b44113effdb4c9f3ebe302c4a45267" exitCode=0 Sep 13 20:22:50.680788 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:50.680348 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s" event={"ID":"9a4c4548-0ea8-4699-a1b4-1497348470e8","Type":"ContainerDied","Data":"9a293649330c09702d8e5eeea9239fc710b44113effdb4c9f3ebe302c4a45267"} Sep 13 20:22:50.682257 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:50.682234 2557 generic.go:356] "Generic (PLEG): container finished" podID="2758e8d2-83b1-4168-9087-bfbe13d231a9" containerID="27d827885cf3cdb177ff26c58295ff81d7d6ddf7a4ddd1473067dd8fa99c6801" exitCode=0 Sep 13 20:22:50.682341 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:50.682320 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz" event={"ID":"2758e8d2-83b1-4168-9087-bfbe13d231a9","Type":"ContainerDied","Data":"27d827885cf3cdb177ff26c58295ff81d7d6ddf7a4ddd1473067dd8fa99c6801"} Sep 13 20:22:50.684393 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:50.684371 2557 generic.go:356] "Generic (PLEG): container finished" podID="24bc8b9f-7482-49a4-acc0-38a451eb0bce" containerID="d0605bc390284630a6393b798cb653edfb05f1d7523daa5b210d419cb9c774b0" exitCode=0 Sep 13 20:22:50.684514 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:50.684454 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24" event={"ID":"24bc8b9f-7482-49a4-acc0-38a451eb0bce","Type":"ContainerDied","Data":"d0605bc390284630a6393b798cb653edfb05f1d7523daa5b210d419cb9c774b0"} Sep 13 20:22:50.686579 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:50.686557 2557 generic.go:356] "Generic (PLEG): container finished" podID="79ce4c0b-e994-43db-9788-4f37b57f19b7" containerID="30358aa2701115e65c0eac54dd62965db20a78e4d7e3f06b2a93f5e23fcc0920" exitCode=0 Sep 13 20:22:50.686661 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:50.686617 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb" event={"ID":"79ce4c0b-e994-43db-9788-4f37b57f19b7","Type":"ContainerDied","Data":"30358aa2701115e65c0eac54dd62965db20a78e4d7e3f06b2a93f5e23fcc0920"} Sep 13 20:22:51.696119 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:51.696086 2557 generic.go:356] "Generic (PLEG): container finished" podID="24bc8b9f-7482-49a4-acc0-38a451eb0bce" containerID="9de8917c0ff35ce458ae2df944bf685990e734831f6fa5020b2e79a4da3d1b62" exitCode=0 Sep 13 20:22:51.696544 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:51.696138 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24" event={"ID":"24bc8b9f-7482-49a4-acc0-38a451eb0bce","Type":"ContainerDied","Data":"9de8917c0ff35ce458ae2df944bf685990e734831f6fa5020b2e79a4da3d1b62"} Sep 13 20:22:51.698166 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:51.698142 2557 generic.go:356] "Generic (PLEG): container finished" podID="79ce4c0b-e994-43db-9788-4f37b57f19b7" containerID="62ff3d8814127bddc0d7eb4f33750111e377037795ac7b88223424997708aafa" exitCode=0 Sep 13 20:22:51.698277 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:51.698225 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb" event={"ID":"79ce4c0b-e994-43db-9788-4f37b57f19b7","Type":"ContainerDied","Data":"62ff3d8814127bddc0d7eb4f33750111e377037795ac7b88223424997708aafa"} Sep 13 20:22:51.700348 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:51.700324 2557 generic.go:356] "Generic (PLEG): container finished" podID="9a4c4548-0ea8-4699-a1b4-1497348470e8" containerID="587866643c8d31a0e001f813b71037756cb40d0321c2d451527d1279f48dfd32" exitCode=0 Sep 13 20:22:51.700433 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:51.700352 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s" event={"ID":"9a4c4548-0ea8-4699-a1b4-1497348470e8","Type":"ContainerDied","Data":"587866643c8d31a0e001f813b71037756cb40d0321c2d451527d1279f48dfd32"} Sep 13 20:22:51.702330 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:51.702311 2557 generic.go:356] "Generic (PLEG): container finished" podID="2758e8d2-83b1-4168-9087-bfbe13d231a9" containerID="71c47ed8533cf483f6ca56d1bf089b2b88db940e4b38d413583a5567485c50ce" exitCode=0 Sep 13 20:22:51.702407 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:51.702355 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz" event={"ID":"2758e8d2-83b1-4168-9087-bfbe13d231a9","Type":"ContainerDied","Data":"71c47ed8533cf483f6ca56d1bf089b2b88db940e4b38d413583a5567485c50ce"} Sep 13 20:22:52.885363 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:52.885338 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz" Sep 13 20:22:52.889258 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:52.889237 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s" Sep 13 20:22:52.897674 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:52.897649 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24" Sep 13 20:22:52.905704 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:52.905679 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb" Sep 13 20:22:52.949389 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:52.949361 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prgxd\" (UniqueName: \"kubernetes.io/projected/2758e8d2-83b1-4168-9087-bfbe13d231a9-kube-api-access-prgxd\") pod \"2758e8d2-83b1-4168-9087-bfbe13d231a9\" (UID: \"2758e8d2-83b1-4168-9087-bfbe13d231a9\") " Sep 13 20:22:52.949555 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:52.949471 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2758e8d2-83b1-4168-9087-bfbe13d231a9-bundle\") pod \"2758e8d2-83b1-4168-9087-bfbe13d231a9\" (UID: \"2758e8d2-83b1-4168-9087-bfbe13d231a9\") " Sep 13 20:22:52.949555 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:52.949510 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2758e8d2-83b1-4168-9087-bfbe13d231a9-util\") pod \"2758e8d2-83b1-4168-9087-bfbe13d231a9\" (UID: \"2758e8d2-83b1-4168-9087-bfbe13d231a9\") " Sep 13 20:22:52.950645 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:52.950614 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2758e8d2-83b1-4168-9087-bfbe13d231a9-bundle" (OuterVolumeSpecName: "bundle") pod "2758e8d2-83b1-4168-9087-bfbe13d231a9" (UID: "2758e8d2-83b1-4168-9087-bfbe13d231a9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 13 20:22:52.951890 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:52.951865 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2758e8d2-83b1-4168-9087-bfbe13d231a9-kube-api-access-prgxd" (OuterVolumeSpecName: "kube-api-access-prgxd") pod "2758e8d2-83b1-4168-9087-bfbe13d231a9" (UID: "2758e8d2-83b1-4168-9087-bfbe13d231a9"). InnerVolumeSpecName "kube-api-access-prgxd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 20:22:52.955146 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:52.955116 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2758e8d2-83b1-4168-9087-bfbe13d231a9-util" (OuterVolumeSpecName: "util") pod "2758e8d2-83b1-4168-9087-bfbe13d231a9" (UID: "2758e8d2-83b1-4168-9087-bfbe13d231a9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 13 20:22:53.050774 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.050736 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24bc8b9f-7482-49a4-acc0-38a451eb0bce-bundle\") pod \"24bc8b9f-7482-49a4-acc0-38a451eb0bce\" (UID: \"24bc8b9f-7482-49a4-acc0-38a451eb0bce\") " Sep 13 20:22:53.050774 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.050782 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a4c4548-0ea8-4699-a1b4-1497348470e8-util\") pod \"9a4c4548-0ea8-4699-a1b4-1497348470e8\" (UID: \"9a4c4548-0ea8-4699-a1b4-1497348470e8\") " Sep 13 20:22:53.051026 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.050799 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79ce4c0b-e994-43db-9788-4f37b57f19b7-bundle\") pod \"79ce4c0b-e994-43db-9788-4f37b57f19b7\" (UID: \"79ce4c0b-e994-43db-9788-4f37b57f19b7\") " Sep 13 20:22:53.051026 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.050820 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lphpf\" (UniqueName: \"kubernetes.io/projected/9a4c4548-0ea8-4699-a1b4-1497348470e8-kube-api-access-lphpf\") pod \"9a4c4548-0ea8-4699-a1b4-1497348470e8\" (UID: \"9a4c4548-0ea8-4699-a1b4-1497348470e8\") " Sep 13 20:22:53.051026 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.050842 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24bc8b9f-7482-49a4-acc0-38a451eb0bce-util\") pod \"24bc8b9f-7482-49a4-acc0-38a451eb0bce\" (UID: \"24bc8b9f-7482-49a4-acc0-38a451eb0bce\") " Sep 13 20:22:53.051026 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.050871 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5bdnx\" (UniqueName: \"kubernetes.io/projected/79ce4c0b-e994-43db-9788-4f37b57f19b7-kube-api-access-5bdnx\") pod \"79ce4c0b-e994-43db-9788-4f37b57f19b7\" (UID: \"79ce4c0b-e994-43db-9788-4f37b57f19b7\") " Sep 13 20:22:53.051026 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.050889 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfr9c\" (UniqueName: \"kubernetes.io/projected/24bc8b9f-7482-49a4-acc0-38a451eb0bce-kube-api-access-hfr9c\") pod \"24bc8b9f-7482-49a4-acc0-38a451eb0bce\" (UID: \"24bc8b9f-7482-49a4-acc0-38a451eb0bce\") " Sep 13 20:22:53.051026 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.050923 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a4c4548-0ea8-4699-a1b4-1497348470e8-bundle\") pod \"9a4c4548-0ea8-4699-a1b4-1497348470e8\" (UID: \"9a4c4548-0ea8-4699-a1b4-1497348470e8\") " Sep 13 20:22:53.051026 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.050952 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79ce4c0b-e994-43db-9788-4f37b57f19b7-util\") pod \"79ce4c0b-e994-43db-9788-4f37b57f19b7\" (UID: \"79ce4c0b-e994-43db-9788-4f37b57f19b7\") " Sep 13 20:22:53.051367 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.051158 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2758e8d2-83b1-4168-9087-bfbe13d231a9-bundle\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:22:53.051367 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.051179 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2758e8d2-83b1-4168-9087-bfbe13d231a9-util\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:22:53.051367 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.051193 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prgxd\" (UniqueName: \"kubernetes.io/projected/2758e8d2-83b1-4168-9087-bfbe13d231a9-kube-api-access-prgxd\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:22:53.051561 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.051535 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24bc8b9f-7482-49a4-acc0-38a451eb0bce-bundle" (OuterVolumeSpecName: "bundle") pod "24bc8b9f-7482-49a4-acc0-38a451eb0bce" (UID: "24bc8b9f-7482-49a4-acc0-38a451eb0bce"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 13 20:22:53.051653 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.051626 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79ce4c0b-e994-43db-9788-4f37b57f19b7-bundle" (OuterVolumeSpecName: "bundle") pod "79ce4c0b-e994-43db-9788-4f37b57f19b7" (UID: "79ce4c0b-e994-43db-9788-4f37b57f19b7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 13 20:22:53.051970 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.051946 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a4c4548-0ea8-4699-a1b4-1497348470e8-bundle" (OuterVolumeSpecName: "bundle") pod "9a4c4548-0ea8-4699-a1b4-1497348470e8" (UID: "9a4c4548-0ea8-4699-a1b4-1497348470e8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 13 20:22:53.053099 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.053075 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a4c4548-0ea8-4699-a1b4-1497348470e8-kube-api-access-lphpf" (OuterVolumeSpecName: "kube-api-access-lphpf") pod "9a4c4548-0ea8-4699-a1b4-1497348470e8" (UID: "9a4c4548-0ea8-4699-a1b4-1497348470e8"). InnerVolumeSpecName "kube-api-access-lphpf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 20:22:53.053695 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.053671 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79ce4c0b-e994-43db-9788-4f37b57f19b7-kube-api-access-5bdnx" (OuterVolumeSpecName: "kube-api-access-5bdnx") pod "79ce4c0b-e994-43db-9788-4f37b57f19b7" (UID: "79ce4c0b-e994-43db-9788-4f37b57f19b7"). InnerVolumeSpecName "kube-api-access-5bdnx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 20:22:53.053880 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.053847 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24bc8b9f-7482-49a4-acc0-38a451eb0bce-kube-api-access-hfr9c" (OuterVolumeSpecName: "kube-api-access-hfr9c") pod "24bc8b9f-7482-49a4-acc0-38a451eb0bce" (UID: "24bc8b9f-7482-49a4-acc0-38a451eb0bce"). InnerVolumeSpecName "kube-api-access-hfr9c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 20:22:53.057025 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.056990 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24bc8b9f-7482-49a4-acc0-38a451eb0bce-util" (OuterVolumeSpecName: "util") pod "24bc8b9f-7482-49a4-acc0-38a451eb0bce" (UID: "24bc8b9f-7482-49a4-acc0-38a451eb0bce"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 13 20:22:53.057582 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.057560 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79ce4c0b-e994-43db-9788-4f37b57f19b7-util" (OuterVolumeSpecName: "util") pod "79ce4c0b-e994-43db-9788-4f37b57f19b7" (UID: "79ce4c0b-e994-43db-9788-4f37b57f19b7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 13 20:22:53.057652 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.057610 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a4c4548-0ea8-4699-a1b4-1497348470e8-util" (OuterVolumeSpecName: "util") pod "9a4c4548-0ea8-4699-a1b4-1497348470e8" (UID: "9a4c4548-0ea8-4699-a1b4-1497348470e8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 13 20:22:53.152075 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.152038 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79ce4c0b-e994-43db-9788-4f37b57f19b7-util\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:22:53.152075 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.152069 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24bc8b9f-7482-49a4-acc0-38a451eb0bce-bundle\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:22:53.152075 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.152078 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a4c4548-0ea8-4699-a1b4-1497348470e8-util\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:22:53.152075 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.152086 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79ce4c0b-e994-43db-9788-4f37b57f19b7-bundle\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:22:53.152342 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.152094 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lphpf\" (UniqueName: \"kubernetes.io/projected/9a4c4548-0ea8-4699-a1b4-1497348470e8-kube-api-access-lphpf\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:22:53.152342 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.152104 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24bc8b9f-7482-49a4-acc0-38a451eb0bce-util\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:22:53.152342 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.152113 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bdnx\" (UniqueName: \"kubernetes.io/projected/79ce4c0b-e994-43db-9788-4f37b57f19b7-kube-api-access-5bdnx\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:22:53.152342 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.152121 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfr9c\" (UniqueName: \"kubernetes.io/projected/24bc8b9f-7482-49a4-acc0-38a451eb0bce-kube-api-access-hfr9c\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:22:53.152342 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.152130 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a4c4548-0ea8-4699-a1b4-1497348470e8-bundle\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:22:53.715227 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.715191 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s" Sep 13 20:22:53.715227 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.715208 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6pz48s" event={"ID":"9a4c4548-0ea8-4699-a1b4-1497348470e8","Type":"ContainerDied","Data":"36416e3b683534ad909615f1b60e401211d722992187ad7672f48562cfcf865c"} Sep 13 20:22:53.715418 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.715258 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="36416e3b683534ad909615f1b60e401211d722992187ad7672f48562cfcf865c" Sep 13 20:22:53.717129 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.717110 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz" Sep 13 20:22:53.717129 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.717119 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amtkfz" event={"ID":"2758e8d2-83b1-4168-9087-bfbe13d231a9","Type":"ContainerDied","Data":"4cb2ff22c433c1d71268f49ed82897778971c7e409ddb72d5ed2b189b4f478b9"} Sep 13 20:22:53.717285 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.717153 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4cb2ff22c433c1d71268f49ed82897778971c7e409ddb72d5ed2b189b4f478b9" Sep 13 20:22:53.719159 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.719135 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24" event={"ID":"24bc8b9f-7482-49a4-acc0-38a451eb0bce","Type":"ContainerDied","Data":"0a49b18387dc0c1873ff0c7d3212c97e1c9d02e76554c5385e638b6ebc1df788"} Sep 13 20:22:53.719159 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.719155 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ccfb24" Sep 13 20:22:53.719314 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.719177 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0a49b18387dc0c1873ff0c7d3212c97e1c9d02e76554c5385e638b6ebc1df788" Sep 13 20:22:53.721410 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.721351 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb" event={"ID":"79ce4c0b-e994-43db-9788-4f37b57f19b7","Type":"ContainerDied","Data":"bf6123bcb6ca0abcca33a527fdd0180cc552a9ba1cd4c9a4156889906ef68500"} Sep 13 20:22:53.721410 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.721377 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t76nb" Sep 13 20:22:53.721410 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:53.721387 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf6123bcb6ca0abcca33a527fdd0180cc552a9ba1cd4c9a4156889906ef68500" Sep 13 20:22:58.926252 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926217 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7fn6r"] Sep 13 20:22:58.926658 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926632 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2758e8d2-83b1-4168-9087-bfbe13d231a9" containerName="util" Sep 13 20:22:58.926658 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926647 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2758e8d2-83b1-4168-9087-bfbe13d231a9" containerName="util" Sep 13 20:22:58.926658 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926654 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a4c4548-0ea8-4699-a1b4-1497348470e8" containerName="pull" Sep 13 20:22:58.926658 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926660 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a4c4548-0ea8-4699-a1b4-1497348470e8" containerName="pull" Sep 13 20:22:58.926786 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926669 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a4c4548-0ea8-4699-a1b4-1497348470e8" containerName="extract" Sep 13 20:22:58.926786 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926676 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a4c4548-0ea8-4699-a1b4-1497348470e8" containerName="extract" Sep 13 20:22:58.926786 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926683 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a4c4548-0ea8-4699-a1b4-1497348470e8" containerName="util" Sep 13 20:22:58.926786 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926688 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a4c4548-0ea8-4699-a1b4-1497348470e8" containerName="util" Sep 13 20:22:58.926786 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926695 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79ce4c0b-e994-43db-9788-4f37b57f19b7" containerName="pull" Sep 13 20:22:58.926786 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926701 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="79ce4c0b-e994-43db-9788-4f37b57f19b7" containerName="pull" Sep 13 20:22:58.926786 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926708 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79ce4c0b-e994-43db-9788-4f37b57f19b7" containerName="extract" Sep 13 20:22:58.926786 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926713 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="79ce4c0b-e994-43db-9788-4f37b57f19b7" containerName="extract" Sep 13 20:22:58.926786 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926721 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24bc8b9f-7482-49a4-acc0-38a451eb0bce" containerName="pull" Sep 13 20:22:58.926786 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926726 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="24bc8b9f-7482-49a4-acc0-38a451eb0bce" containerName="pull" Sep 13 20:22:58.926786 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926738 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24bc8b9f-7482-49a4-acc0-38a451eb0bce" containerName="extract" Sep 13 20:22:58.926786 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926743 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="24bc8b9f-7482-49a4-acc0-38a451eb0bce" containerName="extract" Sep 13 20:22:58.926786 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926749 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24bc8b9f-7482-49a4-acc0-38a451eb0bce" containerName="util" Sep 13 20:22:58.926786 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926753 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="24bc8b9f-7482-49a4-acc0-38a451eb0bce" containerName="util" Sep 13 20:22:58.926786 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926759 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79ce4c0b-e994-43db-9788-4f37b57f19b7" containerName="util" Sep 13 20:22:58.926786 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926764 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="79ce4c0b-e994-43db-9788-4f37b57f19b7" containerName="util" Sep 13 20:22:58.926786 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926771 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2758e8d2-83b1-4168-9087-bfbe13d231a9" containerName="extract" Sep 13 20:22:58.926786 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926776 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2758e8d2-83b1-4168-9087-bfbe13d231a9" containerName="extract" Sep 13 20:22:58.926786 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926783 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2758e8d2-83b1-4168-9087-bfbe13d231a9" containerName="pull" Sep 13 20:22:58.926786 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926788 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2758e8d2-83b1-4168-9087-bfbe13d231a9" containerName="pull" Sep 13 20:22:58.927362 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926837 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a4c4548-0ea8-4699-a1b4-1497348470e8" containerName="extract" Sep 13 20:22:58.927362 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926845 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="24bc8b9f-7482-49a4-acc0-38a451eb0bce" containerName="extract" Sep 13 20:22:58.927362 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926853 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2758e8d2-83b1-4168-9087-bfbe13d231a9" containerName="extract" Sep 13 20:22:58.927362 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.926859 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="79ce4c0b-e994-43db-9788-4f37b57f19b7" containerName="extract" Sep 13 20:22:58.930976 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.930957 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7fn6r" Sep 13 20:22:58.933542 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.933520 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-r7h8j\"" Sep 13 20:22:58.933659 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.933539 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 13 20:22:58.934355 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.934340 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 13 20:22:58.939618 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:58.939599 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7fn6r"] Sep 13 20:22:59.103158 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:59.103117 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjk46\" (UniqueName: \"kubernetes.io/projected/c3a5b2f3-51ba-4452-8e98-431502ba345c-kube-api-access-cjk46\") pod \"limitador-operator-controller-manager-74495c69f8-7fn6r\" (UID: \"c3a5b2f3-51ba-4452-8e98-431502ba345c\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7fn6r" Sep 13 20:22:59.204203 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:59.204118 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjk46\" (UniqueName: \"kubernetes.io/projected/c3a5b2f3-51ba-4452-8e98-431502ba345c-kube-api-access-cjk46\") pod \"limitador-operator-controller-manager-74495c69f8-7fn6r\" (UID: \"c3a5b2f3-51ba-4452-8e98-431502ba345c\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7fn6r" Sep 13 20:22:59.214837 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:59.214808 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjk46\" (UniqueName: \"kubernetes.io/projected/c3a5b2f3-51ba-4452-8e98-431502ba345c-kube-api-access-cjk46\") pod \"limitador-operator-controller-manager-74495c69f8-7fn6r\" (UID: \"c3a5b2f3-51ba-4452-8e98-431502ba345c\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7fn6r" Sep 13 20:22:59.242646 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:59.242620 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7fn6r" Sep 13 20:22:59.514704 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:59.514673 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7fn6r"] Sep 13 20:22:59.516401 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:22:59.516370 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3a5b2f3_51ba_4452_8e98_431502ba345c.slice/crio-37ceb3ca86f08c8d04212e91a51c8968b8c43539154f486ef0b4d6c3f527ea08 WatchSource:0}: Error finding container 37ceb3ca86f08c8d04212e91a51c8968b8c43539154f486ef0b4d6c3f527ea08: Status 404 returned error can't find the container with id 37ceb3ca86f08c8d04212e91a51c8968b8c43539154f486ef0b4d6c3f527ea08 Sep 13 20:22:59.518534 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:59.518514 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 13 20:22:59.757252 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:22:59.757221 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7fn6r" event={"ID":"c3a5b2f3-51ba-4452-8e98-431502ba345c","Type":"ContainerStarted","Data":"37ceb3ca86f08c8d04212e91a51c8968b8c43539154f486ef0b4d6c3f527ea08"} Sep 13 20:23:00.935531 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:00.920868 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-svnhq"] Sep 13 20:23:00.943364 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:00.943300 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-svnhq" Sep 13 20:23:00.945068 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:00.945039 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-svnhq"] Sep 13 20:23:00.946513 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:00.946473 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-st4sf\"" Sep 13 20:23:00.946621 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:00.946521 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Sep 13 20:23:01.120954 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:01.120908 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjzcl\" (UniqueName: \"kubernetes.io/projected/4d37eabc-6cf5-446b-9df3-b5f587e16ee2-kube-api-access-wjzcl\") pod \"dns-operator-controller-manager-7b84cd7c9c-svnhq\" (UID: \"4d37eabc-6cf5-446b-9df3-b5f587e16ee2\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-svnhq" Sep 13 20:23:01.222043 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:01.221938 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjzcl\" (UniqueName: \"kubernetes.io/projected/4d37eabc-6cf5-446b-9df3-b5f587e16ee2-kube-api-access-wjzcl\") pod \"dns-operator-controller-manager-7b84cd7c9c-svnhq\" (UID: \"4d37eabc-6cf5-446b-9df3-b5f587e16ee2\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-svnhq" Sep 13 20:23:01.231270 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:01.231225 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjzcl\" (UniqueName: \"kubernetes.io/projected/4d37eabc-6cf5-446b-9df3-b5f587e16ee2-kube-api-access-wjzcl\") pod \"dns-operator-controller-manager-7b84cd7c9c-svnhq\" (UID: \"4d37eabc-6cf5-446b-9df3-b5f587e16ee2\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-svnhq" Sep 13 20:23:01.258860 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:01.258830 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-svnhq" Sep 13 20:23:01.544987 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:01.544960 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-svnhq"] Sep 13 20:23:02.522691 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:23:02.522660 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d37eabc_6cf5_446b_9df3_b5f587e16ee2.slice/crio-ef7f77e1da8aaafc13318e920165e570e1f615f02b1328c2557a08883d1e0ce6 WatchSource:0}: Error finding container ef7f77e1da8aaafc13318e920165e570e1f615f02b1328c2557a08883d1e0ce6: Status 404 returned error can't find the container with id ef7f77e1da8aaafc13318e920165e570e1f615f02b1328c2557a08883d1e0ce6 Sep 13 20:23:02.776260 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:02.776159 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-svnhq" event={"ID":"4d37eabc-6cf5-446b-9df3-b5f587e16ee2","Type":"ContainerStarted","Data":"ef7f77e1da8aaafc13318e920165e570e1f615f02b1328c2557a08883d1e0ce6"} Sep 13 20:23:02.777896 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:02.777873 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7fn6r" event={"ID":"c3a5b2f3-51ba-4452-8e98-431502ba345c","Type":"ContainerStarted","Data":"d16d2fe4c51bb516a0446638b6d7e6c77f0ef0d462407c5a62fb932afb98e4ff"} Sep 13 20:23:02.778016 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:02.777932 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7fn6r" Sep 13 20:23:02.802081 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:02.802028 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7fn6r" podStartSLOduration=1.7344378310000002 podStartE2EDuration="4.802013563s" totalImagesPullingTime="3.067575732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:22:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:23:02.8001784 +0000 UTC m=+614.976614423" watchObservedRunningTime="2026-09-13 20:23:02.802013563 +0000 UTC m=+614.978449585" Sep 13 20:23:04.790647 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:04.790609 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-svnhq" event={"ID":"4d37eabc-6cf5-446b-9df3-b5f587e16ee2","Type":"ContainerStarted","Data":"e29353e41c8347e5b188c8c59eb6897da5e4cc04f2157d24368ca8c7f2a7f75a"} Sep 13 20:23:04.791065 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:04.790723 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-svnhq" Sep 13 20:23:04.810181 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:04.810127 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-svnhq" podStartSLOduration=3.155702578 podStartE2EDuration="4.81011242s" totalImagesPullingTime="1.654409842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:23:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:23:04.808633627 +0000 UTC m=+616.985069648" watchObservedRunningTime="2026-09-13 20:23:04.81011242 +0000 UTC m=+616.986548442" Sep 13 20:23:05.504980 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:05.504948 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8klnp"] Sep 13 20:23:05.508438 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:05.508421 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8klnp" Sep 13 20:23:05.510784 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:05.510763 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-zpmtn\"" Sep 13 20:23:05.520416 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:05.520392 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8klnp"] Sep 13 20:23:05.665289 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:05.665249 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4qqh\" (UniqueName: \"kubernetes.io/projected/8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9-kube-api-access-v4qqh\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8klnp\" (UID: \"8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8klnp" Sep 13 20:23:05.665450 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:05.665298 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8klnp\" (UID: \"8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8klnp" Sep 13 20:23:05.766028 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:05.765936 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4qqh\" (UniqueName: \"kubernetes.io/projected/8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9-kube-api-access-v4qqh\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8klnp\" (UID: \"8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8klnp" Sep 13 20:23:05.766028 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:05.765977 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8klnp\" (UID: \"8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8klnp" Sep 13 20:23:05.766304 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:05.766288 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8klnp\" (UID: \"8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8klnp" Sep 13 20:23:05.773175 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:05.773149 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4qqh\" (UniqueName: \"kubernetes.io/projected/8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9-kube-api-access-v4qqh\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8klnp\" (UID: \"8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8klnp" Sep 13 20:23:05.818389 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:05.818363 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8klnp" Sep 13 20:23:06.103168 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:06.103142 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8klnp"] Sep 13 20:23:06.104981 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:23:06.104952 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f4c8535_e3bc_43dd_bb5a_dd0dafd7e8f9.slice/crio-b86d4b4fc0fc329a957c63a8fbfa07505713df5300977630b1ed431642dd2d83 WatchSource:0}: Error finding container b86d4b4fc0fc329a957c63a8fbfa07505713df5300977630b1ed431642dd2d83: Status 404 returned error can't find the container with id b86d4b4fc0fc329a957c63a8fbfa07505713df5300977630b1ed431642dd2d83 Sep 13 20:23:06.802402 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:06.802365 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8klnp" event={"ID":"8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9","Type":"ContainerStarted","Data":"b86d4b4fc0fc329a957c63a8fbfa07505713df5300977630b1ed431642dd2d83"} Sep 13 20:23:08.276938 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:08.276904 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-vg8n7"] Sep 13 20:23:08.280721 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:08.280700 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-vg8n7" Sep 13 20:23:08.282897 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:08.282874 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-ppcqh\"" Sep 13 20:23:08.290818 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:08.290575 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-vg8n7"] Sep 13 20:23:08.388687 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:08.388647 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx5z2\" (UniqueName: \"kubernetes.io/projected/3df43dde-d53f-4927-a6a8-68a04b102382-kube-api-access-dx5z2\") pod \"authorino-operator-76b478bd78-vg8n7\" (UID: \"3df43dde-d53f-4927-a6a8-68a04b102382\") " pod="kuadrant-system/authorino-operator-76b478bd78-vg8n7" Sep 13 20:23:08.489354 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:08.489317 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx5z2\" (UniqueName: \"kubernetes.io/projected/3df43dde-d53f-4927-a6a8-68a04b102382-kube-api-access-dx5z2\") pod \"authorino-operator-76b478bd78-vg8n7\" (UID: \"3df43dde-d53f-4927-a6a8-68a04b102382\") " pod="kuadrant-system/authorino-operator-76b478bd78-vg8n7" Sep 13 20:23:08.507009 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:08.506979 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx5z2\" (UniqueName: \"kubernetes.io/projected/3df43dde-d53f-4927-a6a8-68a04b102382-kube-api-access-dx5z2\") pod \"authorino-operator-76b478bd78-vg8n7\" (UID: \"3df43dde-d53f-4927-a6a8-68a04b102382\") " pod="kuadrant-system/authorino-operator-76b478bd78-vg8n7" Sep 13 20:23:08.595970 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:08.595883 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-vg8n7" Sep 13 20:23:08.877671 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:08.877640 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-vg8n7"] Sep 13 20:23:09.975274 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:23:09.975243 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3df43dde_d53f_4927_a6a8_68a04b102382.slice/crio-9de27330e2da3d400b36c741e4d2dc6d22c5c8c3db3ff3feefdfd3b39c70e208 WatchSource:0}: Error finding container 9de27330e2da3d400b36c741e4d2dc6d22c5c8c3db3ff3feefdfd3b39c70e208: Status 404 returned error can't find the container with id 9de27330e2da3d400b36c741e4d2dc6d22c5c8c3db3ff3feefdfd3b39c70e208 Sep 13 20:23:10.826530 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:10.826423 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8klnp" event={"ID":"8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9","Type":"ContainerStarted","Data":"5319d507d8d401349e50592fc4e119fac3cf59c798a1776603b2ac2260096d8d"} Sep 13 20:23:10.826530 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:10.826518 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8klnp" Sep 13 20:23:10.827816 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:10.827791 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-vg8n7" event={"ID":"3df43dde-d53f-4927-a6a8-68a04b102382","Type":"ContainerStarted","Data":"9de27330e2da3d400b36c741e4d2dc6d22c5c8c3db3ff3feefdfd3b39c70e208"} Sep 13 20:23:10.845724 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:10.845676 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8klnp" podStartSLOduration=1.909663517 podStartE2EDuration="5.845661115s" totalImagesPullingTime="3.935997598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:23:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:23:10.843253612 +0000 UTC m=+623.019689634" watchObservedRunningTime="2026-09-13 20:23:10.845661115 +0000 UTC m=+623.022097136" Sep 13 20:23:11.834575 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:11.834531 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-vg8n7" event={"ID":"3df43dde-d53f-4927-a6a8-68a04b102382","Type":"ContainerStarted","Data":"3aab61ad6eed5ca53a3ac7ea265cdea91c91738a7d77b2a393afb0696b593b07"} Sep 13 20:23:11.835026 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:11.834630 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-vg8n7" Sep 13 20:23:11.851941 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:11.851887 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-vg8n7" podStartSLOduration=2.184003675 podStartE2EDuration="3.851873049s" totalImagesPullingTime="1.667869374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:23:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:23:11.849991878 +0000 UTC m=+624.026427902" watchObservedRunningTime="2026-09-13 20:23:11.851873049 +0000 UTC m=+624.028309070" Sep 13 20:23:13.786269 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:13.786233 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7fn6r" Sep 13 20:23:15.798605 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:15.798569 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-svnhq" Sep 13 20:23:20.660123 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.660086 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f897b78bf-cgxtr"] Sep 13 20:23:20.663990 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.663968 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f897b78bf-cgxtr" Sep 13 20:23:20.666178 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.666148 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 13 20:23:20.666445 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.666433 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 13 20:23:20.666512 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.666440 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 13 20:23:20.667345 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.667298 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 13 20:23:20.667345 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.667332 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-njzsp\"" Sep 13 20:23:20.667642 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.667544 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 13 20:23:20.668166 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.668135 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 13 20:23:20.668277 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.668178 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 13 20:23:20.679692 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.679665 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f897b78bf-cgxtr"] Sep 13 20:23:20.680432 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.680407 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 13 20:23:20.692595 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.692569 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2270b1ff-883b-4ec7-b459-4e628c952d16-console-oauth-config\") pod \"console-6f897b78bf-cgxtr\" (UID: \"2270b1ff-883b-4ec7-b459-4e628c952d16\") " pod="openshift-console/console-6f897b78bf-cgxtr" Sep 13 20:23:20.692746 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.692625 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2270b1ff-883b-4ec7-b459-4e628c952d16-console-serving-cert\") pod \"console-6f897b78bf-cgxtr\" (UID: \"2270b1ff-883b-4ec7-b459-4e628c952d16\") " pod="openshift-console/console-6f897b78bf-cgxtr" Sep 13 20:23:20.692746 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.692710 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2270b1ff-883b-4ec7-b459-4e628c952d16-oauth-serving-cert\") pod \"console-6f897b78bf-cgxtr\" (UID: \"2270b1ff-883b-4ec7-b459-4e628c952d16\") " pod="openshift-console/console-6f897b78bf-cgxtr" Sep 13 20:23:20.692834 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.692757 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2270b1ff-883b-4ec7-b459-4e628c952d16-service-ca\") pod \"console-6f897b78bf-cgxtr\" (UID: \"2270b1ff-883b-4ec7-b459-4e628c952d16\") " pod="openshift-console/console-6f897b78bf-cgxtr" Sep 13 20:23:20.692834 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.692790 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2270b1ff-883b-4ec7-b459-4e628c952d16-console-config\") pod \"console-6f897b78bf-cgxtr\" (UID: \"2270b1ff-883b-4ec7-b459-4e628c952d16\") " pod="openshift-console/console-6f897b78bf-cgxtr" Sep 13 20:23:20.692915 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.692847 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf68b\" (UniqueName: \"kubernetes.io/projected/2270b1ff-883b-4ec7-b459-4e628c952d16-kube-api-access-xf68b\") pod \"console-6f897b78bf-cgxtr\" (UID: \"2270b1ff-883b-4ec7-b459-4e628c952d16\") " pod="openshift-console/console-6f897b78bf-cgxtr" Sep 13 20:23:20.692915 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.692887 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2270b1ff-883b-4ec7-b459-4e628c952d16-trusted-ca-bundle\") pod \"console-6f897b78bf-cgxtr\" (UID: \"2270b1ff-883b-4ec7-b459-4e628c952d16\") " pod="openshift-console/console-6f897b78bf-cgxtr" Sep 13 20:23:20.794126 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.794092 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf68b\" (UniqueName: \"kubernetes.io/projected/2270b1ff-883b-4ec7-b459-4e628c952d16-kube-api-access-xf68b\") pod \"console-6f897b78bf-cgxtr\" (UID: \"2270b1ff-883b-4ec7-b459-4e628c952d16\") " pod="openshift-console/console-6f897b78bf-cgxtr" Sep 13 20:23:20.794126 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.794131 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2270b1ff-883b-4ec7-b459-4e628c952d16-trusted-ca-bundle\") pod \"console-6f897b78bf-cgxtr\" (UID: \"2270b1ff-883b-4ec7-b459-4e628c952d16\") " pod="openshift-console/console-6f897b78bf-cgxtr" Sep 13 20:23:20.794366 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.794159 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2270b1ff-883b-4ec7-b459-4e628c952d16-console-oauth-config\") pod \"console-6f897b78bf-cgxtr\" (UID: \"2270b1ff-883b-4ec7-b459-4e628c952d16\") " pod="openshift-console/console-6f897b78bf-cgxtr" Sep 13 20:23:20.794366 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.794182 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2270b1ff-883b-4ec7-b459-4e628c952d16-console-serving-cert\") pod \"console-6f897b78bf-cgxtr\" (UID: \"2270b1ff-883b-4ec7-b459-4e628c952d16\") " pod="openshift-console/console-6f897b78bf-cgxtr" Sep 13 20:23:20.794366 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.794207 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2270b1ff-883b-4ec7-b459-4e628c952d16-oauth-serving-cert\") pod \"console-6f897b78bf-cgxtr\" (UID: \"2270b1ff-883b-4ec7-b459-4e628c952d16\") " pod="openshift-console/console-6f897b78bf-cgxtr" Sep 13 20:23:20.794366 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.794225 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2270b1ff-883b-4ec7-b459-4e628c952d16-service-ca\") pod \"console-6f897b78bf-cgxtr\" (UID: \"2270b1ff-883b-4ec7-b459-4e628c952d16\") " pod="openshift-console/console-6f897b78bf-cgxtr" Sep 13 20:23:20.794366 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.794245 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2270b1ff-883b-4ec7-b459-4e628c952d16-console-config\") pod \"console-6f897b78bf-cgxtr\" (UID: \"2270b1ff-883b-4ec7-b459-4e628c952d16\") " pod="openshift-console/console-6f897b78bf-cgxtr" Sep 13 20:23:20.795141 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.795107 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2270b1ff-883b-4ec7-b459-4e628c952d16-oauth-serving-cert\") pod \"console-6f897b78bf-cgxtr\" (UID: \"2270b1ff-883b-4ec7-b459-4e628c952d16\") " pod="openshift-console/console-6f897b78bf-cgxtr" Sep 13 20:23:20.795141 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.795127 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2270b1ff-883b-4ec7-b459-4e628c952d16-console-config\") pod \"console-6f897b78bf-cgxtr\" (UID: \"2270b1ff-883b-4ec7-b459-4e628c952d16\") " pod="openshift-console/console-6f897b78bf-cgxtr" Sep 13 20:23:20.795332 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.795143 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2270b1ff-883b-4ec7-b459-4e628c952d16-service-ca\") pod \"console-6f897b78bf-cgxtr\" (UID: \"2270b1ff-883b-4ec7-b459-4e628c952d16\") " pod="openshift-console/console-6f897b78bf-cgxtr" Sep 13 20:23:20.795332 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.795120 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2270b1ff-883b-4ec7-b459-4e628c952d16-trusted-ca-bundle\") pod \"console-6f897b78bf-cgxtr\" (UID: \"2270b1ff-883b-4ec7-b459-4e628c952d16\") " pod="openshift-console/console-6f897b78bf-cgxtr" Sep 13 20:23:20.796803 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.796772 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2270b1ff-883b-4ec7-b459-4e628c952d16-console-serving-cert\") pod \"console-6f897b78bf-cgxtr\" (UID: \"2270b1ff-883b-4ec7-b459-4e628c952d16\") " pod="openshift-console/console-6f897b78bf-cgxtr" Sep 13 20:23:20.796906 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.796829 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2270b1ff-883b-4ec7-b459-4e628c952d16-console-oauth-config\") pod \"console-6f897b78bf-cgxtr\" (UID: \"2270b1ff-883b-4ec7-b459-4e628c952d16\") " pod="openshift-console/console-6f897b78bf-cgxtr" Sep 13 20:23:20.802626 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.802605 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf68b\" (UniqueName: \"kubernetes.io/projected/2270b1ff-883b-4ec7-b459-4e628c952d16-kube-api-access-xf68b\") pod \"console-6f897b78bf-cgxtr\" (UID: \"2270b1ff-883b-4ec7-b459-4e628c952d16\") " pod="openshift-console/console-6f897b78bf-cgxtr" Sep 13 20:23:20.976331 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:20.976233 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f897b78bf-cgxtr" Sep 13 20:23:21.250484 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:21.250455 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f897b78bf-cgxtr"] Sep 13 20:23:21.251951 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:23:21.251922 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2270b1ff_883b_4ec7_b459_4e628c952d16.slice/crio-52c6968ebf00ed16a094386b4e98a6959c1e5a8d35ceea7a8e786ea38bb7ba5d WatchSource:0}: Error finding container 52c6968ebf00ed16a094386b4e98a6959c1e5a8d35ceea7a8e786ea38bb7ba5d: Status 404 returned error can't find the container with id 52c6968ebf00ed16a094386b4e98a6959c1e5a8d35ceea7a8e786ea38bb7ba5d Sep 13 20:23:21.837246 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:21.837216 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8klnp" Sep 13 20:23:21.892741 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:21.892691 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f897b78bf-cgxtr" event={"ID":"2270b1ff-883b-4ec7-b459-4e628c952d16","Type":"ContainerStarted","Data":"c949508f47c3f7ff7cb69cde8d4d8411ea3c39559543aea78e151765c4346856"} Sep 13 20:23:21.892930 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:21.892768 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f897b78bf-cgxtr" event={"ID":"2270b1ff-883b-4ec7-b459-4e628c952d16","Type":"ContainerStarted","Data":"52c6968ebf00ed16a094386b4e98a6959c1e5a8d35ceea7a8e786ea38bb7ba5d"} Sep 13 20:23:21.913940 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:21.913886 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f897b78bf-cgxtr" podStartSLOduration=1.913867166 podStartE2EDuration="1.913867166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:23:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:23:21.912235023 +0000 UTC m=+634.088671048" watchObservedRunningTime="2026-09-13 20:23:21.913867166 +0000 UTC m=+634.090303198" Sep 13 20:23:22.843328 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:22.843298 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-vg8n7" Sep 13 20:23:23.869244 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:23.869192 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8klnp"] Sep 13 20:23:23.869735 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:23.869417 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8klnp" podUID="8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9" containerName="manager" containerID="cri-o://5319d507d8d401349e50592fc4e119fac3cf59c798a1776603b2ac2260096d8d" gracePeriod=2 Sep 13 20:23:23.871392 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:23:23.871362 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-8klnp\" is forbidden: User \"system:node:ip-10-0-134-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-99.ec2.internal' and this object" podUID="8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8klnp" Sep 13 20:23:23.871715 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:23.871686 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8klnp"] Sep 13 20:23:23.887488 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:23.887458 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7fn6r"] Sep 13 20:23:23.887710 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:23.887683 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7fn6r" podUID="c3a5b2f3-51ba-4452-8e98-431502ba345c" containerName="manager" containerID="cri-o://d16d2fe4c51bb516a0446638b6d7e6c77f0ef0d462407c5a62fb932afb98e4ff" gracePeriod=2 Sep 13 20:23:23.889830 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:23:23.889700 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-8klnp\" is forbidden: User \"system:node:ip-10-0-134-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-99.ec2.internal' and this object" podUID="8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8klnp" Sep 13 20:23:23.892615 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:23.892592 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vl6vt"] Sep 13 20:23:23.892945 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:23.892932 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9" containerName="manager" Sep 13 20:23:23.892991 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:23.892947 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9" containerName="manager" Sep 13 20:23:23.893030 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:23.893015 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9" containerName="manager" Sep 13 20:23:23.896163 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:23.896140 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vl6vt" Sep 13 20:23:23.898184 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:23.898160 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7fn6r"] Sep 13 20:23:23.902142 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:23:23.902104 2557 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"c3a5b2f3-51ba-4452-8e98-431502ba345c\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-74495c69f8-7fn6r\": pods \"limitador-operator-controller-manager-74495c69f8-7fn6r\" not found" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7fn6r" Sep 13 20:23:23.904042 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:23:23.904009 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-8klnp\" is forbidden: User \"system:node:ip-10-0-134-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-99.ec2.internal' and this object" podUID="8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8klnp" Sep 13 20:23:23.906697 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:23:23.906575 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7fn6r\" is forbidden: User \"system:node:ip-10-0-134-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-99.ec2.internal' and this object" podUID="c3a5b2f3-51ba-4452-8e98-431502ba345c" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7fn6r" Sep 13 20:23:23.910728 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:23.910702 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vl6vt"] Sep 13 20:23:23.917190 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:23.917162 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-rf847"] Sep 13 20:23:23.917932 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:23.917911 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3a5b2f3-51ba-4452-8e98-431502ba345c" containerName="manager" Sep 13 20:23:23.918069 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:23.917935 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3a5b2f3-51ba-4452-8e98-431502ba345c" containerName="manager" Sep 13 20:23:23.918162 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:23.918146 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3a5b2f3-51ba-4452-8e98-431502ba345c" containerName="manager" Sep 13 20:23:23.924781 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:23.924757 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rf847" Sep 13 20:23:23.927173 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:23.927151 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-rf847"] Sep 13 20:23:23.958030 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:23:23.957997 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-8klnp\" is forbidden: User \"system:node:ip-10-0-134-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-99.ec2.internal' and this object" podUID="8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8klnp" Sep 13 20:23:23.959856 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:23:23.959824 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7fn6r\" is forbidden: User \"system:node:ip-10-0-134-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-99.ec2.internal' and this object" podUID="c3a5b2f3-51ba-4452-8e98-431502ba345c" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7fn6r" Sep 13 20:23:24.022346 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.022310 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50fa69f0-09aa-4302-a902-65821965aa38-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-vl6vt\" (UID: \"50fa69f0-09aa-4302-a902-65821965aa38\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vl6vt" Sep 13 20:23:24.022477 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.022358 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brm4l\" (UniqueName: \"kubernetes.io/projected/931353e1-ce74-422f-8946-ae3dc57e17d5-kube-api-access-brm4l\") pod \"limitador-operator-controller-manager-74495c69f8-rf847\" (UID: \"931353e1-ce74-422f-8946-ae3dc57e17d5\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rf847" Sep 13 20:23:24.022565 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.022509 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s75zk\" (UniqueName: \"kubernetes.io/projected/50fa69f0-09aa-4302-a902-65821965aa38-kube-api-access-s75zk\") pod \"kuadrant-operator-controller-manager-76f8f686bf-vl6vt\" (UID: \"50fa69f0-09aa-4302-a902-65821965aa38\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vl6vt" Sep 13 20:23:24.123787 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.123705 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50fa69f0-09aa-4302-a902-65821965aa38-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-vl6vt\" (UID: \"50fa69f0-09aa-4302-a902-65821965aa38\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vl6vt" Sep 13 20:23:24.123787 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.123766 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brm4l\" (UniqueName: \"kubernetes.io/projected/931353e1-ce74-422f-8946-ae3dc57e17d5-kube-api-access-brm4l\") pod \"limitador-operator-controller-manager-74495c69f8-rf847\" (UID: \"931353e1-ce74-422f-8946-ae3dc57e17d5\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rf847" Sep 13 20:23:24.124002 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.123856 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s75zk\" (UniqueName: \"kubernetes.io/projected/50fa69f0-09aa-4302-a902-65821965aa38-kube-api-access-s75zk\") pod \"kuadrant-operator-controller-manager-76f8f686bf-vl6vt\" (UID: \"50fa69f0-09aa-4302-a902-65821965aa38\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vl6vt" Sep 13 20:23:24.124175 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.124152 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50fa69f0-09aa-4302-a902-65821965aa38-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-vl6vt\" (UID: \"50fa69f0-09aa-4302-a902-65821965aa38\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vl6vt" Sep 13 20:23:24.137828 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.137800 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brm4l\" (UniqueName: \"kubernetes.io/projected/931353e1-ce74-422f-8946-ae3dc57e17d5-kube-api-access-brm4l\") pod \"limitador-operator-controller-manager-74495c69f8-rf847\" (UID: \"931353e1-ce74-422f-8946-ae3dc57e17d5\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rf847" Sep 13 20:23:24.138262 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.138240 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s75zk\" (UniqueName: \"kubernetes.io/projected/50fa69f0-09aa-4302-a902-65821965aa38-kube-api-access-s75zk\") pod \"kuadrant-operator-controller-manager-76f8f686bf-vl6vt\" (UID: \"50fa69f0-09aa-4302-a902-65821965aa38\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vl6vt" Sep 13 20:23:24.157186 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.157154 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7fn6r" Sep 13 20:23:24.159233 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:23:24.159207 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-8klnp\" is forbidden: User \"system:node:ip-10-0-134-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-99.ec2.internal' and this object" podUID="8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8klnp" Sep 13 20:23:24.159687 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.159670 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8klnp" Sep 13 20:23:24.161012 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:23:24.160990 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7fn6r\" is forbidden: User \"system:node:ip-10-0-134-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-99.ec2.internal' and this object" podUID="c3a5b2f3-51ba-4452-8e98-431502ba345c" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7fn6r" Sep 13 20:23:24.162728 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:23:24.162705 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7fn6r\" is forbidden: User \"system:node:ip-10-0-134-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-99.ec2.internal' and this object" podUID="c3a5b2f3-51ba-4452-8e98-431502ba345c" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7fn6r" Sep 13 20:23:24.164417 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:23:24.164400 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-8klnp\" is forbidden: User \"system:node:ip-10-0-134-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-99.ec2.internal' and this object" podUID="8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8klnp" Sep 13 20:23:24.224999 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.224970 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9-extensions-socket-volume\") pod \"8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9\" (UID: \"8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9\") " Sep 13 20:23:24.225142 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.225036 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4qqh\" (UniqueName: \"kubernetes.io/projected/8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9-kube-api-access-v4qqh\") pod \"8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9\" (UID: \"8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9\") " Sep 13 20:23:24.225142 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.225057 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjk46\" (UniqueName: \"kubernetes.io/projected/c3a5b2f3-51ba-4452-8e98-431502ba345c-kube-api-access-cjk46\") pod \"c3a5b2f3-51ba-4452-8e98-431502ba345c\" (UID: \"c3a5b2f3-51ba-4452-8e98-431502ba345c\") " Sep 13 20:23:24.225465 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.225437 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9" (UID: "8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 13 20:23:24.227248 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.227227 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3a5b2f3-51ba-4452-8e98-431502ba345c-kube-api-access-cjk46" (OuterVolumeSpecName: "kube-api-access-cjk46") pod "c3a5b2f3-51ba-4452-8e98-431502ba345c" (UID: "c3a5b2f3-51ba-4452-8e98-431502ba345c"). InnerVolumeSpecName "kube-api-access-cjk46". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 20:23:24.227314 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.227267 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9-kube-api-access-v4qqh" (OuterVolumeSpecName: "kube-api-access-v4qqh") pod "8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9" (UID: "8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9"). InnerVolumeSpecName "kube-api-access-v4qqh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 20:23:24.311187 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.311148 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vl6vt" Sep 13 20:23:24.312305 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.312276 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9" path="/var/lib/kubelet/pods/8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9/volumes" Sep 13 20:23:24.312701 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.312685 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3a5b2f3-51ba-4452-8e98-431502ba345c" path="/var/lib/kubelet/pods/c3a5b2f3-51ba-4452-8e98-431502ba345c/volumes" Sep 13 20:23:24.319220 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.319198 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rf847" Sep 13 20:23:24.326376 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.326354 2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9-extensions-socket-volume\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:23:24.326376 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.326377 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4qqh\" (UniqueName: \"kubernetes.io/projected/8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9-kube-api-access-v4qqh\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:23:24.326522 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.326387 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjk46\" (UniqueName: \"kubernetes.io/projected/c3a5b2f3-51ba-4452-8e98-431502ba345c-kube-api-access-cjk46\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:23:24.593877 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.593844 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vl6vt"] Sep 13 20:23:24.595343 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:23:24.595320 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50fa69f0_09aa_4302_a902_65821965aa38.slice/crio-3e42377d56840f1e347cf5ffe4081f173a5d4b6b7250cc6c20107fe83d897654 WatchSource:0}: Error finding container 3e42377d56840f1e347cf5ffe4081f173a5d4b6b7250cc6c20107fe83d897654: Status 404 returned error can't find the container with id 3e42377d56840f1e347cf5ffe4081f173a5d4b6b7250cc6c20107fe83d897654 Sep 13 20:23:24.604290 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.604269 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-rf847"] Sep 13 20:23:24.608740 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:23:24.608715 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod931353e1_ce74_422f_8946_ae3dc57e17d5.slice/crio-4c3abd3a5c102871fddadef0547c05101a99464d2a8d54b753c5a26c1c1590fa WatchSource:0}: Error finding container 4c3abd3a5c102871fddadef0547c05101a99464d2a8d54b753c5a26c1c1590fa: Status 404 returned error can't find the container with id 4c3abd3a5c102871fddadef0547c05101a99464d2a8d54b753c5a26c1c1590fa Sep 13 20:23:24.709732 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.709704 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gqdcv"] Sep 13 20:23:24.713147 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.713126 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gqdcv" Sep 13 20:23:24.726881 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.726855 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gqdcv"] Sep 13 20:23:24.830818 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.830775 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a979679d-9a45-496a-a13a-e31d54844034-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-gqdcv\" (UID: \"a979679d-9a45-496a-a13a-e31d54844034\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gqdcv" Sep 13 20:23:24.830998 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.830826 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7f8h\" (UniqueName: \"kubernetes.io/projected/a979679d-9a45-496a-a13a-e31d54844034-kube-api-access-d7f8h\") pod \"kuadrant-operator-controller-manager-5769f99d9f-gqdcv\" (UID: \"a979679d-9a45-496a-a13a-e31d54844034\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gqdcv" Sep 13 20:23:24.913802 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.913765 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rf847" event={"ID":"931353e1-ce74-422f-8946-ae3dc57e17d5","Type":"ContainerStarted","Data":"742e8657a6ad7b08f4e7f8bb4488e4799a57bdf337042b3ec297984056606167"} Sep 13 20:23:24.914206 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.913830 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rf847" event={"ID":"931353e1-ce74-422f-8946-ae3dc57e17d5","Type":"ContainerStarted","Data":"4c3abd3a5c102871fddadef0547c05101a99464d2a8d54b753c5a26c1c1590fa"} Sep 13 20:23:24.914206 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.913864 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rf847" Sep 13 20:23:24.915226 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.915198 2557 generic.go:356] "Generic (PLEG): container finished" podID="8f4c8535-e3bc-43dd-bb5a-dd0dafd7e8f9" containerID="5319d507d8d401349e50592fc4e119fac3cf59c798a1776603b2ac2260096d8d" exitCode=0 Sep 13 20:23:24.915340 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.915239 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8klnp" Sep 13 20:23:24.915340 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.915300 2557 scope.go:117] "RemoveContainer" containerID="5319d507d8d401349e50592fc4e119fac3cf59c798a1776603b2ac2260096d8d" Sep 13 20:23:24.917396 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.917372 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vl6vt" event={"ID":"50fa69f0-09aa-4302-a902-65821965aa38","Type":"ContainerStarted","Data":"0876712d15fa9cbab541f8fd03f867564dc423f088fd21717229806a600b9289"} Sep 13 20:23:24.917513 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.917421 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vl6vt" event={"ID":"50fa69f0-09aa-4302-a902-65821965aa38","Type":"ContainerStarted","Data":"3e42377d56840f1e347cf5ffe4081f173a5d4b6b7250cc6c20107fe83d897654"} Sep 13 20:23:24.917635 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.917616 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vl6vt" Sep 13 20:23:24.918918 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.918895 2557 generic.go:356] "Generic (PLEG): container finished" podID="c3a5b2f3-51ba-4452-8e98-431502ba345c" containerID="d16d2fe4c51bb516a0446638b6d7e6c77f0ef0d462407c5a62fb932afb98e4ff" exitCode=0 Sep 13 20:23:24.918918 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.918926 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7fn6r" Sep 13 20:23:24.925030 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.925008 2557 scope.go:117] "RemoveContainer" containerID="5319d507d8d401349e50592fc4e119fac3cf59c798a1776603b2ac2260096d8d" Sep 13 20:23:24.925276 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:23:24.925260 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5319d507d8d401349e50592fc4e119fac3cf59c798a1776603b2ac2260096d8d\": container with ID starting with 5319d507d8d401349e50592fc4e119fac3cf59c798a1776603b2ac2260096d8d not found: ID does not exist" containerID="5319d507d8d401349e50592fc4e119fac3cf59c798a1776603b2ac2260096d8d" Sep 13 20:23:24.925333 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.925281 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5319d507d8d401349e50592fc4e119fac3cf59c798a1776603b2ac2260096d8d"} err="failed to get container status \"5319d507d8d401349e50592fc4e119fac3cf59c798a1776603b2ac2260096d8d\": rpc error: code = NotFound desc = could not find container \"5319d507d8d401349e50592fc4e119fac3cf59c798a1776603b2ac2260096d8d\": container with ID starting with 5319d507d8d401349e50592fc4e119fac3cf59c798a1776603b2ac2260096d8d not found: ID does not exist" Sep 13 20:23:24.925333 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.925300 2557 scope.go:117] "RemoveContainer" containerID="d16d2fe4c51bb516a0446638b6d7e6c77f0ef0d462407c5a62fb932afb98e4ff" Sep 13 20:23:24.930827 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.930788 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rf847" podStartSLOduration=1.9307754990000001 podStartE2EDuration="1.930775499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:23:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:23:24.929856142 +0000 UTC m=+637.106292156" watchObservedRunningTime="2026-09-13 20:23:24.930775499 +0000 UTC m=+637.107211520" Sep 13 20:23:24.932085 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.932066 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7f8h\" (UniqueName: \"kubernetes.io/projected/a979679d-9a45-496a-a13a-e31d54844034-kube-api-access-d7f8h\") pod \"kuadrant-operator-controller-manager-5769f99d9f-gqdcv\" (UID: \"a979679d-9a45-496a-a13a-e31d54844034\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gqdcv" Sep 13 20:23:24.932172 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.932153 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a979679d-9a45-496a-a13a-e31d54844034-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-gqdcv\" (UID: \"a979679d-9a45-496a-a13a-e31d54844034\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gqdcv" Sep 13 20:23:24.932504 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.932478 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a979679d-9a45-496a-a13a-e31d54844034-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-gqdcv\" (UID: \"a979679d-9a45-496a-a13a-e31d54844034\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gqdcv" Sep 13 20:23:24.933370 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.933357 2557 scope.go:117] "RemoveContainer" containerID="d16d2fe4c51bb516a0446638b6d7e6c77f0ef0d462407c5a62fb932afb98e4ff" Sep 13 20:23:24.933654 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:23:24.933637 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d16d2fe4c51bb516a0446638b6d7e6c77f0ef0d462407c5a62fb932afb98e4ff\": container with ID starting with d16d2fe4c51bb516a0446638b6d7e6c77f0ef0d462407c5a62fb932afb98e4ff not found: ID does not exist" containerID="d16d2fe4c51bb516a0446638b6d7e6c77f0ef0d462407c5a62fb932afb98e4ff" Sep 13 20:23:24.933699 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.933659 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d16d2fe4c51bb516a0446638b6d7e6c77f0ef0d462407c5a62fb932afb98e4ff"} err="failed to get container status \"d16d2fe4c51bb516a0446638b6d7e6c77f0ef0d462407c5a62fb932afb98e4ff\": rpc error: code = NotFound desc = could not find container \"d16d2fe4c51bb516a0446638b6d7e6c77f0ef0d462407c5a62fb932afb98e4ff\": container with ID starting with d16d2fe4c51bb516a0446638b6d7e6c77f0ef0d462407c5a62fb932afb98e4ff not found: ID does not exist" Sep 13 20:23:24.944664 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.944643 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7f8h\" (UniqueName: \"kubernetes.io/projected/a979679d-9a45-496a-a13a-e31d54844034-kube-api-access-d7f8h\") pod \"kuadrant-operator-controller-manager-5769f99d9f-gqdcv\" (UID: \"a979679d-9a45-496a-a13a-e31d54844034\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gqdcv" Sep 13 20:23:24.950830 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:24.950789 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vl6vt" podStartSLOduration=1.9507759679999999 podStartE2EDuration="1.950775968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:23:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:23:24.948201874 +0000 UTC m=+637.124637896" watchObservedRunningTime="2026-09-13 20:23:24.950775968 +0000 UTC m=+637.127211992" Sep 13 20:23:25.023302 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:25.023270 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gqdcv" Sep 13 20:23:25.303482 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:25.303456 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gqdcv"] Sep 13 20:23:25.305350 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:23:25.305323 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda979679d_9a45_496a_a13a_e31d54844034.slice/crio-5c45021e55ac5879c924b4c2a29f1b269f1e76f67853db77965bd5ea37f64ad1 WatchSource:0}: Error finding container 5c45021e55ac5879c924b4c2a29f1b269f1e76f67853db77965bd5ea37f64ad1: Status 404 returned error can't find the container with id 5c45021e55ac5879c924b4c2a29f1b269f1e76f67853db77965bd5ea37f64ad1 Sep 13 20:23:25.926429 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:25.926384 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gqdcv" event={"ID":"a979679d-9a45-496a-a13a-e31d54844034","Type":"ContainerStarted","Data":"82966b7aa5bcd2b784829454bf2441e3eb0b7ad330d0d3a224ec2bb02f2212d7"} Sep 13 20:23:25.926875 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:25.926484 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gqdcv" Sep 13 20:23:25.926875 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:25.926530 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gqdcv" event={"ID":"a979679d-9a45-496a-a13a-e31d54844034","Type":"ContainerStarted","Data":"5c45021e55ac5879c924b4c2a29f1b269f1e76f67853db77965bd5ea37f64ad1"} Sep 13 20:23:25.949696 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:25.949646 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gqdcv" podStartSLOduration=1.949631881 podStartE2EDuration="1.949631881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:23:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:23:25.947447722 +0000 UTC m=+638.123883743" watchObservedRunningTime="2026-09-13 20:23:25.949631881 +0000 UTC m=+638.126067903" Sep 13 20:23:30.977379 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:30.977337 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f897b78bf-cgxtr" Sep 13 20:23:30.977825 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:30.977406 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f897b78bf-cgxtr" Sep 13 20:23:30.982448 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:30.982420 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f897b78bf-cgxtr" Sep 13 20:23:31.968072 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:31.968039 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f897b78bf-cgxtr" Sep 13 20:23:35.930482 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:35.930451 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rf847" Sep 13 20:23:35.930885 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:35.930537 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vl6vt" Sep 13 20:23:36.937219 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:36.937190 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gqdcv" Sep 13 20:23:36.989002 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:36.988966 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vl6vt"] Sep 13 20:23:36.989265 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:36.989196 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vl6vt" podUID="50fa69f0-09aa-4302-a902-65821965aa38" containerName="manager" containerID="cri-o://0876712d15fa9cbab541f8fd03f867564dc423f088fd21717229806a600b9289" gracePeriod=10 Sep 13 20:23:37.236856 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:37.236831 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vl6vt" Sep 13 20:23:37.344458 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:37.344421 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s75zk\" (UniqueName: \"kubernetes.io/projected/50fa69f0-09aa-4302-a902-65821965aa38-kube-api-access-s75zk\") pod \"50fa69f0-09aa-4302-a902-65821965aa38\" (UID: \"50fa69f0-09aa-4302-a902-65821965aa38\") " Sep 13 20:23:37.344654 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:37.344549 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50fa69f0-09aa-4302-a902-65821965aa38-extensions-socket-volume\") pod \"50fa69f0-09aa-4302-a902-65821965aa38\" (UID: \"50fa69f0-09aa-4302-a902-65821965aa38\") " Sep 13 20:23:37.344963 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:37.344940 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50fa69f0-09aa-4302-a902-65821965aa38-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "50fa69f0-09aa-4302-a902-65821965aa38" (UID: "50fa69f0-09aa-4302-a902-65821965aa38"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 13 20:23:37.346657 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:37.346633 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50fa69f0-09aa-4302-a902-65821965aa38-kube-api-access-s75zk" (OuterVolumeSpecName: "kube-api-access-s75zk") pod "50fa69f0-09aa-4302-a902-65821965aa38" (UID: "50fa69f0-09aa-4302-a902-65821965aa38"). InnerVolumeSpecName "kube-api-access-s75zk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 20:23:37.445379 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:37.445350 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s75zk\" (UniqueName: \"kubernetes.io/projected/50fa69f0-09aa-4302-a902-65821965aa38-kube-api-access-s75zk\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:23:37.445379 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:37.445375 2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50fa69f0-09aa-4302-a902-65821965aa38-extensions-socket-volume\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:23:38.000053 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:38.000020 2557 generic.go:356] "Generic (PLEG): container finished" podID="50fa69f0-09aa-4302-a902-65821965aa38" containerID="0876712d15fa9cbab541f8fd03f867564dc423f088fd21717229806a600b9289" exitCode=0 Sep 13 20:23:38.000520 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:38.000059 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vl6vt" Sep 13 20:23:38.000520 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:38.000104 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vl6vt" event={"ID":"50fa69f0-09aa-4302-a902-65821965aa38","Type":"ContainerDied","Data":"0876712d15fa9cbab541f8fd03f867564dc423f088fd21717229806a600b9289"} Sep 13 20:23:38.000520 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:38.000151 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vl6vt" event={"ID":"50fa69f0-09aa-4302-a902-65821965aa38","Type":"ContainerDied","Data":"3e42377d56840f1e347cf5ffe4081f173a5d4b6b7250cc6c20107fe83d897654"} Sep 13 20:23:38.000520 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:38.000172 2557 scope.go:117] "RemoveContainer" containerID="0876712d15fa9cbab541f8fd03f867564dc423f088fd21717229806a600b9289" Sep 13 20:23:38.013134 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:38.013113 2557 scope.go:117] "RemoveContainer" containerID="0876712d15fa9cbab541f8fd03f867564dc423f088fd21717229806a600b9289" Sep 13 20:23:38.013412 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:23:38.013390 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0876712d15fa9cbab541f8fd03f867564dc423f088fd21717229806a600b9289\": container with ID starting with 0876712d15fa9cbab541f8fd03f867564dc423f088fd21717229806a600b9289 not found: ID does not exist" containerID="0876712d15fa9cbab541f8fd03f867564dc423f088fd21717229806a600b9289" Sep 13 20:23:38.013485 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:38.013424 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0876712d15fa9cbab541f8fd03f867564dc423f088fd21717229806a600b9289"} err="failed to get container status \"0876712d15fa9cbab541f8fd03f867564dc423f088fd21717229806a600b9289\": rpc error: code = NotFound desc = could not find container \"0876712d15fa9cbab541f8fd03f867564dc423f088fd21717229806a600b9289\": container with ID starting with 0876712d15fa9cbab541f8fd03f867564dc423f088fd21717229806a600b9289 not found: ID does not exist" Sep 13 20:23:38.029769 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:38.029737 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vl6vt"] Sep 13 20:23:38.033923 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:38.033899 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vl6vt"] Sep 13 20:23:38.313066 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:38.313036 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50fa69f0-09aa-4302-a902-65821965aa38" path="/var/lib/kubelet/pods/50fa69f0-09aa-4302-a902-65821965aa38/volumes" Sep 13 20:23:55.096245 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:55.096197 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6jnmr"] Sep 13 20:23:55.096872 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:55.096848 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50fa69f0-09aa-4302-a902-65821965aa38" containerName="manager" Sep 13 20:23:55.096949 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:55.096874 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="50fa69f0-09aa-4302-a902-65821965aa38" containerName="manager" Sep 13 20:23:55.097004 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:55.096995 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="50fa69f0-09aa-4302-a902-65821965aa38" containerName="manager" Sep 13 20:23:55.101470 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:55.101450 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-6jnmr" Sep 13 20:23:55.104882 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:55.104846 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-lszmf\"" Sep 13 20:23:55.105047 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:55.104845 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Sep 13 20:23:55.106739 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:55.106713 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6jnmr"] Sep 13 20:23:55.186681 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:55.186604 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phsgs\" (UniqueName: \"kubernetes.io/projected/7d29457b-676d-4b1f-85e8-7b9c4acc15ab-kube-api-access-phsgs\") pod \"limitador-limitador-8588749977-6jnmr\" (UID: \"7d29457b-676d-4b1f-85e8-7b9c4acc15ab\") " pod="kuadrant-system/limitador-limitador-8588749977-6jnmr" Sep 13 20:23:55.187007 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:55.186708 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7d29457b-676d-4b1f-85e8-7b9c4acc15ab-config-file\") pod \"limitador-limitador-8588749977-6jnmr\" (UID: \"7d29457b-676d-4b1f-85e8-7b9c4acc15ab\") " pod="kuadrant-system/limitador-limitador-8588749977-6jnmr" Sep 13 20:23:55.198145 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:55.198118 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6jnmr"] Sep 13 20:23:55.287584 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:55.287550 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7d29457b-676d-4b1f-85e8-7b9c4acc15ab-config-file\") pod \"limitador-limitador-8588749977-6jnmr\" (UID: \"7d29457b-676d-4b1f-85e8-7b9c4acc15ab\") " pod="kuadrant-system/limitador-limitador-8588749977-6jnmr" Sep 13 20:23:55.287740 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:55.287621 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phsgs\" (UniqueName: \"kubernetes.io/projected/7d29457b-676d-4b1f-85e8-7b9c4acc15ab-kube-api-access-phsgs\") pod \"limitador-limitador-8588749977-6jnmr\" (UID: \"7d29457b-676d-4b1f-85e8-7b9c4acc15ab\") " pod="kuadrant-system/limitador-limitador-8588749977-6jnmr" Sep 13 20:23:55.288263 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:55.288234 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7d29457b-676d-4b1f-85e8-7b9c4acc15ab-config-file\") pod \"limitador-limitador-8588749977-6jnmr\" (UID: \"7d29457b-676d-4b1f-85e8-7b9c4acc15ab\") " pod="kuadrant-system/limitador-limitador-8588749977-6jnmr" Sep 13 20:23:55.295488 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:55.295462 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phsgs\" (UniqueName: \"kubernetes.io/projected/7d29457b-676d-4b1f-85e8-7b9c4acc15ab-kube-api-access-phsgs\") pod \"limitador-limitador-8588749977-6jnmr\" (UID: \"7d29457b-676d-4b1f-85e8-7b9c4acc15ab\") " pod="kuadrant-system/limitador-limitador-8588749977-6jnmr" Sep 13 20:23:55.413572 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:55.413468 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-6jnmr" Sep 13 20:23:55.683633 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:55.683606 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6jnmr"] Sep 13 20:23:55.685025 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:23:55.684995 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d29457b_676d_4b1f_85e8_7b9c4acc15ab.slice/crio-f3ca1a8246d8b4db683ed471e29a1b8934c189edcc555f398bedd0ce734541a5 WatchSource:0}: Error finding container f3ca1a8246d8b4db683ed471e29a1b8934c189edcc555f398bedd0ce734541a5: Status 404 returned error can't find the container with id f3ca1a8246d8b4db683ed471e29a1b8934c189edcc555f398bedd0ce734541a5 Sep 13 20:23:55.869764 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:55.869734 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-nhfvg"] Sep 13 20:23:55.874363 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:55.874340 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-nhfvg" Sep 13 20:23:55.876469 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:55.876449 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-9d6ll\"" Sep 13 20:23:55.880379 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:55.880354 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-nhfvg"] Sep 13 20:23:55.993395 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:55.993313 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cfjm\" (UniqueName: \"kubernetes.io/projected/db8e1ed7-3bd8-4b87-8bed-82a15bc679b6-kube-api-access-8cfjm\") pod \"authorino-7bcfd8f9cd-nhfvg\" (UID: \"db8e1ed7-3bd8-4b87-8bed-82a15bc679b6\") " pod="kuadrant-system/authorino-7bcfd8f9cd-nhfvg" Sep 13 20:23:56.094490 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:56.094457 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cfjm\" (UniqueName: \"kubernetes.io/projected/db8e1ed7-3bd8-4b87-8bed-82a15bc679b6-kube-api-access-8cfjm\") pod \"authorino-7bcfd8f9cd-nhfvg\" (UID: \"db8e1ed7-3bd8-4b87-8bed-82a15bc679b6\") " pod="kuadrant-system/authorino-7bcfd8f9cd-nhfvg" Sep 13 20:23:56.100319 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:56.100288 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-6jnmr" event={"ID":"7d29457b-676d-4b1f-85e8-7b9c4acc15ab","Type":"ContainerStarted","Data":"f3ca1a8246d8b4db683ed471e29a1b8934c189edcc555f398bedd0ce734541a5"} Sep 13 20:23:56.101823 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:56.101800 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cfjm\" (UniqueName: \"kubernetes.io/projected/db8e1ed7-3bd8-4b87-8bed-82a15bc679b6-kube-api-access-8cfjm\") pod \"authorino-7bcfd8f9cd-nhfvg\" (UID: \"db8e1ed7-3bd8-4b87-8bed-82a15bc679b6\") " pod="kuadrant-system/authorino-7bcfd8f9cd-nhfvg" Sep 13 20:23:56.185542 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:56.185477 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-nhfvg" Sep 13 20:23:56.455907 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:56.455879 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-nhfvg"] Sep 13 20:23:56.457211 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:23:56.457175 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb8e1ed7_3bd8_4b87_8bed_82a15bc679b6.slice/crio-02559a43c7d4295a34a773ebd72df73a4b4b4686684af68a28b7f4bcb097f317 WatchSource:0}: Error finding container 02559a43c7d4295a34a773ebd72df73a4b4b4686684af68a28b7f4bcb097f317: Status 404 returned error can't find the container with id 02559a43c7d4295a34a773ebd72df73a4b4b4686684af68a28b7f4bcb097f317 Sep 13 20:23:57.107971 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:57.107935 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-nhfvg" event={"ID":"db8e1ed7-3bd8-4b87-8bed-82a15bc679b6","Type":"ContainerStarted","Data":"02559a43c7d4295a34a773ebd72df73a4b4b4686684af68a28b7f4bcb097f317"} Sep 13 20:23:58.121757 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:58.121723 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-6jnmr" event={"ID":"7d29457b-676d-4b1f-85e8-7b9c4acc15ab","Type":"ContainerStarted","Data":"f2cb0a0cb3ae6c6a95969e95410f4ed7e24f6cb7ea46013ebe204d9b6233fb5c"} Sep 13 20:23:58.122212 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:58.121898 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-6jnmr" Sep 13 20:23:58.138059 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:58.138006 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-6jnmr" podStartSLOduration=1.5106527220000001 podStartE2EDuration="3.137989825s" totalImagesPullingTime="1.627337103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:23:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:23:58.135882553 +0000 UTC m=+670.312318577" watchObservedRunningTime="2026-09-13 20:23:58.137989825 +0000 UTC m=+670.314425890" Sep 13 20:23:59.129120 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:59.129085 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-nhfvg" event={"ID":"db8e1ed7-3bd8-4b87-8bed-82a15bc679b6","Type":"ContainerStarted","Data":"76073c60283f9a3e6c702feb876b08a0bd4a8fa838f2bc88b6e4beafc2403dfa"} Sep 13 20:23:59.143859 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:59.143812 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-nhfvg" podStartSLOduration=2.389158816 podStartE2EDuration="4.14379578s" totalImagesPullingTime="1.754636964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:23:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:23:59.143274281 +0000 UTC m=+671.319710305" watchObservedRunningTime="2026-09-13 20:23:59.14379578 +0000 UTC m=+671.320231802" Sep 13 20:23:59.188544 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:23:59.188509 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-nhfvg"] Sep 13 20:24:01.139970 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:01.139920 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-nhfvg" podUID="db8e1ed7-3bd8-4b87-8bed-82a15bc679b6" containerName="authorino" containerID="cri-o://76073c60283f9a3e6c702feb876b08a0bd4a8fa838f2bc88b6e4beafc2403dfa" gracePeriod=30 Sep 13 20:24:01.390799 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:01.390735 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-nhfvg" Sep 13 20:24:01.450883 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:01.450848 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8cfjm\" (UniqueName: \"kubernetes.io/projected/db8e1ed7-3bd8-4b87-8bed-82a15bc679b6-kube-api-access-8cfjm\") pod \"db8e1ed7-3bd8-4b87-8bed-82a15bc679b6\" (UID: \"db8e1ed7-3bd8-4b87-8bed-82a15bc679b6\") " Sep 13 20:24:01.453045 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:01.453015 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db8e1ed7-3bd8-4b87-8bed-82a15bc679b6-kube-api-access-8cfjm" (OuterVolumeSpecName: "kube-api-access-8cfjm") pod "db8e1ed7-3bd8-4b87-8bed-82a15bc679b6" (UID: "db8e1ed7-3bd8-4b87-8bed-82a15bc679b6"). InnerVolumeSpecName "kube-api-access-8cfjm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 20:24:01.552419 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:01.552380 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cfjm\" (UniqueName: \"kubernetes.io/projected/db8e1ed7-3bd8-4b87-8bed-82a15bc679b6-kube-api-access-8cfjm\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:24:02.146871 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:02.146831 2557 generic.go:356] "Generic (PLEG): container finished" podID="db8e1ed7-3bd8-4b87-8bed-82a15bc679b6" containerID="76073c60283f9a3e6c702feb876b08a0bd4a8fa838f2bc88b6e4beafc2403dfa" exitCode=0 Sep 13 20:24:02.146871 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:02.146876 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-nhfvg" Sep 13 20:24:02.147345 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:02.146919 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-nhfvg" event={"ID":"db8e1ed7-3bd8-4b87-8bed-82a15bc679b6","Type":"ContainerDied","Data":"76073c60283f9a3e6c702feb876b08a0bd4a8fa838f2bc88b6e4beafc2403dfa"} Sep 13 20:24:02.147345 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:02.146972 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-nhfvg" event={"ID":"db8e1ed7-3bd8-4b87-8bed-82a15bc679b6","Type":"ContainerDied","Data":"02559a43c7d4295a34a773ebd72df73a4b4b4686684af68a28b7f4bcb097f317"} Sep 13 20:24:02.147345 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:02.146992 2557 scope.go:117] "RemoveContainer" containerID="76073c60283f9a3e6c702feb876b08a0bd4a8fa838f2bc88b6e4beafc2403dfa" Sep 13 20:24:02.159765 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:02.159741 2557 scope.go:117] "RemoveContainer" containerID="76073c60283f9a3e6c702feb876b08a0bd4a8fa838f2bc88b6e4beafc2403dfa" Sep 13 20:24:02.160041 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:24:02.160020 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76073c60283f9a3e6c702feb876b08a0bd4a8fa838f2bc88b6e4beafc2403dfa\": container with ID starting with 76073c60283f9a3e6c702feb876b08a0bd4a8fa838f2bc88b6e4beafc2403dfa not found: ID does not exist" containerID="76073c60283f9a3e6c702feb876b08a0bd4a8fa838f2bc88b6e4beafc2403dfa" Sep 13 20:24:02.160124 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:02.160048 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76073c60283f9a3e6c702feb876b08a0bd4a8fa838f2bc88b6e4beafc2403dfa"} err="failed to get container status \"76073c60283f9a3e6c702feb876b08a0bd4a8fa838f2bc88b6e4beafc2403dfa\": rpc error: code = NotFound desc = could not find container \"76073c60283f9a3e6c702feb876b08a0bd4a8fa838f2bc88b6e4beafc2403dfa\": container with ID starting with 76073c60283f9a3e6c702feb876b08a0bd4a8fa838f2bc88b6e4beafc2403dfa not found: ID does not exist" Sep 13 20:24:02.171086 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:02.171062 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-nhfvg"] Sep 13 20:24:02.173993 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:02.173973 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-nhfvg"] Sep 13 20:24:02.311922 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:02.311886 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db8e1ed7-3bd8-4b87-8bed-82a15bc679b6" path="/var/lib/kubelet/pods/db8e1ed7-3bd8-4b87-8bed-82a15bc679b6/volumes" Sep 13 20:24:09.130861 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:09.130829 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-6jnmr" Sep 13 20:24:10.182758 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:10.182723 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6jnmr"] Sep 13 20:24:10.183364 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:10.182920 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-6jnmr" podUID="7d29457b-676d-4b1f-85e8-7b9c4acc15ab" containerName="limitador" containerID="cri-o://f2cb0a0cb3ae6c6a95969e95410f4ed7e24f6cb7ea46013ebe204d9b6233fb5c" gracePeriod=30 Sep 13 20:24:10.728652 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:10.728624 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-6jnmr" Sep 13 20:24:10.831219 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:10.831183 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7d29457b-676d-4b1f-85e8-7b9c4acc15ab-config-file\") pod \"7d29457b-676d-4b1f-85e8-7b9c4acc15ab\" (UID: \"7d29457b-676d-4b1f-85e8-7b9c4acc15ab\") " Sep 13 20:24:10.831412 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:10.831283 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phsgs\" (UniqueName: \"kubernetes.io/projected/7d29457b-676d-4b1f-85e8-7b9c4acc15ab-kube-api-access-phsgs\") pod \"7d29457b-676d-4b1f-85e8-7b9c4acc15ab\" (UID: \"7d29457b-676d-4b1f-85e8-7b9c4acc15ab\") " Sep 13 20:24:10.831542 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:10.831523 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d29457b-676d-4b1f-85e8-7b9c4acc15ab-config-file" (OuterVolumeSpecName: "config-file") pod "7d29457b-676d-4b1f-85e8-7b9c4acc15ab" (UID: "7d29457b-676d-4b1f-85e8-7b9c4acc15ab"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 13 20:24:10.833273 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:10.833243 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d29457b-676d-4b1f-85e8-7b9c4acc15ab-kube-api-access-phsgs" (OuterVolumeSpecName: "kube-api-access-phsgs") pod "7d29457b-676d-4b1f-85e8-7b9c4acc15ab" (UID: "7d29457b-676d-4b1f-85e8-7b9c4acc15ab"). InnerVolumeSpecName "kube-api-access-phsgs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 20:24:10.932708 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:10.932676 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phsgs\" (UniqueName: \"kubernetes.io/projected/7d29457b-676d-4b1f-85e8-7b9c4acc15ab-kube-api-access-phsgs\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:24:10.932708 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:10.932703 2557 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7d29457b-676d-4b1f-85e8-7b9c4acc15ab-config-file\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:24:11.197985 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:11.197894 2557 generic.go:356] "Generic (PLEG): container finished" podID="7d29457b-676d-4b1f-85e8-7b9c4acc15ab" containerID="f2cb0a0cb3ae6c6a95969e95410f4ed7e24f6cb7ea46013ebe204d9b6233fb5c" exitCode=0 Sep 13 20:24:11.197985 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:11.197938 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-6jnmr" Sep 13 20:24:11.197985 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:11.197977 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-6jnmr" event={"ID":"7d29457b-676d-4b1f-85e8-7b9c4acc15ab","Type":"ContainerDied","Data":"f2cb0a0cb3ae6c6a95969e95410f4ed7e24f6cb7ea46013ebe204d9b6233fb5c"} Sep 13 20:24:11.198512 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:11.198027 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-6jnmr" event={"ID":"7d29457b-676d-4b1f-85e8-7b9c4acc15ab","Type":"ContainerDied","Data":"f3ca1a8246d8b4db683ed471e29a1b8934c189edcc555f398bedd0ce734541a5"} Sep 13 20:24:11.198512 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:11.198050 2557 scope.go:117] "RemoveContainer" containerID="f2cb0a0cb3ae6c6a95969e95410f4ed7e24f6cb7ea46013ebe204d9b6233fb5c" Sep 13 20:24:11.207683 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:11.207664 2557 scope.go:117] "RemoveContainer" containerID="f2cb0a0cb3ae6c6a95969e95410f4ed7e24f6cb7ea46013ebe204d9b6233fb5c" Sep 13 20:24:11.207972 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:24:11.207938 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2cb0a0cb3ae6c6a95969e95410f4ed7e24f6cb7ea46013ebe204d9b6233fb5c\": container with ID starting with f2cb0a0cb3ae6c6a95969e95410f4ed7e24f6cb7ea46013ebe204d9b6233fb5c not found: ID does not exist" containerID="f2cb0a0cb3ae6c6a95969e95410f4ed7e24f6cb7ea46013ebe204d9b6233fb5c" Sep 13 20:24:11.208058 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:11.207968 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2cb0a0cb3ae6c6a95969e95410f4ed7e24f6cb7ea46013ebe204d9b6233fb5c"} err="failed to get container status \"f2cb0a0cb3ae6c6a95969e95410f4ed7e24f6cb7ea46013ebe204d9b6233fb5c\": rpc error: code = NotFound desc = could not find container \"f2cb0a0cb3ae6c6a95969e95410f4ed7e24f6cb7ea46013ebe204d9b6233fb5c\": container with ID starting with f2cb0a0cb3ae6c6a95969e95410f4ed7e24f6cb7ea46013ebe204d9b6233fb5c not found: ID does not exist" Sep 13 20:24:11.217748 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:11.217726 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6jnmr"] Sep 13 20:24:11.221792 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:11.221766 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6jnmr"] Sep 13 20:24:12.312291 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:12.312251 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d29457b-676d-4b1f-85e8-7b9c4acc15ab" path="/var/lib/kubelet/pods/7d29457b-676d-4b1f-85e8-7b9c4acc15ab/volumes" Sep 13 20:24:16.628012 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:16.627982 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-tnmrd"] Sep 13 20:24:16.628383 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:16.628370 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db8e1ed7-3bd8-4b87-8bed-82a15bc679b6" containerName="authorino" Sep 13 20:24:16.628424 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:16.628385 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="db8e1ed7-3bd8-4b87-8bed-82a15bc679b6" containerName="authorino" Sep 13 20:24:16.628424 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:16.628402 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d29457b-676d-4b1f-85e8-7b9c4acc15ab" containerName="limitador" Sep 13 20:24:16.628424 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:16.628409 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d29457b-676d-4b1f-85e8-7b9c4acc15ab" containerName="limitador" Sep 13 20:24:16.628549 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:16.628465 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d29457b-676d-4b1f-85e8-7b9c4acc15ab" containerName="limitador" Sep 13 20:24:16.628549 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:16.628476 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="db8e1ed7-3bd8-4b87-8bed-82a15bc679b6" containerName="authorino" Sep 13 20:24:16.631578 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:16.631549 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tnmrd" Sep 13 20:24:16.633864 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:16.633841 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Sep 13 20:24:16.634007 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:16.633943 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 13 20:24:16.634007 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:16.633992 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-lr92m\"" Sep 13 20:24:16.634767 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:16.634748 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 13 20:24:16.641356 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:16.641334 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-tnmrd"] Sep 13 20:24:16.681197 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:16.681169 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/095a6d9d-ffa3-4537-af24-beb1793c0d43-data\") pod \"postgres-69fb4594cf-tnmrd\" (UID: \"095a6d9d-ffa3-4537-af24-beb1793c0d43\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tnmrd" Sep 13 20:24:16.681306 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:16.681271 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x42b8\" (UniqueName: \"kubernetes.io/projected/095a6d9d-ffa3-4537-af24-beb1793c0d43-kube-api-access-x42b8\") pod \"postgres-69fb4594cf-tnmrd\" (UID: \"095a6d9d-ffa3-4537-af24-beb1793c0d43\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tnmrd" Sep 13 20:24:16.782187 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:16.782141 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x42b8\" (UniqueName: \"kubernetes.io/projected/095a6d9d-ffa3-4537-af24-beb1793c0d43-kube-api-access-x42b8\") pod \"postgres-69fb4594cf-tnmrd\" (UID: \"095a6d9d-ffa3-4537-af24-beb1793c0d43\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tnmrd" Sep 13 20:24:16.782187 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:16.782195 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/095a6d9d-ffa3-4537-af24-beb1793c0d43-data\") pod \"postgres-69fb4594cf-tnmrd\" (UID: \"095a6d9d-ffa3-4537-af24-beb1793c0d43\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tnmrd" Sep 13 20:24:16.782559 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:16.782543 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/095a6d9d-ffa3-4537-af24-beb1793c0d43-data\") pod \"postgres-69fb4594cf-tnmrd\" (UID: \"095a6d9d-ffa3-4537-af24-beb1793c0d43\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tnmrd" Sep 13 20:24:16.790401 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:16.790372 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x42b8\" (UniqueName: \"kubernetes.io/projected/095a6d9d-ffa3-4537-af24-beb1793c0d43-kube-api-access-x42b8\") pod \"postgres-69fb4594cf-tnmrd\" (UID: \"095a6d9d-ffa3-4537-af24-beb1793c0d43\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tnmrd" Sep 13 20:24:16.942626 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:16.942551 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tnmrd" Sep 13 20:24:17.211416 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:17.211388 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-tnmrd"] Sep 13 20:24:17.212896 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:24:17.212867 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod095a6d9d_ffa3_4537_af24_beb1793c0d43.slice/crio-7c8277ee7fee6856aa70b881f76a0e3d0e86bdf890fe686cc26b22782d6f5d0c WatchSource:0}: Error finding container 7c8277ee7fee6856aa70b881f76a0e3d0e86bdf890fe686cc26b22782d6f5d0c: Status 404 returned error can't find the container with id 7c8277ee7fee6856aa70b881f76a0e3d0e86bdf890fe686cc26b22782d6f5d0c Sep 13 20:24:17.239482 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:17.239455 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tnmrd" event={"ID":"095a6d9d-ffa3-4537-af24-beb1793c0d43","Type":"ContainerStarted","Data":"7c8277ee7fee6856aa70b881f76a0e3d0e86bdf890fe686cc26b22782d6f5d0c"} Sep 13 20:24:22.274726 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:22.274688 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tnmrd" event={"ID":"095a6d9d-ffa3-4537-af24-beb1793c0d43","Type":"ContainerStarted","Data":"0db9836364fb6338dfa81fe843cda7236dfb538ed7f74d58668a3ba631f64922"} Sep 13 20:24:22.275193 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:22.274763 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tnmrd" Sep 13 20:24:22.289514 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:22.289454 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tnmrd" podStartSLOduration=1.44810549 podStartE2EDuration="6.2894387s" totalImagesPullingTime="4.84133321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:24:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:24:22.287732925 +0000 UTC m=+694.464168946" watchObservedRunningTime="2026-09-13 20:24:22.2894387 +0000 UTC m=+694.465874757" Sep 13 20:24:28.312369 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:28.312343 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tnmrd" Sep 13 20:24:28.822876 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:28.822839 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-2n2tf"] Sep 13 20:24:28.829299 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:28.829276 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-2n2tf" Sep 13 20:24:28.831545 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:28.831519 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-9d6ll\"" Sep 13 20:24:28.833000 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:28.832973 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-2n2tf"] Sep 13 20:24:28.876448 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:28.876415 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-2n2tf"] Sep 13 20:24:28.876690 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:24:28.876670 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-wlfnb], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-2n2tf" podUID="5b65a8dc-d677-42e5-a633-7082c8c10896" Sep 13 20:24:28.893374 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:28.893344 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlfnb\" (UniqueName: \"kubernetes.io/projected/5b65a8dc-d677-42e5-a633-7082c8c10896-kube-api-access-wlfnb\") pod \"authorino-557f876f69-2n2tf\" (UID: \"5b65a8dc-d677-42e5-a633-7082c8c10896\") " pod="kuadrant-system/authorino-557f876f69-2n2tf" Sep 13 20:24:28.896836 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:28.896815 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-pw27v"] Sep 13 20:24:28.901466 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:28.901449 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-pw27v" Sep 13 20:24:28.903616 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:28.903596 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Sep 13 20:24:28.909711 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:28.909688 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-pw27v"] Sep 13 20:24:28.994069 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:28.994040 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlfnb\" (UniqueName: \"kubernetes.io/projected/5b65a8dc-d677-42e5-a633-7082c8c10896-kube-api-access-wlfnb\") pod \"authorino-557f876f69-2n2tf\" (UID: \"5b65a8dc-d677-42e5-a633-7082c8c10896\") " pod="kuadrant-system/authorino-557f876f69-2n2tf" Sep 13 20:24:28.994234 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:28.994110 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ca130692-5007-49ad-9f96-a62d13b02047-tls-cert\") pod \"authorino-6d4dfc649f-pw27v\" (UID: \"ca130692-5007-49ad-9f96-a62d13b02047\") " pod="kuadrant-system/authorino-6d4dfc649f-pw27v" Sep 13 20:24:28.994234 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:28.994146 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrch6\" (UniqueName: \"kubernetes.io/projected/ca130692-5007-49ad-9f96-a62d13b02047-kube-api-access-wrch6\") pod \"authorino-6d4dfc649f-pw27v\" (UID: \"ca130692-5007-49ad-9f96-a62d13b02047\") " pod="kuadrant-system/authorino-6d4dfc649f-pw27v" Sep 13 20:24:29.001860 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:29.001834 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlfnb\" (UniqueName: \"kubernetes.io/projected/5b65a8dc-d677-42e5-a633-7082c8c10896-kube-api-access-wlfnb\") pod \"authorino-557f876f69-2n2tf\" (UID: \"5b65a8dc-d677-42e5-a633-7082c8c10896\") " pod="kuadrant-system/authorino-557f876f69-2n2tf" Sep 13 20:24:29.057873 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:29.057844 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-pw27v"] Sep 13 20:24:29.058112 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:24:29.058093 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-wrch6 tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-pw27v" podUID="ca130692-5007-49ad-9f96-a62d13b02047" Sep 13 20:24:29.095399 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:29.095317 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrch6\" (UniqueName: \"kubernetes.io/projected/ca130692-5007-49ad-9f96-a62d13b02047-kube-api-access-wrch6\") pod \"authorino-6d4dfc649f-pw27v\" (UID: \"ca130692-5007-49ad-9f96-a62d13b02047\") " pod="kuadrant-system/authorino-6d4dfc649f-pw27v" Sep 13 20:24:29.095584 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:29.095416 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ca130692-5007-49ad-9f96-a62d13b02047-tls-cert\") pod \"authorino-6d4dfc649f-pw27v\" (UID: \"ca130692-5007-49ad-9f96-a62d13b02047\") " pod="kuadrant-system/authorino-6d4dfc649f-pw27v" Sep 13 20:24:29.098091 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:29.098065 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ca130692-5007-49ad-9f96-a62d13b02047-tls-cert\") pod \"authorino-6d4dfc649f-pw27v\" (UID: \"ca130692-5007-49ad-9f96-a62d13b02047\") " pod="kuadrant-system/authorino-6d4dfc649f-pw27v" Sep 13 20:24:29.102307 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:29.102279 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrch6\" (UniqueName: \"kubernetes.io/projected/ca130692-5007-49ad-9f96-a62d13b02047-kube-api-access-wrch6\") pod \"authorino-6d4dfc649f-pw27v\" (UID: \"ca130692-5007-49ad-9f96-a62d13b02047\") " pod="kuadrant-system/authorino-6d4dfc649f-pw27v" Sep 13 20:24:29.314533 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:29.314475 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-pw27v" Sep 13 20:24:29.314915 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:29.314479 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-2n2tf" Sep 13 20:24:29.323198 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:29.323176 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-pw27v" Sep 13 20:24:29.326216 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:29.326195 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-2n2tf" Sep 13 20:24:29.398848 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:29.398759 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wlfnb\" (UniqueName: \"kubernetes.io/projected/5b65a8dc-d677-42e5-a633-7082c8c10896-kube-api-access-wlfnb\") pod \"5b65a8dc-d677-42e5-a633-7082c8c10896\" (UID: \"5b65a8dc-d677-42e5-a633-7082c8c10896\") " Sep 13 20:24:29.398848 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:29.398800 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrch6\" (UniqueName: \"kubernetes.io/projected/ca130692-5007-49ad-9f96-a62d13b02047-kube-api-access-wrch6\") pod \"ca130692-5007-49ad-9f96-a62d13b02047\" (UID: \"ca130692-5007-49ad-9f96-a62d13b02047\") " Sep 13 20:24:29.399081 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:29.398861 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ca130692-5007-49ad-9f96-a62d13b02047-tls-cert\") pod \"ca130692-5007-49ad-9f96-a62d13b02047\" (UID: \"ca130692-5007-49ad-9f96-a62d13b02047\") " Sep 13 20:24:29.401035 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:29.401008 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca130692-5007-49ad-9f96-a62d13b02047-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "ca130692-5007-49ad-9f96-a62d13b02047" (UID: "ca130692-5007-49ad-9f96-a62d13b02047"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 20:24:29.401387 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:29.401357 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca130692-5007-49ad-9f96-a62d13b02047-kube-api-access-wrch6" (OuterVolumeSpecName: "kube-api-access-wrch6") pod "ca130692-5007-49ad-9f96-a62d13b02047" (UID: "ca130692-5007-49ad-9f96-a62d13b02047"). InnerVolumeSpecName "kube-api-access-wrch6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 20:24:29.401387 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:29.401371 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b65a8dc-d677-42e5-a633-7082c8c10896-kube-api-access-wlfnb" (OuterVolumeSpecName: "kube-api-access-wlfnb") pod "5b65a8dc-d677-42e5-a633-7082c8c10896" (UID: "5b65a8dc-d677-42e5-a633-7082c8c10896"). InnerVolumeSpecName "kube-api-access-wlfnb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 20:24:29.499563 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:29.499531 2557 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ca130692-5007-49ad-9f96-a62d13b02047-tls-cert\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:24:29.499563 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:29.499557 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlfnb\" (UniqueName: \"kubernetes.io/projected/5b65a8dc-d677-42e5-a633-7082c8c10896-kube-api-access-wlfnb\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:24:29.499563 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:29.499569 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrch6\" (UniqueName: \"kubernetes.io/projected/ca130692-5007-49ad-9f96-a62d13b02047-kube-api-access-wrch6\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:24:30.323190 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:30.323057 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-2n2tf" Sep 13 20:24:30.323720 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:30.323211 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-pw27v" Sep 13 20:24:30.353457 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:30.353424 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-pw27v"] Sep 13 20:24:30.358858 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:30.358825 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-pw27v"] Sep 13 20:24:30.379729 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:30.379692 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-2n2tf"] Sep 13 20:24:30.381749 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:30.381728 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-2n2tf"] Sep 13 20:24:32.311690 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:32.311662 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b65a8dc-d677-42e5-a633-7082c8c10896" path="/var/lib/kubelet/pods/5b65a8dc-d677-42e5-a633-7082c8c10896/volumes" Sep 13 20:24:32.312084 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:32.311918 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca130692-5007-49ad-9f96-a62d13b02047" path="/var/lib/kubelet/pods/ca130692-5007-49ad-9f96-a62d13b02047/volumes" Sep 13 20:24:34.848149 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:34.848116 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-98f5d4d4f-7wck2"] Sep 13 20:24:34.860356 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:34.860315 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-98f5d4d4f-7wck2" Sep 13 20:24:34.863048 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:34.863020 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 13 20:24:34.863606 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:34.863584 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 13 20:24:34.864153 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:34.864135 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 13 20:24:34.864605 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:34.864582 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-xkvj2\"" Sep 13 20:24:34.864699 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:34.864676 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-98f5d4d4f-7wck2"] Sep 13 20:24:34.943692 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:34.943657 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76c4v\" (UniqueName: \"kubernetes.io/projected/bf7650cc-88fd-4cba-bde8-c394e73944f0-kube-api-access-76c4v\") pod \"maas-controller-98f5d4d4f-7wck2\" (UID: \"bf7650cc-88fd-4cba-bde8-c394e73944f0\") " pod="opendatahub/maas-controller-98f5d4d4f-7wck2" Sep 13 20:24:34.943894 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:34.943698 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bf7650cc-88fd-4cba-bde8-c394e73944f0-webhook-cert\") pod \"maas-controller-98f5d4d4f-7wck2\" (UID: \"bf7650cc-88fd-4cba-bde8-c394e73944f0\") " pod="opendatahub/maas-controller-98f5d4d4f-7wck2" Sep 13 20:24:34.943894 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:34.943842 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf7650cc-88fd-4cba-bde8-c394e73944f0-metrics-certs\") pod \"maas-controller-98f5d4d4f-7wck2\" (UID: \"bf7650cc-88fd-4cba-bde8-c394e73944f0\") " pod="opendatahub/maas-controller-98f5d4d4f-7wck2" Sep 13 20:24:35.005685 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:35.005648 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7fbc5958cd-xww4d"] Sep 13 20:24:35.009445 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:35.009427 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fbc5958cd-xww4d" Sep 13 20:24:35.018100 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:35.018068 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fbc5958cd-xww4d"] Sep 13 20:24:35.044786 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:35.044753 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf7650cc-88fd-4cba-bde8-c394e73944f0-metrics-certs\") pod \"maas-controller-98f5d4d4f-7wck2\" (UID: \"bf7650cc-88fd-4cba-bde8-c394e73944f0\") " pod="opendatahub/maas-controller-98f5d4d4f-7wck2" Sep 13 20:24:35.044915 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:35.044809 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76c4v\" (UniqueName: \"kubernetes.io/projected/bf7650cc-88fd-4cba-bde8-c394e73944f0-kube-api-access-76c4v\") pod \"maas-controller-98f5d4d4f-7wck2\" (UID: \"bf7650cc-88fd-4cba-bde8-c394e73944f0\") " pod="opendatahub/maas-controller-98f5d4d4f-7wck2" Sep 13 20:24:35.044915 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:35.044854 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bf7650cc-88fd-4cba-bde8-c394e73944f0-webhook-cert\") pod \"maas-controller-98f5d4d4f-7wck2\" (UID: \"bf7650cc-88fd-4cba-bde8-c394e73944f0\") " pod="opendatahub/maas-controller-98f5d4d4f-7wck2" Sep 13 20:24:35.045009 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:24:35.044983 2557 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Sep 13 20:24:35.045070 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:24:35.045058 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf7650cc-88fd-4cba-bde8-c394e73944f0-webhook-cert podName:bf7650cc-88fd-4cba-bde8-c394e73944f0 nodeName:}" failed. No retries permitted until 2026-09-13 20:24:35.545034903 +0000 UTC m=+707.721470904 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/bf7650cc-88fd-4cba-bde8-c394e73944f0-webhook-cert") pod "maas-controller-98f5d4d4f-7wck2" (UID: "bf7650cc-88fd-4cba-bde8-c394e73944f0") : secret "maas-controller-webhook-cert" not found Sep 13 20:24:35.047351 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:35.047326 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf7650cc-88fd-4cba-bde8-c394e73944f0-metrics-certs\") pod \"maas-controller-98f5d4d4f-7wck2\" (UID: \"bf7650cc-88fd-4cba-bde8-c394e73944f0\") " pod="opendatahub/maas-controller-98f5d4d4f-7wck2" Sep 13 20:24:35.052465 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:35.052441 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76c4v\" (UniqueName: \"kubernetes.io/projected/bf7650cc-88fd-4cba-bde8-c394e73944f0-kube-api-access-76c4v\") pod \"maas-controller-98f5d4d4f-7wck2\" (UID: \"bf7650cc-88fd-4cba-bde8-c394e73944f0\") " pod="opendatahub/maas-controller-98f5d4d4f-7wck2" Sep 13 20:24:35.145906 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:35.145822 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/533f5397-7ed5-4ff8-af5e-2d7a108aecc3-metrics-certs\") pod \"maas-controller-7fbc5958cd-xww4d\" (UID: \"533f5397-7ed5-4ff8-af5e-2d7a108aecc3\") " pod="opendatahub/maas-controller-7fbc5958cd-xww4d" Sep 13 20:24:35.146051 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:35.145938 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkh2d\" (UniqueName: \"kubernetes.io/projected/533f5397-7ed5-4ff8-af5e-2d7a108aecc3-kube-api-access-gkh2d\") pod \"maas-controller-7fbc5958cd-xww4d\" (UID: \"533f5397-7ed5-4ff8-af5e-2d7a108aecc3\") " pod="opendatahub/maas-controller-7fbc5958cd-xww4d" Sep 13 20:24:35.146051 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:35.145972 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/533f5397-7ed5-4ff8-af5e-2d7a108aecc3-webhook-cert\") pod \"maas-controller-7fbc5958cd-xww4d\" (UID: \"533f5397-7ed5-4ff8-af5e-2d7a108aecc3\") " pod="opendatahub/maas-controller-7fbc5958cd-xww4d" Sep 13 20:24:35.246672 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:35.246635 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkh2d\" (UniqueName: \"kubernetes.io/projected/533f5397-7ed5-4ff8-af5e-2d7a108aecc3-kube-api-access-gkh2d\") pod \"maas-controller-7fbc5958cd-xww4d\" (UID: \"533f5397-7ed5-4ff8-af5e-2d7a108aecc3\") " pod="opendatahub/maas-controller-7fbc5958cd-xww4d" Sep 13 20:24:35.246873 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:35.246681 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/533f5397-7ed5-4ff8-af5e-2d7a108aecc3-webhook-cert\") pod \"maas-controller-7fbc5958cd-xww4d\" (UID: \"533f5397-7ed5-4ff8-af5e-2d7a108aecc3\") " pod="opendatahub/maas-controller-7fbc5958cd-xww4d" Sep 13 20:24:35.246873 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:35.246746 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/533f5397-7ed5-4ff8-af5e-2d7a108aecc3-metrics-certs\") pod \"maas-controller-7fbc5958cd-xww4d\" (UID: \"533f5397-7ed5-4ff8-af5e-2d7a108aecc3\") " pod="opendatahub/maas-controller-7fbc5958cd-xww4d" Sep 13 20:24:35.249182 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:35.249157 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/533f5397-7ed5-4ff8-af5e-2d7a108aecc3-webhook-cert\") pod \"maas-controller-7fbc5958cd-xww4d\" (UID: \"533f5397-7ed5-4ff8-af5e-2d7a108aecc3\") " pod="opendatahub/maas-controller-7fbc5958cd-xww4d" Sep 13 20:24:35.249310 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:35.249196 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/533f5397-7ed5-4ff8-af5e-2d7a108aecc3-metrics-certs\") pod \"maas-controller-7fbc5958cd-xww4d\" (UID: \"533f5397-7ed5-4ff8-af5e-2d7a108aecc3\") " pod="opendatahub/maas-controller-7fbc5958cd-xww4d" Sep 13 20:24:35.254766 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:35.254743 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkh2d\" (UniqueName: \"kubernetes.io/projected/533f5397-7ed5-4ff8-af5e-2d7a108aecc3-kube-api-access-gkh2d\") pod \"maas-controller-7fbc5958cd-xww4d\" (UID: \"533f5397-7ed5-4ff8-af5e-2d7a108aecc3\") " pod="opendatahub/maas-controller-7fbc5958cd-xww4d" Sep 13 20:24:35.321049 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:35.321016 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fbc5958cd-xww4d" Sep 13 20:24:35.549424 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:35.549388 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bf7650cc-88fd-4cba-bde8-c394e73944f0-webhook-cert\") pod \"maas-controller-98f5d4d4f-7wck2\" (UID: \"bf7650cc-88fd-4cba-bde8-c394e73944f0\") " pod="opendatahub/maas-controller-98f5d4d4f-7wck2" Sep 13 20:24:35.551784 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:35.551758 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bf7650cc-88fd-4cba-bde8-c394e73944f0-webhook-cert\") pod \"maas-controller-98f5d4d4f-7wck2\" (UID: \"bf7650cc-88fd-4cba-bde8-c394e73944f0\") " pod="opendatahub/maas-controller-98f5d4d4f-7wck2" Sep 13 20:24:35.590875 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:35.590846 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fbc5958cd-xww4d"] Sep 13 20:24:35.592264 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:24:35.592224 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod533f5397_7ed5_4ff8_af5e_2d7a108aecc3.slice/crio-8257e7ae1a3c368d7ec9ef5cb66b8e045cce0ea56994f56eefbaeb73bf9d1f90 WatchSource:0}: Error finding container 8257e7ae1a3c368d7ec9ef5cb66b8e045cce0ea56994f56eefbaeb73bf9d1f90: Status 404 returned error can't find the container with id 8257e7ae1a3c368d7ec9ef5cb66b8e045cce0ea56994f56eefbaeb73bf9d1f90 Sep 13 20:24:35.773145 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:35.773113 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-98f5d4d4f-7wck2" Sep 13 20:24:36.044889 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:36.044776 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-98f5d4d4f-7wck2"] Sep 13 20:24:36.047655 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:24:36.047624 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf7650cc_88fd_4cba_bde8_c394e73944f0.slice/crio-42e3270371c057e96b3dea834e3188ed30afa76edddacadd8a6b6d4150c2e4dd WatchSource:0}: Error finding container 42e3270371c057e96b3dea834e3188ed30afa76edddacadd8a6b6d4150c2e4dd: Status 404 returned error can't find the container with id 42e3270371c057e96b3dea834e3188ed30afa76edddacadd8a6b6d4150c2e4dd Sep 13 20:24:36.358815 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:36.358734 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-98f5d4d4f-7wck2" event={"ID":"bf7650cc-88fd-4cba-bde8-c394e73944f0","Type":"ContainerStarted","Data":"42e3270371c057e96b3dea834e3188ed30afa76edddacadd8a6b6d4150c2e4dd"} Sep 13 20:24:36.360627 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:36.360595 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fbc5958cd-xww4d" event={"ID":"533f5397-7ed5-4ff8-af5e-2d7a108aecc3","Type":"ContainerStarted","Data":"8257e7ae1a3c368d7ec9ef5cb66b8e045cce0ea56994f56eefbaeb73bf9d1f90"} Sep 13 20:24:38.374682 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:38.374646 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-98f5d4d4f-7wck2" event={"ID":"bf7650cc-88fd-4cba-bde8-c394e73944f0","Type":"ContainerStarted","Data":"142b43fa9d9238f179afd24ee068af4757b07020c82cb25711d28889e416642f"} Sep 13 20:24:38.375146 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:38.374711 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-98f5d4d4f-7wck2" Sep 13 20:24:38.376317 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:38.376288 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fbc5958cd-xww4d" event={"ID":"533f5397-7ed5-4ff8-af5e-2d7a108aecc3","Type":"ContainerStarted","Data":"9394b3abda152fda937f3ebb0954725a275607771c221e946fadada1da1fc075"} Sep 13 20:24:38.376438 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:38.376336 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7fbc5958cd-xww4d" Sep 13 20:24:38.396670 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:38.396613 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-98f5d4d4f-7wck2" podStartSLOduration=2.251307614 podStartE2EDuration="4.396598616s" totalImagesPullingTime="2.145291002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:24:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:24:38.396204671 +0000 UTC m=+710.572640693" watchObservedRunningTime="2026-09-13 20:24:38.396598616 +0000 UTC m=+710.573034649" Sep 13 20:24:38.420283 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:38.420222 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7fbc5958cd-xww4d" podStartSLOduration=1.8262728780000002 podStartE2EDuration="4.420205911s" totalImagesPullingTime="2.593933033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:24:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:24:38.419352068 +0000 UTC m=+710.595788091" watchObservedRunningTime="2026-09-13 20:24:38.420205911 +0000 UTC m=+710.596641933" Sep 13 20:24:49.383678 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:49.383645 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-98f5d4d4f-7wck2" Sep 13 20:24:49.384055 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:49.383721 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7fbc5958cd-xww4d" Sep 13 20:24:49.439024 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:49.438955 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-98f5d4d4f-7wck2"] Sep 13 20:24:49.440200 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:49.440131 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-98f5d4d4f-7wck2" podUID="bf7650cc-88fd-4cba-bde8-c394e73944f0" containerName="manager" containerID="cri-o://142b43fa9d9238f179afd24ee068af4757b07020c82cb25711d28889e416642f" gracePeriod=30 Sep 13 20:24:49.688464 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:49.688438 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-98f5d4d4f-7wck2" Sep 13 20:24:49.773459 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:49.773422 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf7650cc-88fd-4cba-bde8-c394e73944f0-metrics-certs\") pod \"bf7650cc-88fd-4cba-bde8-c394e73944f0\" (UID: \"bf7650cc-88fd-4cba-bde8-c394e73944f0\") " Sep 13 20:24:49.773676 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:49.773479 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76c4v\" (UniqueName: \"kubernetes.io/projected/bf7650cc-88fd-4cba-bde8-c394e73944f0-kube-api-access-76c4v\") pod \"bf7650cc-88fd-4cba-bde8-c394e73944f0\" (UID: \"bf7650cc-88fd-4cba-bde8-c394e73944f0\") " Sep 13 20:24:49.773676 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:49.773620 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bf7650cc-88fd-4cba-bde8-c394e73944f0-webhook-cert\") pod \"bf7650cc-88fd-4cba-bde8-c394e73944f0\" (UID: \"bf7650cc-88fd-4cba-bde8-c394e73944f0\") " Sep 13 20:24:49.775676 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:49.775644 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf7650cc-88fd-4cba-bde8-c394e73944f0-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "bf7650cc-88fd-4cba-bde8-c394e73944f0" (UID: "bf7650cc-88fd-4cba-bde8-c394e73944f0"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 20:24:49.775783 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:49.775699 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf7650cc-88fd-4cba-bde8-c394e73944f0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bf7650cc-88fd-4cba-bde8-c394e73944f0" (UID: "bf7650cc-88fd-4cba-bde8-c394e73944f0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 20:24:49.775783 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:49.775715 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf7650cc-88fd-4cba-bde8-c394e73944f0-kube-api-access-76c4v" (OuterVolumeSpecName: "kube-api-access-76c4v") pod "bf7650cc-88fd-4cba-bde8-c394e73944f0" (UID: "bf7650cc-88fd-4cba-bde8-c394e73944f0"). InnerVolumeSpecName "kube-api-access-76c4v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 20:24:49.874992 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:49.874958 2557 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bf7650cc-88fd-4cba-bde8-c394e73944f0-webhook-cert\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:24:49.874992 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:49.874987 2557 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf7650cc-88fd-4cba-bde8-c394e73944f0-metrics-certs\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:24:49.874992 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:49.875001 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76c4v\" (UniqueName: \"kubernetes.io/projected/bf7650cc-88fd-4cba-bde8-c394e73944f0-kube-api-access-76c4v\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:24:50.446559 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:50.446520 2557 generic.go:356] "Generic (PLEG): container finished" podID="bf7650cc-88fd-4cba-bde8-c394e73944f0" containerID="142b43fa9d9238f179afd24ee068af4757b07020c82cb25711d28889e416642f" exitCode=0 Sep 13 20:24:50.446559 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:50.446555 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-98f5d4d4f-7wck2" Sep 13 20:24:50.447076 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:50.446587 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-98f5d4d4f-7wck2" event={"ID":"bf7650cc-88fd-4cba-bde8-c394e73944f0","Type":"ContainerDied","Data":"142b43fa9d9238f179afd24ee068af4757b07020c82cb25711d28889e416642f"} Sep 13 20:24:50.447076 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:50.446645 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-98f5d4d4f-7wck2" event={"ID":"bf7650cc-88fd-4cba-bde8-c394e73944f0","Type":"ContainerDied","Data":"42e3270371c057e96b3dea834e3188ed30afa76edddacadd8a6b6d4150c2e4dd"} Sep 13 20:24:50.447076 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:50.446670 2557 scope.go:117] "RemoveContainer" containerID="142b43fa9d9238f179afd24ee068af4757b07020c82cb25711d28889e416642f" Sep 13 20:24:50.455318 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:50.455263 2557 scope.go:117] "RemoveContainer" containerID="142b43fa9d9238f179afd24ee068af4757b07020c82cb25711d28889e416642f" Sep 13 20:24:50.455547 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:24:50.455529 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"142b43fa9d9238f179afd24ee068af4757b07020c82cb25711d28889e416642f\": container with ID starting with 142b43fa9d9238f179afd24ee068af4757b07020c82cb25711d28889e416642f not found: ID does not exist" containerID="142b43fa9d9238f179afd24ee068af4757b07020c82cb25711d28889e416642f" Sep 13 20:24:50.455608 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:50.455556 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"142b43fa9d9238f179afd24ee068af4757b07020c82cb25711d28889e416642f"} err="failed to get container status \"142b43fa9d9238f179afd24ee068af4757b07020c82cb25711d28889e416642f\": rpc error: code = NotFound desc = could not find container \"142b43fa9d9238f179afd24ee068af4757b07020c82cb25711d28889e416642f\": container with ID starting with 142b43fa9d9238f179afd24ee068af4757b07020c82cb25711d28889e416642f not found: ID does not exist" Sep 13 20:24:50.463221 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:50.463200 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-98f5d4d4f-7wck2"] Sep 13 20:24:50.470291 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:50.470271 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-98f5d4d4f-7wck2"] Sep 13 20:24:52.312898 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:24:52.312864 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf7650cc-88fd-4cba-bde8-c394e73944f0" path="/var/lib/kubelet/pods/bf7650cc-88fd-4cba-bde8-c394e73944f0/volumes" Sep 13 20:25:11.635937 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:11.635898 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-mtpn9"] Sep 13 20:25:11.636413 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:11.636284 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf7650cc-88fd-4cba-bde8-c394e73944f0" containerName="manager" Sep 13 20:25:11.636413 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:11.636295 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf7650cc-88fd-4cba-bde8-c394e73944f0" containerName="manager" Sep 13 20:25:11.636413 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:11.636365 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf7650cc-88fd-4cba-bde8-c394e73944f0" containerName="manager" Sep 13 20:25:11.662490 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:11.662449 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-mtpn9"] Sep 13 20:25:11.662490 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:11.662508 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-mtpn9" Sep 13 20:25:11.665015 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:11.664993 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Sep 13 20:25:11.665792 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:11.665775 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-qhjv2\"" Sep 13 20:25:11.665886 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:11.665842 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 13 20:25:11.754800 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:11.754760 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a0152e1-e1e1-4805-a0ca-abaaddfcd66d-maas-api-tls\") pod \"maas-api-59f6566bbc-mtpn9\" (UID: \"3a0152e1-e1e1-4805-a0ca-abaaddfcd66d\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-mtpn9" Sep 13 20:25:11.754979 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:11.754824 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a0152e1-e1e1-4805-a0ca-abaaddfcd66d-openshift-service-ca\") pod \"maas-api-59f6566bbc-mtpn9\" (UID: \"3a0152e1-e1e1-4805-a0ca-abaaddfcd66d\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-mtpn9" Sep 13 20:25:11.754979 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:11.754849 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcgpl\" (UniqueName: \"kubernetes.io/projected/3a0152e1-e1e1-4805-a0ca-abaaddfcd66d-kube-api-access-kcgpl\") pod \"maas-api-59f6566bbc-mtpn9\" (UID: \"3a0152e1-e1e1-4805-a0ca-abaaddfcd66d\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-mtpn9" Sep 13 20:25:11.754979 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:11.754908 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a0152e1-e1e1-4805-a0ca-abaaddfcd66d-metrics-certs\") pod \"maas-api-59f6566bbc-mtpn9\" (UID: \"3a0152e1-e1e1-4805-a0ca-abaaddfcd66d\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-mtpn9" Sep 13 20:25:11.855758 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:11.855725 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a0152e1-e1e1-4805-a0ca-abaaddfcd66d-openshift-service-ca\") pod \"maas-api-59f6566bbc-mtpn9\" (UID: \"3a0152e1-e1e1-4805-a0ca-abaaddfcd66d\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-mtpn9" Sep 13 20:25:11.855916 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:11.855771 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcgpl\" (UniqueName: \"kubernetes.io/projected/3a0152e1-e1e1-4805-a0ca-abaaddfcd66d-kube-api-access-kcgpl\") pod \"maas-api-59f6566bbc-mtpn9\" (UID: \"3a0152e1-e1e1-4805-a0ca-abaaddfcd66d\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-mtpn9" Sep 13 20:25:11.855916 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:11.855848 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a0152e1-e1e1-4805-a0ca-abaaddfcd66d-metrics-certs\") pod \"maas-api-59f6566bbc-mtpn9\" (UID: \"3a0152e1-e1e1-4805-a0ca-abaaddfcd66d\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-mtpn9" Sep 13 20:25:11.855916 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:11.855901 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a0152e1-e1e1-4805-a0ca-abaaddfcd66d-maas-api-tls\") pod \"maas-api-59f6566bbc-mtpn9\" (UID: \"3a0152e1-e1e1-4805-a0ca-abaaddfcd66d\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-mtpn9" Sep 13 20:25:11.856044 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:25:11.856017 2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Sep 13 20:25:11.856095 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:25:11.856084 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a0152e1-e1e1-4805-a0ca-abaaddfcd66d-maas-api-tls podName:3a0152e1-e1e1-4805-a0ca-abaaddfcd66d nodeName:}" failed. No retries permitted until 2026-09-13 20:25:12.356061681 +0000 UTC m=+744.532497685 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3a0152e1-e1e1-4805-a0ca-abaaddfcd66d-maas-api-tls") pod "maas-api-59f6566bbc-mtpn9" (UID: "3a0152e1-e1e1-4805-a0ca-abaaddfcd66d") : secret "maas-api-serving-cert" not found Sep 13 20:25:11.856478 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:11.856453 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a0152e1-e1e1-4805-a0ca-abaaddfcd66d-openshift-service-ca\") pod \"maas-api-59f6566bbc-mtpn9\" (UID: \"3a0152e1-e1e1-4805-a0ca-abaaddfcd66d\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-mtpn9" Sep 13 20:25:11.858458 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:11.858427 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a0152e1-e1e1-4805-a0ca-abaaddfcd66d-metrics-certs\") pod \"maas-api-59f6566bbc-mtpn9\" (UID: \"3a0152e1-e1e1-4805-a0ca-abaaddfcd66d\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-mtpn9" Sep 13 20:25:11.864386 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:11.864359 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcgpl\" (UniqueName: \"kubernetes.io/projected/3a0152e1-e1e1-4805-a0ca-abaaddfcd66d-kube-api-access-kcgpl\") pod \"maas-api-59f6566bbc-mtpn9\" (UID: \"3a0152e1-e1e1-4805-a0ca-abaaddfcd66d\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-mtpn9" Sep 13 20:25:12.110067 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.110025 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-9mrsb"] Sep 13 20:25:12.113927 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.113905 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9mrsb" Sep 13 20:25:12.117751 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.117727 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-k9jgj\"" Sep 13 20:25:12.117908 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.117738 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 13 20:25:12.125482 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.125459 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-9mrsb"] Sep 13 20:25:12.151161 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.151119 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-w2lf9"] Sep 13 20:25:12.156019 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.155992 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-w2lf9" Sep 13 20:25:12.159163 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.159137 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f291e44-3e5f-4d94-8b6c-95335c7ca511-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-9mrsb\" (UID: \"4f291e44-3e5f-4d94-8b6c-95335c7ca511\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9mrsb" Sep 13 20:25:12.159304 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.159286 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvlhc\" (UniqueName: \"kubernetes.io/projected/4f291e44-3e5f-4d94-8b6c-95335c7ca511-kube-api-access-kvlhc\") pod \"payload-pre-processing-59fbc47bf8-9mrsb\" (UID: \"4f291e44-3e5f-4d94-8b6c-95335c7ca511\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9mrsb" Sep 13 20:25:12.162926 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.162901 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-w2lf9"] Sep 13 20:25:12.260052 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.260015 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvlhc\" (UniqueName: \"kubernetes.io/projected/4f291e44-3e5f-4d94-8b6c-95335c7ca511-kube-api-access-kvlhc\") pod \"payload-pre-processing-59fbc47bf8-9mrsb\" (UID: \"4f291e44-3e5f-4d94-8b6c-95335c7ca511\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9mrsb" Sep 13 20:25:12.260235 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.260092 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e0aabcd-223b-46a8-a78b-11c25c1ce236-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-w2lf9\" (UID: \"2e0aabcd-223b-46a8-a78b-11c25c1ce236\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-w2lf9" Sep 13 20:25:12.260235 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.260110 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f291e44-3e5f-4d94-8b6c-95335c7ca511-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-9mrsb\" (UID: \"4f291e44-3e5f-4d94-8b6c-95335c7ca511\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9mrsb" Sep 13 20:25:12.260235 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.260137 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh7xx\" (UniqueName: \"kubernetes.io/projected/2e0aabcd-223b-46a8-a78b-11c25c1ce236-kube-api-access-zh7xx\") pod \"payload-processing-5b54b7fbb7-w2lf9\" (UID: \"2e0aabcd-223b-46a8-a78b-11c25c1ce236\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-w2lf9" Sep 13 20:25:12.260863 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.260844 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f291e44-3e5f-4d94-8b6c-95335c7ca511-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-9mrsb\" (UID: \"4f291e44-3e5f-4d94-8b6c-95335c7ca511\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9mrsb" Sep 13 20:25:12.268021 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.267991 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvlhc\" (UniqueName: \"kubernetes.io/projected/4f291e44-3e5f-4d94-8b6c-95335c7ca511-kube-api-access-kvlhc\") pod \"payload-pre-processing-59fbc47bf8-9mrsb\" (UID: \"4f291e44-3e5f-4d94-8b6c-95335c7ca511\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9mrsb" Sep 13 20:25:12.361146 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.361062 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a0152e1-e1e1-4805-a0ca-abaaddfcd66d-maas-api-tls\") pod \"maas-api-59f6566bbc-mtpn9\" (UID: \"3a0152e1-e1e1-4805-a0ca-abaaddfcd66d\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-mtpn9" Sep 13 20:25:12.361146 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.361106 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e0aabcd-223b-46a8-a78b-11c25c1ce236-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-w2lf9\" (UID: \"2e0aabcd-223b-46a8-a78b-11c25c1ce236\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-w2lf9" Sep 13 20:25:12.361146 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.361142 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh7xx\" (UniqueName: \"kubernetes.io/projected/2e0aabcd-223b-46a8-a78b-11c25c1ce236-kube-api-access-zh7xx\") pod \"payload-processing-5b54b7fbb7-w2lf9\" (UID: \"2e0aabcd-223b-46a8-a78b-11c25c1ce236\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-w2lf9" Sep 13 20:25:12.362126 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.362081 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e0aabcd-223b-46a8-a78b-11c25c1ce236-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-w2lf9\" (UID: \"2e0aabcd-223b-46a8-a78b-11c25c1ce236\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-w2lf9" Sep 13 20:25:12.363951 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.363930 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a0152e1-e1e1-4805-a0ca-abaaddfcd66d-maas-api-tls\") pod \"maas-api-59f6566bbc-mtpn9\" (UID: \"3a0152e1-e1e1-4805-a0ca-abaaddfcd66d\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-mtpn9" Sep 13 20:25:12.368702 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.368681 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh7xx\" (UniqueName: \"kubernetes.io/projected/2e0aabcd-223b-46a8-a78b-11c25c1ce236-kube-api-access-zh7xx\") pod \"payload-processing-5b54b7fbb7-w2lf9\" (UID: \"2e0aabcd-223b-46a8-a78b-11c25c1ce236\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-w2lf9" Sep 13 20:25:12.425579 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.425549 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9mrsb" Sep 13 20:25:12.469188 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.469145 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-w2lf9" Sep 13 20:25:12.574596 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.574565 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-mtpn9" Sep 13 20:25:12.699543 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.699489 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-9mrsb"] Sep 13 20:25:12.701385 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:25:12.701332 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f291e44_3e5f_4d94_8b6c_95335c7ca511.slice/crio-3cbee1a2b8544a6c766dd26e252bfe908f51e038d90a882f6bc04e9a5609e489 WatchSource:0}: Error finding container 3cbee1a2b8544a6c766dd26e252bfe908f51e038d90a882f6bc04e9a5609e489: Status 404 returned error can't find the container with id 3cbee1a2b8544a6c766dd26e252bfe908f51e038d90a882f6bc04e9a5609e489 Sep 13 20:25:12.757883 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.757852 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-w2lf9"] Sep 13 20:25:12.758812 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:25:12.758786 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e0aabcd_223b_46a8_a78b_11c25c1ce236.slice/crio-1ea8b2689146be8c0a75a46c78131a91f2412e210c8687302ccaba79ecc46a15 WatchSource:0}: Error finding container 1ea8b2689146be8c0a75a46c78131a91f2412e210c8687302ccaba79ecc46a15: Status 404 returned error can't find the container with id 1ea8b2689146be8c0a75a46c78131a91f2412e210c8687302ccaba79ecc46a15 Sep 13 20:25:12.849700 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:12.849668 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-mtpn9"] Sep 13 20:25:12.851229 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:25:12.851206 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a0152e1_e1e1_4805_a0ca_abaaddfcd66d.slice/crio-8b59c7535f2365138f2da7e0d65585cd77c4f523e8315ee5ebacf0cad2e52984 WatchSource:0}: Error finding container 8b59c7535f2365138f2da7e0d65585cd77c4f523e8315ee5ebacf0cad2e52984: Status 404 returned error can't find the container with id 8b59c7535f2365138f2da7e0d65585cd77c4f523e8315ee5ebacf0cad2e52984 Sep 13 20:25:13.580546 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:13.580507 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-mtpn9" event={"ID":"3a0152e1-e1e1-4805-a0ca-abaaddfcd66d","Type":"ContainerStarted","Data":"8b59c7535f2365138f2da7e0d65585cd77c4f523e8315ee5ebacf0cad2e52984"} Sep 13 20:25:13.582431 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:13.582398 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-w2lf9" event={"ID":"2e0aabcd-223b-46a8-a78b-11c25c1ce236","Type":"ContainerStarted","Data":"1ea8b2689146be8c0a75a46c78131a91f2412e210c8687302ccaba79ecc46a15"} Sep 13 20:25:13.585346 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:13.585313 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9mrsb" event={"ID":"4f291e44-3e5f-4d94-8b6c-95335c7ca511","Type":"ContainerStarted","Data":"3cbee1a2b8544a6c766dd26e252bfe908f51e038d90a882f6bc04e9a5609e489"} Sep 13 20:25:15.597244 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:15.597208 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-w2lf9" event={"ID":"2e0aabcd-223b-46a8-a78b-11c25c1ce236","Type":"ContainerStarted","Data":"6d95b66d0e943958900ebbd99f06b5ef79ae36fc4de0c501ecf859f413fcace5"} Sep 13 20:25:15.597678 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:15.597276 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-w2lf9" Sep 13 20:25:15.598936 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:15.598907 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9mrsb" event={"ID":"4f291e44-3e5f-4d94-8b6c-95335c7ca511","Type":"ContainerStarted","Data":"5d5a3b77507270844e65bf55cec1b447518eceb5cc87858207f8fe39f89028a5"} Sep 13 20:25:15.599063 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:15.598965 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9mrsb" Sep 13 20:25:15.600378 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:15.600353 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-mtpn9" event={"ID":"3a0152e1-e1e1-4805-a0ca-abaaddfcd66d","Type":"ContainerStarted","Data":"8fe2ff58d5c1578ad9c997383235e139b1100f94783fa8739e95f8d9ea9a4f78"} Sep 13 20:25:15.600505 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:15.600478 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-mtpn9" Sep 13 20:25:15.613982 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:15.613941 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-w2lf9" podStartSLOduration=1.813920226 podStartE2EDuration="3.613928848s" totalImagesPullingTime="1.800008622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:25:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:25:15.612395618 +0000 UTC m=+747.788831641" watchObservedRunningTime="2026-09-13 20:25:15.613928848 +0000 UTC m=+747.790364916" Sep 13 20:25:15.630643 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:15.630599 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-mtpn9" podStartSLOduration=2.87898957 podStartE2EDuration="4.630580888s" totalImagesPullingTime="1.751591318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:25:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:25:15.627082424 +0000 UTC m=+747.803518447" watchObservedRunningTime="2026-09-13 20:25:15.630580888 +0000 UTC m=+747.807016917" Sep 13 20:25:15.643974 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:15.643927 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9mrsb" podStartSLOduration=1.790889758 podStartE2EDuration="3.643913813s" totalImagesPullingTime="1.853024055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:25:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:25:15.640410261 +0000 UTC m=+747.816846284" watchObservedRunningTime="2026-09-13 20:25:15.643913813 +0000 UTC m=+747.820349839" Sep 13 20:25:21.611257 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:21.611228 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-mtpn9" Sep 13 20:25:26.608139 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:26.608110 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-w2lf9" Sep 13 20:25:26.608635 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:26.608191 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9mrsb" Sep 13 20:25:37.396300 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:37.396265 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr"] Sep 13 20:25:37.401414 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:37.401391 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr" Sep 13 20:25:37.403626 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:37.403601 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 13 20:25:37.403737 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:37.403601 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-9rx5w\"" Sep 13 20:25:37.404487 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:37.404464 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 13 20:25:37.404627 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:37.404528 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Sep 13 20:25:37.408760 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:37.408739 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr"] Sep 13 20:25:37.583758 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:37.583717 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c5d089c-68ac-40bc-bcb8-8e86fe70c866-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr\" (UID: \"8c5d089c-68ac-40bc-bcb8-8e86fe70c866\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr" Sep 13 20:25:37.583922 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:37.583769 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htdz5\" (UniqueName: \"kubernetes.io/projected/8c5d089c-68ac-40bc-bcb8-8e86fe70c866-kube-api-access-htdz5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr\" (UID: \"8c5d089c-68ac-40bc-bcb8-8e86fe70c866\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr" Sep 13 20:25:37.583922 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:37.583843 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c5d089c-68ac-40bc-bcb8-8e86fe70c866-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr\" (UID: \"8c5d089c-68ac-40bc-bcb8-8e86fe70c866\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr" Sep 13 20:25:37.584000 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:37.583914 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c5d089c-68ac-40bc-bcb8-8e86fe70c866-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr\" (UID: \"8c5d089c-68ac-40bc-bcb8-8e86fe70c866\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr" Sep 13 20:25:37.584000 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:37.583953 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c5d089c-68ac-40bc-bcb8-8e86fe70c866-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr\" (UID: \"8c5d089c-68ac-40bc-bcb8-8e86fe70c866\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr" Sep 13 20:25:37.584000 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:37.583984 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c5d089c-68ac-40bc-bcb8-8e86fe70c866-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr\" (UID: \"8c5d089c-68ac-40bc-bcb8-8e86fe70c866\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr" Sep 13 20:25:37.685074 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:37.684976 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c5d089c-68ac-40bc-bcb8-8e86fe70c866-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr\" (UID: \"8c5d089c-68ac-40bc-bcb8-8e86fe70c866\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr" Sep 13 20:25:37.685074 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:37.685026 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c5d089c-68ac-40bc-bcb8-8e86fe70c866-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr\" (UID: \"8c5d089c-68ac-40bc-bcb8-8e86fe70c866\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr" Sep 13 20:25:37.685074 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:37.685062 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c5d089c-68ac-40bc-bcb8-8e86fe70c866-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr\" (UID: \"8c5d089c-68ac-40bc-bcb8-8e86fe70c866\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr" Sep 13 20:25:37.685361 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:37.685242 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c5d089c-68ac-40bc-bcb8-8e86fe70c866-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr\" (UID: \"8c5d089c-68ac-40bc-bcb8-8e86fe70c866\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr" Sep 13 20:25:37.685361 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:37.685309 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htdz5\" (UniqueName: \"kubernetes.io/projected/8c5d089c-68ac-40bc-bcb8-8e86fe70c866-kube-api-access-htdz5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr\" (UID: \"8c5d089c-68ac-40bc-bcb8-8e86fe70c866\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr" Sep 13 20:25:37.685475 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:37.685357 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c5d089c-68ac-40bc-bcb8-8e86fe70c866-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr\" (UID: \"8c5d089c-68ac-40bc-bcb8-8e86fe70c866\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr" Sep 13 20:25:37.685475 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:37.685427 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c5d089c-68ac-40bc-bcb8-8e86fe70c866-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr\" (UID: \"8c5d089c-68ac-40bc-bcb8-8e86fe70c866\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr" Sep 13 20:25:37.685614 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:37.685520 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c5d089c-68ac-40bc-bcb8-8e86fe70c866-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr\" (UID: \"8c5d089c-68ac-40bc-bcb8-8e86fe70c866\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr" Sep 13 20:25:37.685721 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:37.685702 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c5d089c-68ac-40bc-bcb8-8e86fe70c866-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr\" (UID: \"8c5d089c-68ac-40bc-bcb8-8e86fe70c866\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr" Sep 13 20:25:37.687462 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:37.687434 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c5d089c-68ac-40bc-bcb8-8e86fe70c866-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr\" (UID: \"8c5d089c-68ac-40bc-bcb8-8e86fe70c866\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr" Sep 13 20:25:37.687690 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:37.687675 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c5d089c-68ac-40bc-bcb8-8e86fe70c866-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr\" (UID: \"8c5d089c-68ac-40bc-bcb8-8e86fe70c866\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr" Sep 13 20:25:37.692640 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:37.692607 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htdz5\" (UniqueName: \"kubernetes.io/projected/8c5d089c-68ac-40bc-bcb8-8e86fe70c866-kube-api-access-htdz5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr\" (UID: \"8c5d089c-68ac-40bc-bcb8-8e86fe70c866\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr" Sep 13 20:25:37.713561 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:37.713535 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr" Sep 13 20:25:37.993911 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:37.993879 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr"] Sep 13 20:25:37.995438 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:25:37.995396 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c5d089c_68ac_40bc_bcb8_8e86fe70c866.slice/crio-05536855aa38255955ff68e6ef79df90ce27e245f10bff8d201decef7c8c3242 WatchSource:0}: Error finding container 05536855aa38255955ff68e6ef79df90ce27e245f10bff8d201decef7c8c3242: Status 404 returned error can't find the container with id 05536855aa38255955ff68e6ef79df90ce27e245f10bff8d201decef7c8c3242 Sep 13 20:25:38.736861 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:38.736824 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr" event={"ID":"8c5d089c-68ac-40bc-bcb8-8e86fe70c866","Type":"ContainerStarted","Data":"05536855aa38255955ff68e6ef79df90ce27e245f10bff8d201decef7c8c3242"} Sep 13 20:25:39.746305 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:39.746261 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr" event={"ID":"8c5d089c-68ac-40bc-bcb8-8e86fe70c866","Type":"ContainerStarted","Data":"6ef53268c5ebda4659ef013d97f556c783812a0de650d7ecfe4531ed0b9bc920"} Sep 13 20:25:39.762681 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:39.762633 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr" podStartSLOduration=1.50441499 podStartE2EDuration="2.76261738s" totalImagesPullingTime="1.25820239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:25:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:25:39.761391534 +0000 UTC m=+771.937827557" watchObservedRunningTime="2026-09-13 20:25:39.76261738 +0000 UTC m=+771.939053406" Sep 13 20:25:47.714571 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:47.714530 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr" Sep 13 20:25:47.714998 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:47.714622 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr" Sep 13 20:25:47.727224 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:47.727201 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr" Sep 13 20:25:47.802508 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:47.802458 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dh8mr" Sep 13 20:25:51.401232 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:51.401194 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk"] Sep 13 20:25:51.406270 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:51.406246 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk" Sep 13 20:25:51.408695 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:51.408673 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\"" Sep 13 20:25:51.418065 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:51.418040 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk"] Sep 13 20:25:51.507284 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:51.507247 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e568518e-8595-4506-83c2-1d81b6f5454a-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk\" (UID: \"e568518e-8595-4506-83c2-1d81b6f5454a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk" Sep 13 20:25:51.507284 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:51.507280 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e568518e-8595-4506-83c2-1d81b6f5454a-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk\" (UID: \"e568518e-8595-4506-83c2-1d81b6f5454a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk" Sep 13 20:25:51.507483 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:51.507309 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e568518e-8595-4506-83c2-1d81b6f5454a-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk\" (UID: \"e568518e-8595-4506-83c2-1d81b6f5454a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk" Sep 13 20:25:51.507483 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:51.507405 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e568518e-8595-4506-83c2-1d81b6f5454a-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk\" (UID: \"e568518e-8595-4506-83c2-1d81b6f5454a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk" Sep 13 20:25:51.507483 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:51.507425 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dhkg\" (UniqueName: \"kubernetes.io/projected/e568518e-8595-4506-83c2-1d81b6f5454a-kube-api-access-4dhkg\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk\" (UID: \"e568518e-8595-4506-83c2-1d81b6f5454a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk" Sep 13 20:25:51.507483 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:51.507459 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e568518e-8595-4506-83c2-1d81b6f5454a-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk\" (UID: \"e568518e-8595-4506-83c2-1d81b6f5454a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk" Sep 13 20:25:51.607952 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:51.607917 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e568518e-8595-4506-83c2-1d81b6f5454a-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk\" (UID: \"e568518e-8595-4506-83c2-1d81b6f5454a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk" Sep 13 20:25:51.608147 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:51.607971 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e568518e-8595-4506-83c2-1d81b6f5454a-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk\" (UID: \"e568518e-8595-4506-83c2-1d81b6f5454a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk" Sep 13 20:25:51.608147 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:51.607994 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e568518e-8595-4506-83c2-1d81b6f5454a-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk\" (UID: \"e568518e-8595-4506-83c2-1d81b6f5454a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk" Sep 13 20:25:51.608147 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:51.608025 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e568518e-8595-4506-83c2-1d81b6f5454a-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk\" (UID: \"e568518e-8595-4506-83c2-1d81b6f5454a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk" Sep 13 20:25:51.608147 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:51.608077 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e568518e-8595-4506-83c2-1d81b6f5454a-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk\" (UID: \"e568518e-8595-4506-83c2-1d81b6f5454a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk" Sep 13 20:25:51.608147 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:51.608105 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dhkg\" (UniqueName: \"kubernetes.io/projected/e568518e-8595-4506-83c2-1d81b6f5454a-kube-api-access-4dhkg\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk\" (UID: \"e568518e-8595-4506-83c2-1d81b6f5454a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk" Sep 13 20:25:51.608522 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:51.608483 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e568518e-8595-4506-83c2-1d81b6f5454a-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk\" (UID: \"e568518e-8595-4506-83c2-1d81b6f5454a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk" Sep 13 20:25:51.608605 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:51.608540 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e568518e-8595-4506-83c2-1d81b6f5454a-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk\" (UID: \"e568518e-8595-4506-83c2-1d81b6f5454a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk" Sep 13 20:25:51.608605 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:51.608590 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e568518e-8595-4506-83c2-1d81b6f5454a-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk\" (UID: \"e568518e-8595-4506-83c2-1d81b6f5454a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk" Sep 13 20:25:51.610539 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:51.610491 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e568518e-8595-4506-83c2-1d81b6f5454a-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk\" (UID: \"e568518e-8595-4506-83c2-1d81b6f5454a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk" Sep 13 20:25:51.610844 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:51.610825 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e568518e-8595-4506-83c2-1d81b6f5454a-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk\" (UID: \"e568518e-8595-4506-83c2-1d81b6f5454a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk" Sep 13 20:25:51.617181 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:51.617151 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dhkg\" (UniqueName: \"kubernetes.io/projected/e568518e-8595-4506-83c2-1d81b6f5454a-kube-api-access-4dhkg\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk\" (UID: \"e568518e-8595-4506-83c2-1d81b6f5454a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk" Sep 13 20:25:51.717562 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:51.717456 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk" Sep 13 20:25:51.996652 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:51.996610 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk"] Sep 13 20:25:51.997926 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:25:51.997898 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode568518e_8595_4506_83c2_1d81b6f5454a.slice/crio-a9baff5de8d394009f88eefbca8c54609924862c8b9c91d33a438e4a5d354b65 WatchSource:0}: Error finding container a9baff5de8d394009f88eefbca8c54609924862c8b9c91d33a438e4a5d354b65: Status 404 returned error can't find the container with id a9baff5de8d394009f88eefbca8c54609924862c8b9c91d33a438e4a5d354b65 Sep 13 20:25:52.822014 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:52.821976 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk" event={"ID":"e568518e-8595-4506-83c2-1d81b6f5454a","Type":"ContainerStarted","Data":"714b367834bcbf935fd1b34ebb4745716fead00c0d0e8a47816d6aecaaed2206"} Sep 13 20:25:52.822390 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:52.822031 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk" event={"ID":"e568518e-8595-4506-83c2-1d81b6f5454a","Type":"ContainerStarted","Data":"a9baff5de8d394009f88eefbca8c54609924862c8b9c91d33a438e4a5d354b65"} Sep 13 20:25:52.840828 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:25:52.840762 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk" podStartSLOduration=1.652769401 podStartE2EDuration="1.840741414s" totalImagesPullingTime="187.972013ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:25:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:25:52.837309306 +0000 UTC m=+785.013745330" watchObservedRunningTime="2026-09-13 20:25:52.840741414 +0000 UTC m=+785.017177437" Sep 13 20:26:01.718232 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:26:01.718173 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk" Sep 13 20:26:01.718763 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:26:01.718268 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk" Sep 13 20:26:01.730838 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:26:01.730811 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk" Sep 13 20:26:01.883926 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:26:01.883893 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8pgjk" Sep 13 20:27:44.438800 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:44.438757 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7fbc5958cd-xww4d"] Sep 13 20:27:44.439328 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:44.438963 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7fbc5958cd-xww4d" podUID="533f5397-7ed5-4ff8-af5e-2d7a108aecc3" containerName="manager" containerID="cri-o://9394b3abda152fda937f3ebb0954725a275607771c221e946fadada1da1fc075" gracePeriod=30 Sep 13 20:27:44.676057 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:44.676027 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-w2lf9"] Sep 13 20:27:44.676328 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:44.676287 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-5b54b7fbb7-w2lf9" podUID="2e0aabcd-223b-46a8-a78b-11c25c1ce236" containerName="payload-processing" containerID="cri-o://6d95b66d0e943958900ebbd99f06b5ef79ae36fc4de0c501ecf859f413fcace5" gracePeriod=30 Sep 13 20:27:44.757730 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:44.757696 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fbc5958cd-xww4d" Sep 13 20:27:44.803163 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:44.803130 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-9mrsb"] Sep 13 20:27:44.803414 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:44.803380 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9mrsb" podUID="4f291e44-3e5f-4d94-8b6c-95335c7ca511" containerName="payload-pre-processing" containerID="cri-o://5d5a3b77507270844e65bf55cec1b447518eceb5cc87858207f8fe39f89028a5" gracePeriod=30 Sep 13 20:27:44.814958 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:44.814933 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkh2d\" (UniqueName: \"kubernetes.io/projected/533f5397-7ed5-4ff8-af5e-2d7a108aecc3-kube-api-access-gkh2d\") pod \"533f5397-7ed5-4ff8-af5e-2d7a108aecc3\" (UID: \"533f5397-7ed5-4ff8-af5e-2d7a108aecc3\") " Sep 13 20:27:44.815081 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:44.814967 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/533f5397-7ed5-4ff8-af5e-2d7a108aecc3-webhook-cert\") pod \"533f5397-7ed5-4ff8-af5e-2d7a108aecc3\" (UID: \"533f5397-7ed5-4ff8-af5e-2d7a108aecc3\") " Sep 13 20:27:44.816167 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:44.815200 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/533f5397-7ed5-4ff8-af5e-2d7a108aecc3-metrics-certs\") pod \"533f5397-7ed5-4ff8-af5e-2d7a108aecc3\" (UID: \"533f5397-7ed5-4ff8-af5e-2d7a108aecc3\") " Sep 13 20:27:44.817574 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:44.817533 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/533f5397-7ed5-4ff8-af5e-2d7a108aecc3-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "533f5397-7ed5-4ff8-af5e-2d7a108aecc3" (UID: "533f5397-7ed5-4ff8-af5e-2d7a108aecc3"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 20:27:44.817737 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:44.817590 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/533f5397-7ed5-4ff8-af5e-2d7a108aecc3-kube-api-access-gkh2d" (OuterVolumeSpecName: "kube-api-access-gkh2d") pod "533f5397-7ed5-4ff8-af5e-2d7a108aecc3" (UID: "533f5397-7ed5-4ff8-af5e-2d7a108aecc3"). InnerVolumeSpecName "kube-api-access-gkh2d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 20:27:44.817809 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:44.817770 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/533f5397-7ed5-4ff8-af5e-2d7a108aecc3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "533f5397-7ed5-4ff8-af5e-2d7a108aecc3" (UID: "533f5397-7ed5-4ff8-af5e-2d7a108aecc3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 13 20:27:44.917531 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:44.917489 2557 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/533f5397-7ed5-4ff8-af5e-2d7a108aecc3-metrics-certs\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:27:44.917531 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:44.917529 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkh2d\" (UniqueName: \"kubernetes.io/projected/533f5397-7ed5-4ff8-af5e-2d7a108aecc3-kube-api-access-gkh2d\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:27:44.917685 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:44.917540 2557 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/533f5397-7ed5-4ff8-af5e-2d7a108aecc3-webhook-cert\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:27:44.921569 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:44.921547 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-w2lf9" Sep 13 20:27:45.017993 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.017964 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e0aabcd-223b-46a8-a78b-11c25c1ce236-plugins-config-volume\") pod \"2e0aabcd-223b-46a8-a78b-11c25c1ce236\" (UID: \"2e0aabcd-223b-46a8-a78b-11c25c1ce236\") " Sep 13 20:27:45.018175 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.018063 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zh7xx\" (UniqueName: \"kubernetes.io/projected/2e0aabcd-223b-46a8-a78b-11c25c1ce236-kube-api-access-zh7xx\") pod \"2e0aabcd-223b-46a8-a78b-11c25c1ce236\" (UID: \"2e0aabcd-223b-46a8-a78b-11c25c1ce236\") " Sep 13 20:27:45.018407 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.018376 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e0aabcd-223b-46a8-a78b-11c25c1ce236-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2e0aabcd-223b-46a8-a78b-11c25c1ce236" (UID: "2e0aabcd-223b-46a8-a78b-11c25c1ce236"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 13 20:27:45.021357 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.021331 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e0aabcd-223b-46a8-a78b-11c25c1ce236-kube-api-access-zh7xx" (OuterVolumeSpecName: "kube-api-access-zh7xx") pod "2e0aabcd-223b-46a8-a78b-11c25c1ce236" (UID: "2e0aabcd-223b-46a8-a78b-11c25c1ce236"). InnerVolumeSpecName "kube-api-access-zh7xx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 20:27:45.048259 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.048227 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9mrsb" Sep 13 20:27:45.118736 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.118700 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvlhc\" (UniqueName: \"kubernetes.io/projected/4f291e44-3e5f-4d94-8b6c-95335c7ca511-kube-api-access-kvlhc\") pod \"4f291e44-3e5f-4d94-8b6c-95335c7ca511\" (UID: \"4f291e44-3e5f-4d94-8b6c-95335c7ca511\") " Sep 13 20:27:45.118914 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.118763 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f291e44-3e5f-4d94-8b6c-95335c7ca511-plugins-config-volume\") pod \"4f291e44-3e5f-4d94-8b6c-95335c7ca511\" (UID: \"4f291e44-3e5f-4d94-8b6c-95335c7ca511\") " Sep 13 20:27:45.118987 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.118971 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zh7xx\" (UniqueName: \"kubernetes.io/projected/2e0aabcd-223b-46a8-a78b-11c25c1ce236-kube-api-access-zh7xx\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:27:45.119040 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.118994 2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e0aabcd-223b-46a8-a78b-11c25c1ce236-plugins-config-volume\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:27:45.119157 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.119127 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f291e44-3e5f-4d94-8b6c-95335c7ca511-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4f291e44-3e5f-4d94-8b6c-95335c7ca511" (UID: "4f291e44-3e5f-4d94-8b6c-95335c7ca511"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 13 20:27:45.120765 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.120745 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f291e44-3e5f-4d94-8b6c-95335c7ca511-kube-api-access-kvlhc" (OuterVolumeSpecName: "kube-api-access-kvlhc") pod "4f291e44-3e5f-4d94-8b6c-95335c7ca511" (UID: "4f291e44-3e5f-4d94-8b6c-95335c7ca511"). InnerVolumeSpecName "kube-api-access-kvlhc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 20:27:45.219631 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.219598 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvlhc\" (UniqueName: \"kubernetes.io/projected/4f291e44-3e5f-4d94-8b6c-95335c7ca511-kube-api-access-kvlhc\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:27:45.219631 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.219627 2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f291e44-3e5f-4d94-8b6c-95335c7ca511-plugins-config-volume\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:27:45.492055 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.492025 2557 generic.go:356] "Generic (PLEG): container finished" podID="4f291e44-3e5f-4d94-8b6c-95335c7ca511" containerID="5d5a3b77507270844e65bf55cec1b447518eceb5cc87858207f8fe39f89028a5" exitCode=0 Sep 13 20:27:45.492439 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.492073 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9mrsb" Sep 13 20:27:45.492439 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.492111 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9mrsb" event={"ID":"4f291e44-3e5f-4d94-8b6c-95335c7ca511","Type":"ContainerDied","Data":"5d5a3b77507270844e65bf55cec1b447518eceb5cc87858207f8fe39f89028a5"} Sep 13 20:27:45.492439 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.492168 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9mrsb" event={"ID":"4f291e44-3e5f-4d94-8b6c-95335c7ca511","Type":"ContainerDied","Data":"3cbee1a2b8544a6c766dd26e252bfe908f51e038d90a882f6bc04e9a5609e489"} Sep 13 20:27:45.492439 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.492199 2557 scope.go:117] "RemoveContainer" containerID="5d5a3b77507270844e65bf55cec1b447518eceb5cc87858207f8fe39f89028a5" Sep 13 20:27:45.493587 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.493470 2557 generic.go:356] "Generic (PLEG): container finished" podID="533f5397-7ed5-4ff8-af5e-2d7a108aecc3" containerID="9394b3abda152fda937f3ebb0954725a275607771c221e946fadada1da1fc075" exitCode=0 Sep 13 20:27:45.493587 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.493537 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fbc5958cd-xww4d" Sep 13 20:27:45.493587 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.493562 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fbc5958cd-xww4d" event={"ID":"533f5397-7ed5-4ff8-af5e-2d7a108aecc3","Type":"ContainerDied","Data":"9394b3abda152fda937f3ebb0954725a275607771c221e946fadada1da1fc075"} Sep 13 20:27:45.493747 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.493623 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fbc5958cd-xww4d" event={"ID":"533f5397-7ed5-4ff8-af5e-2d7a108aecc3","Type":"ContainerDied","Data":"8257e7ae1a3c368d7ec9ef5cb66b8e045cce0ea56994f56eefbaeb73bf9d1f90"} Sep 13 20:27:45.495244 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.495218 2557 generic.go:356] "Generic (PLEG): container finished" podID="2e0aabcd-223b-46a8-a78b-11c25c1ce236" containerID="6d95b66d0e943958900ebbd99f06b5ef79ae36fc4de0c501ecf859f413fcace5" exitCode=0 Sep 13 20:27:45.495373 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.495262 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-w2lf9" event={"ID":"2e0aabcd-223b-46a8-a78b-11c25c1ce236","Type":"ContainerDied","Data":"6d95b66d0e943958900ebbd99f06b5ef79ae36fc4de0c501ecf859f413fcace5"} Sep 13 20:27:45.495373 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.495295 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-w2lf9" Sep 13 20:27:45.495485 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.495302 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-w2lf9" event={"ID":"2e0aabcd-223b-46a8-a78b-11c25c1ce236","Type":"ContainerDied","Data":"1ea8b2689146be8c0a75a46c78131a91f2412e210c8687302ccaba79ecc46a15"} Sep 13 20:27:45.503904 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.503883 2557 scope.go:117] "RemoveContainer" containerID="5d5a3b77507270844e65bf55cec1b447518eceb5cc87858207f8fe39f89028a5" Sep 13 20:27:45.504202 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:27:45.504180 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d5a3b77507270844e65bf55cec1b447518eceb5cc87858207f8fe39f89028a5\": container with ID starting with 5d5a3b77507270844e65bf55cec1b447518eceb5cc87858207f8fe39f89028a5 not found: ID does not exist" containerID="5d5a3b77507270844e65bf55cec1b447518eceb5cc87858207f8fe39f89028a5" Sep 13 20:27:45.504308 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.504211 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d5a3b77507270844e65bf55cec1b447518eceb5cc87858207f8fe39f89028a5"} err="failed to get container status \"5d5a3b77507270844e65bf55cec1b447518eceb5cc87858207f8fe39f89028a5\": rpc error: code = NotFound desc = could not find container \"5d5a3b77507270844e65bf55cec1b447518eceb5cc87858207f8fe39f89028a5\": container with ID starting with 5d5a3b77507270844e65bf55cec1b447518eceb5cc87858207f8fe39f89028a5 not found: ID does not exist" Sep 13 20:27:45.504308 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.504233 2557 scope.go:117] "RemoveContainer" containerID="9394b3abda152fda937f3ebb0954725a275607771c221e946fadada1da1fc075" Sep 13 20:27:45.513115 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.513098 2557 scope.go:117] "RemoveContainer" containerID="9394b3abda152fda937f3ebb0954725a275607771c221e946fadada1da1fc075" Sep 13 20:27:45.513382 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:27:45.513366 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9394b3abda152fda937f3ebb0954725a275607771c221e946fadada1da1fc075\": container with ID starting with 9394b3abda152fda937f3ebb0954725a275607771c221e946fadada1da1fc075 not found: ID does not exist" containerID="9394b3abda152fda937f3ebb0954725a275607771c221e946fadada1da1fc075" Sep 13 20:27:45.513436 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.513388 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9394b3abda152fda937f3ebb0954725a275607771c221e946fadada1da1fc075"} err="failed to get container status \"9394b3abda152fda937f3ebb0954725a275607771c221e946fadada1da1fc075\": rpc error: code = NotFound desc = could not find container \"9394b3abda152fda937f3ebb0954725a275607771c221e946fadada1da1fc075\": container with ID starting with 9394b3abda152fda937f3ebb0954725a275607771c221e946fadada1da1fc075 not found: ID does not exist" Sep 13 20:27:45.513436 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.513402 2557 scope.go:117] "RemoveContainer" containerID="6d95b66d0e943958900ebbd99f06b5ef79ae36fc4de0c501ecf859f413fcace5" Sep 13 20:27:45.520190 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.520167 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-9mrsb"] Sep 13 20:27:45.522519 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.522479 2557 scope.go:117] "RemoveContainer" containerID="6d95b66d0e943958900ebbd99f06b5ef79ae36fc4de0c501ecf859f413fcace5" Sep 13 20:27:45.522808 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:27:45.522793 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d95b66d0e943958900ebbd99f06b5ef79ae36fc4de0c501ecf859f413fcace5\": container with ID starting with 6d95b66d0e943958900ebbd99f06b5ef79ae36fc4de0c501ecf859f413fcace5 not found: ID does not exist" containerID="6d95b66d0e943958900ebbd99f06b5ef79ae36fc4de0c501ecf859f413fcace5" Sep 13 20:27:45.522850 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.522815 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d95b66d0e943958900ebbd99f06b5ef79ae36fc4de0c501ecf859f413fcace5"} err="failed to get container status \"6d95b66d0e943958900ebbd99f06b5ef79ae36fc4de0c501ecf859f413fcace5\": rpc error: code = NotFound desc = could not find container \"6d95b66d0e943958900ebbd99f06b5ef79ae36fc4de0c501ecf859f413fcace5\": container with ID starting with 6d95b66d0e943958900ebbd99f06b5ef79ae36fc4de0c501ecf859f413fcace5 not found: ID does not exist" Sep 13 20:27:45.529061 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.529041 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-5d6c47b998-2m2vv"] Sep 13 20:27:45.529414 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.529402 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f291e44-3e5f-4d94-8b6c-95335c7ca511" containerName="payload-pre-processing" Sep 13 20:27:45.529466 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.529416 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f291e44-3e5f-4d94-8b6c-95335c7ca511" containerName="payload-pre-processing" Sep 13 20:27:45.529466 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.529437 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e0aabcd-223b-46a8-a78b-11c25c1ce236" containerName="payload-processing" Sep 13 20:27:45.529466 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.529443 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e0aabcd-223b-46a8-a78b-11c25c1ce236" containerName="payload-processing" Sep 13 20:27:45.529466 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.529451 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="533f5397-7ed5-4ff8-af5e-2d7a108aecc3" containerName="manager" Sep 13 20:27:45.529466 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.529456 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="533f5397-7ed5-4ff8-af5e-2d7a108aecc3" containerName="manager" Sep 13 20:27:45.529657 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.529538 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e0aabcd-223b-46a8-a78b-11c25c1ce236" containerName="payload-processing" Sep 13 20:27:45.529657 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.529550 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="533f5397-7ed5-4ff8-af5e-2d7a108aecc3" containerName="manager" Sep 13 20:27:45.529657 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.529557 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f291e44-3e5f-4d94-8b6c-95335c7ca511" containerName="payload-pre-processing" Sep 13 20:27:45.534062 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.534047 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-9mrsb"] Sep 13 20:27:45.534126 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.534121 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5d6c47b998-2m2vv" Sep 13 20:27:45.536142 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.536126 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\"" Sep 13 20:27:45.536213 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.536148 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-gmdct\"" Sep 13 20:27:45.543852 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.543831 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-w2lf9"] Sep 13 20:27:45.547254 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.547234 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-5d6c47b998-2m2vv"] Sep 13 20:27:45.549743 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.549720 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-w2lf9"] Sep 13 20:27:45.557721 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.557699 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7fbc5958cd-xww4d"] Sep 13 20:27:45.560238 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.560218 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7fbc5958cd-xww4d"] Sep 13 20:27:45.623252 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.623219 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxb5p\" (UniqueName: \"kubernetes.io/projected/e02140e6-3140-4126-83e0-e5b3389bf24a-kube-api-access-hxb5p\") pod \"ai-gateway-controller-5d6c47b998-2m2vv\" (UID: \"e02140e6-3140-4126-83e0-e5b3389bf24a\") " pod="opendatahub/ai-gateway-controller-5d6c47b998-2m2vv" Sep 13 20:27:45.724576 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.724537 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxb5p\" (UniqueName: \"kubernetes.io/projected/e02140e6-3140-4126-83e0-e5b3389bf24a-kube-api-access-hxb5p\") pod \"ai-gateway-controller-5d6c47b998-2m2vv\" (UID: \"e02140e6-3140-4126-83e0-e5b3389bf24a\") " pod="opendatahub/ai-gateway-controller-5d6c47b998-2m2vv" Sep 13 20:27:45.732205 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.732137 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxb5p\" (UniqueName: \"kubernetes.io/projected/e02140e6-3140-4126-83e0-e5b3389bf24a-kube-api-access-hxb5p\") pod \"ai-gateway-controller-5d6c47b998-2m2vv\" (UID: \"e02140e6-3140-4126-83e0-e5b3389bf24a\") " pod="opendatahub/ai-gateway-controller-5d6c47b998-2m2vv" Sep 13 20:27:45.845171 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:45.845136 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5d6c47b998-2m2vv" Sep 13 20:27:46.114619 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:46.114594 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-5d6c47b998-2m2vv"] Sep 13 20:27:46.116041 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:27:46.116009 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode02140e6_3140_4126_83e0_e5b3389bf24a.slice/crio-6504ff75218e972eca026b439305c150e58b175eaf0cfece27ac7c3ca226a2fe WatchSource:0}: Error finding container 6504ff75218e972eca026b439305c150e58b175eaf0cfece27ac7c3ca226a2fe: Status 404 returned error can't find the container with id 6504ff75218e972eca026b439305c150e58b175eaf0cfece27ac7c3ca226a2fe Sep 13 20:27:46.313610 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:46.313577 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e0aabcd-223b-46a8-a78b-11c25c1ce236" path="/var/lib/kubelet/pods/2e0aabcd-223b-46a8-a78b-11c25c1ce236/volumes" Sep 13 20:27:46.314035 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:46.314014 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f291e44-3e5f-4d94-8b6c-95335c7ca511" path="/var/lib/kubelet/pods/4f291e44-3e5f-4d94-8b6c-95335c7ca511/volumes" Sep 13 20:27:46.314399 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:46.314383 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="533f5397-7ed5-4ff8-af5e-2d7a108aecc3" path="/var/lib/kubelet/pods/533f5397-7ed5-4ff8-af5e-2d7a108aecc3/volumes" Sep 13 20:27:46.505851 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:46.505764 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d6c47b998-2m2vv" event={"ID":"e02140e6-3140-4126-83e0-e5b3389bf24a","Type":"ContainerStarted","Data":"6504ff75218e972eca026b439305c150e58b175eaf0cfece27ac7c3ca226a2fe"} Sep 13 20:27:47.514370 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:47.514331 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d6c47b998-2m2vv" event={"ID":"e02140e6-3140-4126-83e0-e5b3389bf24a","Type":"ContainerStarted","Data":"96df76ec56ba1af54993aed57039e7ed940b8b001cf2b836d7c725ec4ec9f2c5"} Sep 13 20:27:47.514738 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:47.514406 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-5d6c47b998-2m2vv" Sep 13 20:27:47.530936 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:47.530885 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-5d6c47b998-2m2vv" podStartSLOduration=1.230832282 podStartE2EDuration="2.530870421s" totalImagesPullingTime="1.300038139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:27:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:27:47.528202131 +0000 UTC m=+899.704638152" watchObservedRunningTime="2026-09-13 20:27:47.530870421 +0000 UTC m=+899.707306443" Sep 13 20:27:58.521689 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:58.521658 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-5d6c47b998-2m2vv" Sep 13 20:27:58.698712 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:58.698675 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-67d46498b8-4rm52"] Sep 13 20:27:58.702445 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:58.702426 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-67d46498b8-4rm52" Sep 13 20:27:58.709347 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:58.709313 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-67d46498b8-4rm52"] Sep 13 20:27:58.844274 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:58.844186 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc6gg\" (UniqueName: \"kubernetes.io/projected/59630cc9-8929-409b-9209-6ee02946e421-kube-api-access-rc6gg\") pod \"ai-gateway-controller-67d46498b8-4rm52\" (UID: \"59630cc9-8929-409b-9209-6ee02946e421\") " pod="opendatahub/ai-gateway-controller-67d46498b8-4rm52" Sep 13 20:27:58.945095 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:58.945062 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc6gg\" (UniqueName: \"kubernetes.io/projected/59630cc9-8929-409b-9209-6ee02946e421-kube-api-access-rc6gg\") pod \"ai-gateway-controller-67d46498b8-4rm52\" (UID: \"59630cc9-8929-409b-9209-6ee02946e421\") " pod="opendatahub/ai-gateway-controller-67d46498b8-4rm52" Sep 13 20:27:58.952484 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:58.952457 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc6gg\" (UniqueName: \"kubernetes.io/projected/59630cc9-8929-409b-9209-6ee02946e421-kube-api-access-rc6gg\") pod \"ai-gateway-controller-67d46498b8-4rm52\" (UID: \"59630cc9-8929-409b-9209-6ee02946e421\") " pod="opendatahub/ai-gateway-controller-67d46498b8-4rm52" Sep 13 20:27:59.015381 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:59.015348 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-67d46498b8-4rm52" Sep 13 20:27:59.286279 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:59.286247 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-67d46498b8-4rm52"] Sep 13 20:27:59.287695 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:27:59.287670 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59630cc9_8929_409b_9209_6ee02946e421.slice/crio-ce6421445f78ef9e3d16e9af0404d9b1fda4002e59fde02f53cc96e58830cef3 WatchSource:0}: Error finding container ce6421445f78ef9e3d16e9af0404d9b1fda4002e59fde02f53cc96e58830cef3: Status 404 returned error can't find the container with id ce6421445f78ef9e3d16e9af0404d9b1fda4002e59fde02f53cc96e58830cef3 Sep 13 20:27:59.586594 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:27:59.586522 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-67d46498b8-4rm52" event={"ID":"59630cc9-8929-409b-9209-6ee02946e421","Type":"ContainerStarted","Data":"ce6421445f78ef9e3d16e9af0404d9b1fda4002e59fde02f53cc96e58830cef3"} Sep 13 20:28:00.593749 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:28:00.593712 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-67d46498b8-4rm52" event={"ID":"59630cc9-8929-409b-9209-6ee02946e421","Type":"ContainerStarted","Data":"58df0b26efc8b32dbac64785e9d9fd1d9239dc782ec73d8db91845edd27f9243"} Sep 13 20:28:00.594212 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:28:00.593921 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-67d46498b8-4rm52" Sep 13 20:28:00.610276 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:28:00.610227 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-67d46498b8-4rm52" podStartSLOduration=2.3034923 podStartE2EDuration="2.610211419s" totalImagesPullingTime="306.719119ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:27:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:28:00.607462411 +0000 UTC m=+912.783898433" watchObservedRunningTime="2026-09-13 20:28:00.610211419 +0000 UTC m=+912.786647441" Sep 13 20:28:11.602255 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:28:11.602222 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-67d46498b8-4rm52" Sep 13 20:28:11.641995 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:28:11.641958 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-5d6c47b998-2m2vv"] Sep 13 20:28:11.642189 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:28:11.642162 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-5d6c47b998-2m2vv" podUID="e02140e6-3140-4126-83e0-e5b3389bf24a" containerName="manager" containerID="cri-o://96df76ec56ba1af54993aed57039e7ed940b8b001cf2b836d7c725ec4ec9f2c5" gracePeriod=30 Sep 13 20:28:11.888147 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:28:11.888121 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5d6c47b998-2m2vv" Sep 13 20:28:11.949705 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:28:11.949671 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxb5p\" (UniqueName: \"kubernetes.io/projected/e02140e6-3140-4126-83e0-e5b3389bf24a-kube-api-access-hxb5p\") pod \"e02140e6-3140-4126-83e0-e5b3389bf24a\" (UID: \"e02140e6-3140-4126-83e0-e5b3389bf24a\") " Sep 13 20:28:11.951750 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:28:11.951723 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e02140e6-3140-4126-83e0-e5b3389bf24a-kube-api-access-hxb5p" (OuterVolumeSpecName: "kube-api-access-hxb5p") pod "e02140e6-3140-4126-83e0-e5b3389bf24a" (UID: "e02140e6-3140-4126-83e0-e5b3389bf24a"). InnerVolumeSpecName "kube-api-access-hxb5p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 13 20:28:12.050737 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:28:12.050704 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxb5p\" (UniqueName: \"kubernetes.io/projected/e02140e6-3140-4126-83e0-e5b3389bf24a-kube-api-access-hxb5p\") on node \"ip-10-0-134-99.ec2.internal\" DevicePath \"\"" Sep 13 20:28:12.668641 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:28:12.668540 2557 generic.go:356] "Generic (PLEG): container finished" podID="e02140e6-3140-4126-83e0-e5b3389bf24a" containerID="96df76ec56ba1af54993aed57039e7ed940b8b001cf2b836d7c725ec4ec9f2c5" exitCode=0 Sep 13 20:28:12.668641 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:28:12.668579 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5d6c47b998-2m2vv" Sep 13 20:28:12.668641 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:28:12.668608 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d6c47b998-2m2vv" event={"ID":"e02140e6-3140-4126-83e0-e5b3389bf24a","Type":"ContainerDied","Data":"96df76ec56ba1af54993aed57039e7ed940b8b001cf2b836d7c725ec4ec9f2c5"} Sep 13 20:28:12.669205 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:28:12.668661 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d6c47b998-2m2vv" event={"ID":"e02140e6-3140-4126-83e0-e5b3389bf24a","Type":"ContainerDied","Data":"6504ff75218e972eca026b439305c150e58b175eaf0cfece27ac7c3ca226a2fe"} Sep 13 20:28:12.669205 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:28:12.668688 2557 scope.go:117] "RemoveContainer" containerID="96df76ec56ba1af54993aed57039e7ed940b8b001cf2b836d7c725ec4ec9f2c5" Sep 13 20:28:12.677678 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:28:12.677653 2557 scope.go:117] "RemoveContainer" containerID="96df76ec56ba1af54993aed57039e7ed940b8b001cf2b836d7c725ec4ec9f2c5" Sep 13 20:28:12.677918 ip-10-0-134-99 kubenswrapper[2557]: E0913 20:28:12.677896 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96df76ec56ba1af54993aed57039e7ed940b8b001cf2b836d7c725ec4ec9f2c5\": container with ID starting with 96df76ec56ba1af54993aed57039e7ed940b8b001cf2b836d7c725ec4ec9f2c5 not found: ID does not exist" containerID="96df76ec56ba1af54993aed57039e7ed940b8b001cf2b836d7c725ec4ec9f2c5" Sep 13 20:28:12.677969 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:28:12.677928 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96df76ec56ba1af54993aed57039e7ed940b8b001cf2b836d7c725ec4ec9f2c5"} err="failed to get container status \"96df76ec56ba1af54993aed57039e7ed940b8b001cf2b836d7c725ec4ec9f2c5\": rpc error: code = NotFound desc = could not find container \"96df76ec56ba1af54993aed57039e7ed940b8b001cf2b836d7c725ec4ec9f2c5\": container with ID starting with 96df76ec56ba1af54993aed57039e7ed940b8b001cf2b836d7c725ec4ec9f2c5 not found: ID does not exist" Sep 13 20:28:12.683743 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:28:12.683720 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-5d6c47b998-2m2vv"] Sep 13 20:28:12.687061 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:28:12.687040 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-5d6c47b998-2m2vv"] Sep 13 20:28:14.313048 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:28:14.313014 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e02140e6-3140-4126-83e0-e5b3389bf24a" path="/var/lib/kubelet/pods/e02140e6-3140-4126-83e0-e5b3389bf24a/volumes" Sep 13 20:31:16.010633 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:16.010588 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-67d46498b8-4rm52_59630cc9-8929-409b-9209-6ee02946e421/manager/0.log" Sep 13 20:31:16.153703 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:16.153670 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7fbc5958cd-72xhb"] Sep 13 20:31:16.154054 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:16.154042 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e02140e6-3140-4126-83e0-e5b3389bf24a" containerName="manager" Sep 13 20:31:16.154104 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:16.154056 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e02140e6-3140-4126-83e0-e5b3389bf24a" containerName="manager" Sep 13 20:31:16.154140 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:16.154130 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="e02140e6-3140-4126-83e0-e5b3389bf24a" containerName="manager" Sep 13 20:31:16.157184 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:16.157164 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fbc5958cd-72xhb" Sep 13 20:31:16.160480 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:16.160445 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-xkvj2\"" Sep 13 20:31:16.160630 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:16.160565 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 13 20:31:16.160630 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:16.160591 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 13 20:31:16.160630 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:16.160568 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 13 20:31:16.166006 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:16.165982 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fbc5958cd-72xhb"] Sep 13 20:31:16.253799 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:16.253770 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2jg8\" (UniqueName: \"kubernetes.io/projected/11bf9b18-849d-43e7-8b56-fff7a8ea8beb-kube-api-access-r2jg8\") pod \"maas-controller-7fbc5958cd-72xhb\" (UID: \"11bf9b18-849d-43e7-8b56-fff7a8ea8beb\") " pod="opendatahub/maas-controller-7fbc5958cd-72xhb" Sep 13 20:31:16.253967 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:16.253826 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11bf9b18-849d-43e7-8b56-fff7a8ea8beb-metrics-certs\") pod \"maas-controller-7fbc5958cd-72xhb\" (UID: \"11bf9b18-849d-43e7-8b56-fff7a8ea8beb\") " pod="opendatahub/maas-controller-7fbc5958cd-72xhb" Sep 13 20:31:16.253967 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:16.253893 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/11bf9b18-849d-43e7-8b56-fff7a8ea8beb-webhook-cert\") pod \"maas-controller-7fbc5958cd-72xhb\" (UID: \"11bf9b18-849d-43e7-8b56-fff7a8ea8beb\") " pod="opendatahub/maas-controller-7fbc5958cd-72xhb" Sep 13 20:31:16.354817 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:16.354782 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/11bf9b18-849d-43e7-8b56-fff7a8ea8beb-webhook-cert\") pod \"maas-controller-7fbc5958cd-72xhb\" (UID: \"11bf9b18-849d-43e7-8b56-fff7a8ea8beb\") " pod="opendatahub/maas-controller-7fbc5958cd-72xhb" Sep 13 20:31:16.354984 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:16.354931 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2jg8\" (UniqueName: \"kubernetes.io/projected/11bf9b18-849d-43e7-8b56-fff7a8ea8beb-kube-api-access-r2jg8\") pod \"maas-controller-7fbc5958cd-72xhb\" (UID: \"11bf9b18-849d-43e7-8b56-fff7a8ea8beb\") " pod="opendatahub/maas-controller-7fbc5958cd-72xhb" Sep 13 20:31:16.354984 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:16.354976 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11bf9b18-849d-43e7-8b56-fff7a8ea8beb-metrics-certs\") pod \"maas-controller-7fbc5958cd-72xhb\" (UID: \"11bf9b18-849d-43e7-8b56-fff7a8ea8beb\") " pod="opendatahub/maas-controller-7fbc5958cd-72xhb" Sep 13 20:31:16.357230 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:16.357208 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11bf9b18-849d-43e7-8b56-fff7a8ea8beb-metrics-certs\") pod \"maas-controller-7fbc5958cd-72xhb\" (UID: \"11bf9b18-849d-43e7-8b56-fff7a8ea8beb\") " pod="opendatahub/maas-controller-7fbc5958cd-72xhb" Sep 13 20:31:16.357365 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:16.357342 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/11bf9b18-849d-43e7-8b56-fff7a8ea8beb-webhook-cert\") pod \"maas-controller-7fbc5958cd-72xhb\" (UID: \"11bf9b18-849d-43e7-8b56-fff7a8ea8beb\") " pod="opendatahub/maas-controller-7fbc5958cd-72xhb" Sep 13 20:31:16.365104 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:16.365081 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2jg8\" (UniqueName: \"kubernetes.io/projected/11bf9b18-849d-43e7-8b56-fff7a8ea8beb-kube-api-access-r2jg8\") pod \"maas-controller-7fbc5958cd-72xhb\" (UID: \"11bf9b18-849d-43e7-8b56-fff7a8ea8beb\") " pod="opendatahub/maas-controller-7fbc5958cd-72xhb" Sep 13 20:31:16.469318 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:16.469288 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fbc5958cd-72xhb" Sep 13 20:31:16.741876 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:16.741846 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fbc5958cd-72xhb"] Sep 13 20:31:16.743008 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:31:16.742984 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11bf9b18_849d_43e7_8b56_fff7a8ea8beb.slice/crio-0e870b79c47cd192128576a37c5a19583debef853f1dc950d2a5cfbb36003f96 WatchSource:0}: Error finding container 0e870b79c47cd192128576a37c5a19583debef853f1dc950d2a5cfbb36003f96: Status 404 returned error can't find the container with id 0e870b79c47cd192128576a37c5a19583debef853f1dc950d2a5cfbb36003f96 Sep 13 20:31:16.744288 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:16.744269 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 13 20:31:17.725289 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:17.725206 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fbc5958cd-72xhb" event={"ID":"11bf9b18-849d-43e7-8b56-fff7a8ea8beb","Type":"ContainerStarted","Data":"62005a7d07c8627879dba96c00206449deedd5170f3b5bfdd28a7df0c28cbfdd"} Sep 13 20:31:17.725289 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:17.725263 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fbc5958cd-72xhb" event={"ID":"11bf9b18-849d-43e7-8b56-fff7a8ea8beb","Type":"ContainerStarted","Data":"0e870b79c47cd192128576a37c5a19583debef853f1dc950d2a5cfbb36003f96"} Sep 13 20:31:17.725717 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:17.725297 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7fbc5958cd-72xhb" Sep 13 20:31:17.742286 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:17.742237 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7fbc5958cd-72xhb" podStartSLOduration=1.173126706 podStartE2EDuration="1.742218772s" totalImagesPullingTime="569.092066ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:31:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:31:17.741216795 +0000 UTC m=+1109.917652816" watchObservedRunningTime="2026-09-13 20:31:17.742218772 +0000 UTC m=+1109.918654784" Sep 13 20:31:28.732895 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:28.732866 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7fbc5958cd-72xhb" Sep 13 20:31:29.723139 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:29.723103 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6p8pc/must-gather-s7q82"] Sep 13 20:31:29.727009 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:29.726987 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6p8pc/must-gather-s7q82" Sep 13 20:31:29.729223 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:29.729201 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6p8pc\"/\"default-dockercfg-kbdk7\"" Sep 13 20:31:29.729341 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:29.729250 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6p8pc\"/\"kube-root-ca.crt\"" Sep 13 20:31:29.729418 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:29.729381 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6p8pc\"/\"openshift-service-ca.crt\"" Sep 13 20:31:29.734196 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:29.734158 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6p8pc/must-gather-s7q82"] Sep 13 20:31:29.773730 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:29.773700 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/19144def-0671-49b6-9c39-1dd4c4c76339-must-gather-output\") pod \"must-gather-s7q82\" (UID: \"19144def-0671-49b6-9c39-1dd4c4c76339\") " pod="openshift-must-gather-6p8pc/must-gather-s7q82" Sep 13 20:31:29.773833 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:29.773737 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sh6p\" (UniqueName: \"kubernetes.io/projected/19144def-0671-49b6-9c39-1dd4c4c76339-kube-api-access-8sh6p\") pod \"must-gather-s7q82\" (UID: \"19144def-0671-49b6-9c39-1dd4c4c76339\") " pod="openshift-must-gather-6p8pc/must-gather-s7q82" Sep 13 20:31:29.874839 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:29.874814 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/19144def-0671-49b6-9c39-1dd4c4c76339-must-gather-output\") pod \"must-gather-s7q82\" (UID: \"19144def-0671-49b6-9c39-1dd4c4c76339\") " pod="openshift-must-gather-6p8pc/must-gather-s7q82" Sep 13 20:31:29.874990 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:29.874847 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sh6p\" (UniqueName: \"kubernetes.io/projected/19144def-0671-49b6-9c39-1dd4c4c76339-kube-api-access-8sh6p\") pod \"must-gather-s7q82\" (UID: \"19144def-0671-49b6-9c39-1dd4c4c76339\") " pod="openshift-must-gather-6p8pc/must-gather-s7q82" Sep 13 20:31:29.875149 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:29.875130 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/19144def-0671-49b6-9c39-1dd4c4c76339-must-gather-output\") pod \"must-gather-s7q82\" (UID: \"19144def-0671-49b6-9c39-1dd4c4c76339\") " pod="openshift-must-gather-6p8pc/must-gather-s7q82" Sep 13 20:31:29.881925 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:29.881902 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sh6p\" (UniqueName: \"kubernetes.io/projected/19144def-0671-49b6-9c39-1dd4c4c76339-kube-api-access-8sh6p\") pod \"must-gather-s7q82\" (UID: \"19144def-0671-49b6-9c39-1dd4c4c76339\") " pod="openshift-must-gather-6p8pc/must-gather-s7q82" Sep 13 20:31:30.038569 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:30.038538 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6p8pc/must-gather-s7q82" Sep 13 20:31:30.313529 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:31:30.313474 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod19144def_0671_49b6_9c39_1dd4c4c76339.slice/crio-efa0d79b7f4bafcd264d87bb231e05371e953cdee2fb7bb7496d295987543000 WatchSource:0}: Error finding container efa0d79b7f4bafcd264d87bb231e05371e953cdee2fb7bb7496d295987543000: Status 404 returned error can't find the container with id efa0d79b7f4bafcd264d87bb231e05371e953cdee2fb7bb7496d295987543000 Sep 13 20:31:30.314117 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:30.314089 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6p8pc/must-gather-s7q82"] Sep 13 20:31:30.800881 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:30.800838 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6p8pc/must-gather-s7q82" event={"ID":"19144def-0671-49b6-9c39-1dd4c4c76339","Type":"ContainerStarted","Data":"efa0d79b7f4bafcd264d87bb231e05371e953cdee2fb7bb7496d295987543000"} Sep 13 20:31:31.811890 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:31.811834 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6p8pc/must-gather-s7q82" event={"ID":"19144def-0671-49b6-9c39-1dd4c4c76339","Type":"ContainerStarted","Data":"d89711f7b02982e23f24dae756f728eb66792cc0dc0105b5b8bcd27bce2887a9"} Sep 13 20:31:31.812379 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:31.811904 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6p8pc/must-gather-s7q82" event={"ID":"19144def-0671-49b6-9c39-1dd4c4c76339","Type":"ContainerStarted","Data":"91d104e2a639da477f89e81757121ab0ed345c4eb74b5127129920c814ff0355"} Sep 13 20:31:31.828701 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:31.828637 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6p8pc/must-gather-s7q82" podStartSLOduration=2.16916871 podStartE2EDuration="2.828617847s" totalImagesPullingTime="659.449137ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:31:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:31:31.824950367 +0000 UTC m=+1124.001386390" watchObservedRunningTime="2026-09-13 20:31:31.828617847 +0000 UTC m=+1124.005053873" Sep 13 20:31:32.484627 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:32.484600 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6shg5_855047d0-780f-4f76-a546-e77902ec5399/global-pull-secret-syncer/0.log" Sep 13 20:31:32.642263 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:32.642208 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9zrsq_ca71999d-6b6c-46b6-9c32-819b8b72807c/konnectivity-agent/0.log" Sep 13 20:31:32.685901 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:32.685870 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-99.ec2.internal_7e971f67b78934963c4de7819bc2ede9/haproxy/0.log" Sep 13 20:31:37.554442 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:37.554191 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-vg8n7_3df43dde-d53f-4927-a6a8-68a04b102382/manager/0.log" Sep 13 20:31:37.585763 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:37.585714 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-svnhq_4d37eabc-6cf5-446b-9df3-b5f587e16ee2/manager/0.log" Sep 13 20:31:37.665853 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:37.665821 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-gqdcv_a979679d-9a45-496a-a13a-e31d54844034/manager/0.log" Sep 13 20:31:37.728384 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:37.728348 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-rf847_931353e1-ce74-422f-8946-ae3dc57e17d5/manager/0.log" Sep 13 20:31:38.270087 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:38.270050 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-mp2bf"] Sep 13 20:31:38.281798 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:38.277176 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-mp2bf" Sep 13 20:31:38.281798 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:38.281026 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-k9jgj\"" Sep 13 20:31:38.281798 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:38.281279 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 13 20:31:38.287082 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:38.284898 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-mp2bf"] Sep 13 20:31:38.358547 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:38.358465 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56nnp\" (UniqueName: \"kubernetes.io/projected/cee71a39-7284-4eb8-9b3d-9eb1aec32c21-kube-api-access-56nnp\") pod \"payload-processing-5b54b7fbb7-mp2bf\" (UID: \"cee71a39-7284-4eb8-9b3d-9eb1aec32c21\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-mp2bf" Sep 13 20:31:38.359482 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:38.359448 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cee71a39-7284-4eb8-9b3d-9eb1aec32c21-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-mp2bf\" (UID: \"cee71a39-7284-4eb8-9b3d-9eb1aec32c21\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-mp2bf" Sep 13 20:31:38.477006 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:38.476964 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56nnp\" (UniqueName: \"kubernetes.io/projected/cee71a39-7284-4eb8-9b3d-9eb1aec32c21-kube-api-access-56nnp\") pod \"payload-processing-5b54b7fbb7-mp2bf\" (UID: \"cee71a39-7284-4eb8-9b3d-9eb1aec32c21\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-mp2bf" Sep 13 20:31:38.477787 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:38.477758 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cee71a39-7284-4eb8-9b3d-9eb1aec32c21-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-mp2bf\" (UID: \"cee71a39-7284-4eb8-9b3d-9eb1aec32c21\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-mp2bf" Sep 13 20:31:38.478807 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:38.478732 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cee71a39-7284-4eb8-9b3d-9eb1aec32c21-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-mp2bf\" (UID: \"cee71a39-7284-4eb8-9b3d-9eb1aec32c21\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-mp2bf" Sep 13 20:31:38.495515 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:38.488045 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56nnp\" (UniqueName: \"kubernetes.io/projected/cee71a39-7284-4eb8-9b3d-9eb1aec32c21-kube-api-access-56nnp\") pod \"payload-processing-5b54b7fbb7-mp2bf\" (UID: \"cee71a39-7284-4eb8-9b3d-9eb1aec32c21\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-mp2bf" Sep 13 20:31:38.596057 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:38.595983 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-mp2bf" Sep 13 20:31:38.940538 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:38.932183 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-mp2bf"] Sep 13 20:31:38.952219 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:31:38.952185 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcee71a39_7284_4eb8_9b3d_9eb1aec32c21.slice/crio-86992a7017d53af273ee339ff2f9f10b02df1c3f4d0f5b4b9d70ea94172e16cf WatchSource:0}: Error finding container 86992a7017d53af273ee339ff2f9f10b02df1c3f4d0f5b4b9d70ea94172e16cf: Status 404 returned error can't find the container with id 86992a7017d53af273ee339ff2f9f10b02df1c3f4d0f5b4b9d70ea94172e16cf Sep 13 20:31:39.541433 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:39.541394 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-pg8lq_28166755-2fd2-4b23-ac8a-55a8121e0b00/cluster-monitoring-operator/0.log" Sep 13 20:31:39.697745 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:39.697712 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-89257_727a9bfe-987a-40b3-ba33-53844a2084f3/node-exporter/0.log" Sep 13 20:31:39.723713 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:39.723687 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-89257_727a9bfe-987a-40b3-ba33-53844a2084f3/kube-rbac-proxy/0.log" Sep 13 20:31:39.747425 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:39.747398 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-89257_727a9bfe-987a-40b3-ba33-53844a2084f3/init-textfile/0.log" Sep 13 20:31:39.891650 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:39.890725 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-mp2bf" event={"ID":"cee71a39-7284-4eb8-9b3d-9eb1aec32c21","Type":"ContainerStarted","Data":"4f0e80e0b9c2e6a1298499f341a8798ce364e02ddbbb5781c86377326abcc62c"} Sep 13 20:31:39.891650 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:39.890791 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-mp2bf" event={"ID":"cee71a39-7284-4eb8-9b3d-9eb1aec32c21","Type":"ContainerStarted","Data":"86992a7017d53af273ee339ff2f9f10b02df1c3f4d0f5b4b9d70ea94172e16cf"} Sep 13 20:31:39.891650 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:39.891544 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-mp2bf" Sep 13 20:31:39.952778 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:39.952752 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-dvjks_9555dc90-eaf9-4bb6-8fe2-03372a4e44c5/kube-rbac-proxy-main/0.log" Sep 13 20:31:39.985149 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:39.985119 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-dvjks_9555dc90-eaf9-4bb6-8fe2-03372a4e44c5/kube-rbac-proxy-self/0.log" Sep 13 20:31:40.028316 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:40.028287 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-dvjks_9555dc90-eaf9-4bb6-8fe2-03372a4e44c5/openshift-state-metrics/0.log" Sep 13 20:31:40.278746 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:40.278714 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-qld5g_271622a3-b60c-4537-b553-6fd0aaf16d0e/prometheus-operator/0.log" Sep 13 20:31:40.300159 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:40.300132 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-qld5g_271622a3-b60c-4537-b553-6fd0aaf16d0e/kube-rbac-proxy/0.log" Sep 13 20:31:42.049266 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:42.049233 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-6lr4c_ba047ed2-663d-47be-bd5f-08548fdcdec9/networking-console-plugin/0.log" Sep 13 20:31:42.844200 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:42.844146 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-mp2bf" podStartSLOduration=4.844125204 podStartE2EDuration="4.844125204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-13 20:31:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:31:39.911985499 +0000 UTC m=+1132.088421523" watchObservedRunningTime="2026-09-13 20:31:42.844125204 +0000 UTC m=+1135.020561226" Sep 13 20:31:42.845983 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:42.845959 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6p8pc/perf-node-gather-daemonset-dq7dp"] Sep 13 20:31:42.852066 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:42.852039 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6p8pc/perf-node-gather-daemonset-dq7dp" Sep 13 20:31:42.858888 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:42.858856 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6p8pc/perf-node-gather-daemonset-dq7dp"] Sep 13 20:31:42.925578 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:42.925540 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08-lib-modules\") pod \"perf-node-gather-daemonset-dq7dp\" (UID: \"7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08\") " pod="openshift-must-gather-6p8pc/perf-node-gather-daemonset-dq7dp" Sep 13 20:31:42.925767 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:42.925693 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08-podres\") pod \"perf-node-gather-daemonset-dq7dp\" (UID: \"7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08\") " pod="openshift-must-gather-6p8pc/perf-node-gather-daemonset-dq7dp" Sep 13 20:31:42.925823 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:42.925801 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08-sys\") pod \"perf-node-gather-daemonset-dq7dp\" (UID: \"7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08\") " pod="openshift-must-gather-6p8pc/perf-node-gather-daemonset-dq7dp" Sep 13 20:31:42.925860 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:42.925835 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncxpm\" (UniqueName: \"kubernetes.io/projected/7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08-kube-api-access-ncxpm\") pod \"perf-node-gather-daemonset-dq7dp\" (UID: \"7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08\") " pod="openshift-must-gather-6p8pc/perf-node-gather-daemonset-dq7dp" Sep 13 20:31:42.925899 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:42.925879 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08-proc\") pod \"perf-node-gather-daemonset-dq7dp\" (UID: \"7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08\") " pod="openshift-must-gather-6p8pc/perf-node-gather-daemonset-dq7dp" Sep 13 20:31:43.027174 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:43.027130 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08-podres\") pod \"perf-node-gather-daemonset-dq7dp\" (UID: \"7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08\") " pod="openshift-must-gather-6p8pc/perf-node-gather-daemonset-dq7dp" Sep 13 20:31:43.027337 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:43.027231 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08-sys\") pod \"perf-node-gather-daemonset-dq7dp\" (UID: \"7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08\") " pod="openshift-must-gather-6p8pc/perf-node-gather-daemonset-dq7dp" Sep 13 20:31:43.027337 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:43.027262 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncxpm\" (UniqueName: \"kubernetes.io/projected/7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08-kube-api-access-ncxpm\") pod \"perf-node-gather-daemonset-dq7dp\" (UID: \"7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08\") " pod="openshift-must-gather-6p8pc/perf-node-gather-daemonset-dq7dp" Sep 13 20:31:43.027337 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:43.027324 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08-proc\") pod \"perf-node-gather-daemonset-dq7dp\" (UID: \"7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08\") " pod="openshift-must-gather-6p8pc/perf-node-gather-daemonset-dq7dp" Sep 13 20:31:43.027521 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:43.027332 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08-podres\") pod \"perf-node-gather-daemonset-dq7dp\" (UID: \"7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08\") " pod="openshift-must-gather-6p8pc/perf-node-gather-daemonset-dq7dp" Sep 13 20:31:43.027521 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:43.027352 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08-lib-modules\") pod \"perf-node-gather-daemonset-dq7dp\" (UID: \"7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08\") " pod="openshift-must-gather-6p8pc/perf-node-gather-daemonset-dq7dp" Sep 13 20:31:43.027521 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:43.027367 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08-sys\") pod \"perf-node-gather-daemonset-dq7dp\" (UID: \"7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08\") " pod="openshift-must-gather-6p8pc/perf-node-gather-daemonset-dq7dp" Sep 13 20:31:43.027521 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:43.027415 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08-proc\") pod \"perf-node-gather-daemonset-dq7dp\" (UID: \"7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08\") " pod="openshift-must-gather-6p8pc/perf-node-gather-daemonset-dq7dp" Sep 13 20:31:43.027729 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:43.027575 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08-lib-modules\") pod \"perf-node-gather-daemonset-dq7dp\" (UID: \"7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08\") " pod="openshift-must-gather-6p8pc/perf-node-gather-daemonset-dq7dp" Sep 13 20:31:43.035328 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:43.035304 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncxpm\" (UniqueName: \"kubernetes.io/projected/7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08-kube-api-access-ncxpm\") pod \"perf-node-gather-daemonset-dq7dp\" (UID: \"7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08\") " pod="openshift-must-gather-6p8pc/perf-node-gather-daemonset-dq7dp" Sep 13 20:31:43.165349 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:43.165261 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6p8pc/perf-node-gather-daemonset-dq7dp" Sep 13 20:31:43.228934 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:43.228890 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6f897b78bf-cgxtr_2270b1ff-883b-4ec7-b459-4e628c952d16/console/0.log" Sep 13 20:31:43.453171 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:43.453143 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6p8pc/perf-node-gather-daemonset-dq7dp"] Sep 13 20:31:43.454677 ip-10-0-134-99 kubenswrapper[2557]: W0913 20:31:43.454641 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7aebfa1e_4c2a_4dba_be28_fbdb1ad56a08.slice/crio-262536452a507b250405d5a352189ed9ef18f63c009847a2bf7d8fc81fcbefb5 WatchSource:0}: Error finding container 262536452a507b250405d5a352189ed9ef18f63c009847a2bf7d8fc81fcbefb5: Status 404 returned error can't find the container with id 262536452a507b250405d5a352189ed9ef18f63c009847a2bf7d8fc81fcbefb5 Sep 13 20:31:43.893633 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:43.893587 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-tjppt_7c028e5d-3eb9-4423-9201-4aeaefc67d49/volume-data-source-validator/0.log" Sep 13 20:31:43.932202 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:43.932047 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6p8pc/perf-node-gather-daemonset-dq7dp" event={"ID":"7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08","Type":"ContainerStarted","Data":"c9c6cfeb5dbf9667ff5dc60e10579d0bbfa6f2340f806147575916c2de6d2b89"} Sep 13 20:31:43.932202 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:43.932117 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6p8pc/perf-node-gather-daemonset-dq7dp" event={"ID":"7aebfa1e-4c2a-4dba-be28-fbdb1ad56a08","Type":"ContainerStarted","Data":"262536452a507b250405d5a352189ed9ef18f63c009847a2bf7d8fc81fcbefb5"} Sep 13 20:31:43.932451 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:43.932265 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6p8pc/perf-node-gather-daemonset-dq7dp" Sep 13 20:31:43.950310 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:43.950254 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6p8pc/perf-node-gather-daemonset-dq7dp" podStartSLOduration=1.9502336740000001 podStartE2EDuration="1.950233674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-13 20:31:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-13 20:31:43.945324549 +0000 UTC m=+1136.121760584" watchObservedRunningTime="2026-09-13 20:31:43.950233674 +0000 UTC m=+1136.126669700" Sep 13 20:31:44.965367 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:44.965326 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fw949_a99d1dc1-43a2-4ada-99dd-a4a05c5faf68/dns/0.log" Sep 13 20:31:44.984646 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:44.984607 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fw949_a99d1dc1-43a2-4ada-99dd-a4a05c5faf68/kube-rbac-proxy/0.log" Sep 13 20:31:45.077419 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:45.077387 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mgvpg_e80af952-0069-4b10-9b75-2bbef4b3a126/dns-node-resolver/0.log" Sep 13 20:31:45.652293 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:45.652262 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xdjrr_2a742e26-18ce-4bcb-8735-a69419ffb81a/node-ca/0.log" Sep 13 20:31:46.607194 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:46.607166 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54976cff48-8rdkx_0720a4d9-96d6-48eb-839a-ec1f25d7bc56/kube-auth-proxy/0.log" Sep 13 20:31:46.671675 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:46.671639 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-mp2bf_cee71a39-7284-4eb8-9b3d-9eb1aec32c21/payload-processing/0.log" Sep 13 20:31:46.693046 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:46.693018 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-655c4b696-bgtsn_ff4cd6d7-3b88-4122-9d4e-d1560d389a70/router/0.log" Sep 13 20:31:47.311410 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:47.311386 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-k8c5m_5b70f512-38d7-4546-be8f-c40233eabd8f/serve-healthcheck-canary/0.log" Sep 13 20:31:47.879517 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:47.879475 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lnffs_260a45fa-ca24-4f0f-8248-34f44c75e581/kube-rbac-proxy/0.log" Sep 13 20:31:47.897339 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:47.897313 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lnffs_260a45fa-ca24-4f0f-8248-34f44c75e581/exporter/0.log" Sep 13 20:31:47.916432 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:47.916400 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lnffs_260a45fa-ca24-4f0f-8248-34f44c75e581/extractor/0.log" Sep 13 20:31:49.955219 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:49.955183 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6p8pc/perf-node-gather-daemonset-dq7dp" Sep 13 20:31:50.050878 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:50.050823 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-67d46498b8-4rm52_59630cc9-8929-409b-9209-6ee02946e421/manager/0.log" Sep 13 20:31:50.074390 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:50.074357 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-q2h9r_39c26102-f478-43d5-aaf2-9d1cbe372f8d/manager/0.log" Sep 13 20:31:50.112249 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:50.112209 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-n7zns_b047f42d-3486-452a-8daf-9f367be63881/manager/0.log" Sep 13 20:31:50.134254 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:50.134222 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7fbc5958cd-72xhb_11bf9b18-849d-43e7-8b56-fff7a8ea8beb/manager/0.log" Sep 13 20:31:50.159525 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:50.159479 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-pjpgr_0e74cb73-9d45-47fc-9ce4-6b88d88fbdae/server/0.log" Sep 13 20:31:50.285511 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:50.285462 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84f9f8b966-n6gq9_ca47fa53-4db9-4770-9c33-101323783338/manager/0.log" Sep 13 20:31:50.303343 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:50.303319 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84f9f8b966-n6gq9_ca47fa53-4db9-4770-9c33-101323783338/copy-manifests/0.log" Sep 13 20:31:51.589263 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:51.589231 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-8rbp7_e2d0c129-2513-4406-af34-77b1769be0d0/openshift-lws-operator/0.log" Sep 13 20:31:51.911703 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:51.911672 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-mp2bf" Sep 13 20:31:53.512187 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:53.512150 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54976cff48-8rdkx_0720a4d9-96d6-48eb-839a-ec1f25d7bc56/kube-auth-proxy/0.log" Sep 13 20:31:53.561599 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:53.561540 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-mp2bf_cee71a39-7284-4eb8-9b3d-9eb1aec32c21/payload-processing/0.log" Sep 13 20:31:53.580109 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:53.580079 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-655c4b696-bgtsn_ff4cd6d7-3b88-4122-9d4e-d1560d389a70/router/0.log" Sep 13 20:31:54.819552 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:54.819520 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54976cff48-8rdkx_0720a4d9-96d6-48eb-839a-ec1f25d7bc56/kube-auth-proxy/0.log" Sep 13 20:31:54.872039 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:54.871932 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-mp2bf_cee71a39-7284-4eb8-9b3d-9eb1aec32c21/payload-processing/0.log" Sep 13 20:31:54.887997 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:54.887968 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-655c4b696-bgtsn_ff4cd6d7-3b88-4122-9d4e-d1560d389a70/router/0.log" Sep 13 20:31:56.756775 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:56.756749 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-p72zr_ec2b9095-1ce0-4b18-8ab0-ef6491e838cb/migrator/0.log" Sep 13 20:31:56.776534 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:56.776491 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-p72zr_ec2b9095-1ce0-4b18-8ab0-ef6491e838cb/graceful-termination/0.log" Sep 13 20:31:57.128672 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:57.128638 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-g7j4k_ee7be3e3-c579-4378-9a12-94b54e80d33e/kube-storage-version-migrator-operator/1.log" Sep 13 20:31:57.129421 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:57.129383 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-g7j4k_ee7be3e3-c579-4378-9a12-94b54e80d33e/kube-storage-version-migrator-operator/0.log" Sep 13 20:31:58.370376 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:58.370350 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vkdmh_f071dc13-4600-49f7-9396-4ee02d4a2c6f/kube-multus-additional-cni-plugins/0.log" Sep 13 20:31:58.390322 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:58.390275 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vkdmh_f071dc13-4600-49f7-9396-4ee02d4a2c6f/egress-router-binary-copy/0.log" Sep 13 20:31:58.409213 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:58.409179 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vkdmh_f071dc13-4600-49f7-9396-4ee02d4a2c6f/cni-plugins/0.log" Sep 13 20:31:58.426690 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:58.426646 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vkdmh_f071dc13-4600-49f7-9396-4ee02d4a2c6f/bond-cni-plugin/0.log" Sep 13 20:31:58.444482 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:58.444451 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vkdmh_f071dc13-4600-49f7-9396-4ee02d4a2c6f/routeoverride-cni/0.log" Sep 13 20:31:58.462325 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:58.462261 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vkdmh_f071dc13-4600-49f7-9396-4ee02d4a2c6f/whereabouts-cni-bincopy/0.log" Sep 13 20:31:58.480337 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:58.480307 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vkdmh_f071dc13-4600-49f7-9396-4ee02d4a2c6f/whereabouts-cni/0.log" Sep 13 20:31:58.551950 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:58.551922 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-w4brn_5b2436ba-54ce-4ba5-9ad8-48613ecd34aa/kube-multus/0.log" Sep 13 20:31:58.612231 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:58.612196 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-95zg2_32f68c96-97d3-4904-a675-45f93df87572/network-metrics-daemon/0.log" Sep 13 20:31:58.628381 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:58.628301 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-95zg2_32f68c96-97d3-4904-a675-45f93df87572/kube-rbac-proxy/0.log" Sep 13 20:31:59.428074 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:59.428026 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5h748_f997e865-7a2e-4898-a3ed-056bf8a4124d/ovn-controller/0.log" Sep 13 20:31:59.442064 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:59.442037 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5h748_f997e865-7a2e-4898-a3ed-056bf8a4124d/ovn-acl-logging/0.log" Sep 13 20:31:59.465986 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:59.465941 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5h748_f997e865-7a2e-4898-a3ed-056bf8a4124d/kube-rbac-proxy-node/0.log" Sep 13 20:31:59.484399 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:59.484373 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5h748_f997e865-7a2e-4898-a3ed-056bf8a4124d/kube-rbac-proxy-ovn-metrics/0.log" Sep 13 20:31:59.499233 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:59.499209 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5h748_f997e865-7a2e-4898-a3ed-056bf8a4124d/northd/0.log" Sep 13 20:31:59.519586 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:59.519555 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5h748_f997e865-7a2e-4898-a3ed-056bf8a4124d/nbdb/0.log" Sep 13 20:31:59.537431 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:59.537406 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5h748_f997e865-7a2e-4898-a3ed-056bf8a4124d/sbdb/0.log" Sep 13 20:31:59.580172 ip-10-0-134-99 kubenswrapper[2557]: I0913 20:31:59.580143 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5h748_f997e865-7a2e-4898-a3ed-056bf8a4124d/ovnkube-controller/0.log"