Sep 11 18:44:38.424800 ip-10-0-134-192 systemd[1]: Starting Kubernetes Kubelet... Sep 11 18:44:38.807096 ip-10-0-134-192 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 11 18:44:38.807096 ip-10-0-134-192 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 11 18:44:38.807096 ip-10-0-134-192 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 11 18:44:38.807096 ip-10-0-134-192 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 11 18:44:38.807096 ip-10-0-134-192 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 11 18:44:38.807744 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.807121 2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 11 18:44:38.810793 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810774 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 18:44:38.810793 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810790 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 18:44:38.810793 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810794 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 18:44:38.810793 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810797 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 18:44:38.810943 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810801 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 18:44:38.810943 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810803 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 18:44:38.810943 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810806 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 18:44:38.810943 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810809 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 18:44:38.810943 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810812 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 18:44:38.810943 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810816 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 18:44:38.810943 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810819 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 18:44:38.810943 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810821 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 18:44:38.810943 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810824 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 18:44:38.810943 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810827 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 18:44:38.810943 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810829 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 18:44:38.810943 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810832 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 18:44:38.810943 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810835 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 18:44:38.810943 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810837 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 18:44:38.810943 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810842 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 18:44:38.810943 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810846 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 18:44:38.810943 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810849 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 18:44:38.810943 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810852 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 18:44:38.810943 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810855 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 18:44:38.810943 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810857 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 18:44:38.811430 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810860 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 18:44:38.811430 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810863 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 18:44:38.811430 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810865 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 18:44:38.811430 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810868 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 18:44:38.811430 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810871 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 18:44:38.811430 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810873 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 18:44:38.811430 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810876 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 18:44:38.811430 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810878 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 11 18:44:38.811430 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810881 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 18:44:38.811430 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810884 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 18:44:38.811430 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810887 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 18:44:38.811430 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810889 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 18:44:38.811430 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810892 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 18:44:38.811430 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810895 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 18:44:38.811430 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810898 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 18:44:38.811430 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810900 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 18:44:38.811430 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810903 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 18:44:38.811430 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810906 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 18:44:38.811430 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810910 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 18:44:38.811430 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810913 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 18:44:38.811931 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810915 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 18:44:38.811931 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810918 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 18:44:38.811931 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810920 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 18:44:38.811931 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810923 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 18:44:38.811931 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810925 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 18:44:38.811931 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810928 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 18:44:38.811931 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810931 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 18:44:38.811931 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810933 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 18:44:38.811931 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810936 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 18:44:38.811931 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810938 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 18:44:38.811931 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810941 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 18:44:38.811931 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810943 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 18:44:38.811931 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810946 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 18:44:38.811931 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810949 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 18:44:38.811931 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810951 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 18:44:38.811931 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810954 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 18:44:38.811931 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810958 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 18:44:38.811931 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810961 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 18:44:38.811931 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810963 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 18:44:38.811931 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810966 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 18:44:38.812419 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810969 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 18:44:38.812419 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810971 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 18:44:38.812419 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810973 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 18:44:38.812419 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810976 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 18:44:38.812419 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810981 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 18:44:38.812419 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810987 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 18:44:38.812419 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810990 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 18:44:38.812419 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810993 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 18:44:38.812419 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810996 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 18:44:38.812419 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.810999 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 18:44:38.812419 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811001 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 18:44:38.812419 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811004 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 18:44:38.812419 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811007 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 18:44:38.812419 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811010 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 18:44:38.812419 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811014 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 18:44:38.812419 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811017 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 18:44:38.812419 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811020 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 18:44:38.812419 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811023 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 18:44:38.812419 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811025 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 18:44:38.812913 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811028 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 18:44:38.812913 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811030 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 18:44:38.812913 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811033 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 18:44:38.812913 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811035 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 18:44:38.812913 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811038 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 18:44:38.812913 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811040 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 18:44:38.812913 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811043 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 18:44:38.812913 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811045 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 18:44:38.812913 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811048 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 18:44:38.812913 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811050 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 18:44:38.812913 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811053 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 18:44:38.812913 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811056 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 18:44:38.812913 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811059 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 18:44:38.812913 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811061 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 18:44:38.812913 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811064 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 18:44:38.812913 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811066 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 18:44:38.812913 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811786 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 18:44:38.812913 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811793 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 18:44:38.812913 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811797 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 18:44:38.812913 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811800 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 18:44:38.813404 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811802 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 18:44:38.813404 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811805 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 18:44:38.813404 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811808 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 18:44:38.813404 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811811 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 18:44:38.813404 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811813 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 18:44:38.813404 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811816 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 18:44:38.813404 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811819 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 18:44:38.813404 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811821 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 18:44:38.813404 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811824 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 18:44:38.813404 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811827 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 18:44:38.813404 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811830 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 18:44:38.813404 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811832 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 18:44:38.813404 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811835 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 18:44:38.813404 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811838 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 18:44:38.813404 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811842 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 18:44:38.813404 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811844 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 18:44:38.813404 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811847 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 18:44:38.813404 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811850 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 18:44:38.813404 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811852 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 18:44:38.813404 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811855 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 18:44:38.814119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811857 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 18:44:38.814119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811860 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 18:44:38.814119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811863 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 18:44:38.814119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811866 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 18:44:38.814119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811869 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 18:44:38.814119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811872 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 18:44:38.814119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811875 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 18:44:38.814119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811877 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 18:44:38.814119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811880 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 18:44:38.814119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811883 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 18:44:38.814119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811886 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 18:44:38.814119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811889 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 18:44:38.814119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811891 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 18:44:38.814119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811894 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 18:44:38.814119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811897 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 18:44:38.814119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811899 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 18:44:38.814119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811902 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 18:44:38.814119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811905 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 18:44:38.814119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811909 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 18:44:38.814119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811911 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 18:44:38.814920 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811914 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 18:44:38.814920 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811917 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 18:44:38.814920 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811920 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 18:44:38.814920 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811922 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 18:44:38.814920 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811925 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 18:44:38.814920 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811927 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 18:44:38.814920 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811931 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 18:44:38.814920 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811936 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 18:44:38.814920 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811939 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 18:44:38.814920 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811942 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 18:44:38.814920 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811945 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 11 18:44:38.814920 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811948 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 18:44:38.814920 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811950 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 18:44:38.814920 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811953 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 18:44:38.814920 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811956 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 18:44:38.814920 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811959 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 18:44:38.814920 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811963 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 18:44:38.814920 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811966 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 18:44:38.814920 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811968 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 18:44:38.814920 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811971 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 18:44:38.815484 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811974 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 18:44:38.815484 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811976 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 18:44:38.815484 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811979 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 18:44:38.815484 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811982 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 18:44:38.815484 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811984 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 18:44:38.815484 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811987 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 18:44:38.815484 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811989 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 18:44:38.815484 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811993 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 18:44:38.815484 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.811996 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 18:44:38.815484 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812000 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 18:44:38.815484 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812003 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 18:44:38.815484 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812005 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 18:44:38.815484 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812008 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 18:44:38.815484 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812011 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 18:44:38.815484 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812014 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 18:44:38.815484 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812016 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 18:44:38.815484 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812019 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 18:44:38.815484 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812021 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 18:44:38.815484 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812024 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 18:44:38.816119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812027 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 18:44:38.816119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812029 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 18:44:38.816119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812032 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 18:44:38.816119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812034 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 18:44:38.816119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812037 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 18:44:38.816119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812040 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 18:44:38.816119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812044 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 18:44:38.816119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812047 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 18:44:38.816119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812049 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 18:44:38.816119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812052 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 18:44:38.816119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812058 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 18:44:38.816119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812060 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 18:44:38.816119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812063 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 18:44:38.816119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812065 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 18:44:38.816119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812068 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 18:44:38.816119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812071 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 18:44:38.816119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812428 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 18:44:38.816119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812432 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 18:44:38.816119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812435 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 18:44:38.816614 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812438 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 18:44:38.816614 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812441 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 18:44:38.816614 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812444 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 18:44:38.816614 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812447 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 18:44:38.816614 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812450 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 18:44:38.816614 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812452 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 18:44:38.816614 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812455 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 18:44:38.816614 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812457 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 18:44:38.816614 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812460 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 18:44:38.816614 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812463 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 18:44:38.816614 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812465 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 18:44:38.816614 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812468 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 18:44:38.816614 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812471 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 18:44:38.816614 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812473 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 18:44:38.816614 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812475 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 18:44:38.816614 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812478 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 18:44:38.816614 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812481 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 18:44:38.816614 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812483 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 18:44:38.816614 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812486 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 18:44:38.816614 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812488 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 18:44:38.817119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812491 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 18:44:38.817119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812493 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 18:44:38.817119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812495 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 18:44:38.817119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812498 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 18:44:38.817119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812517 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 18:44:38.817119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812521 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 18:44:38.817119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812525 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 18:44:38.817119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812533 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 18:44:38.817119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812538 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 18:44:38.817119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812541 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 11 18:44:38.817119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812543 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 18:44:38.817119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812546 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 18:44:38.817119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812549 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 18:44:38.817119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812551 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 18:44:38.817119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812554 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 18:44:38.817119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812558 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 18:44:38.817119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812561 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 18:44:38.817119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812564 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 18:44:38.817119 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812567 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 18:44:38.817596 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812570 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 18:44:38.817596 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812573 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 18:44:38.817596 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812576 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 18:44:38.817596 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812578 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 18:44:38.817596 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812581 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 18:44:38.817596 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812583 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 18:44:38.817596 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812586 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 18:44:38.817596 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812588 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 18:44:38.817596 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812591 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 18:44:38.817596 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812593 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 18:44:38.817596 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812596 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 18:44:38.817596 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812599 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 18:44:38.817596 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812601 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 18:44:38.817596 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812605 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 18:44:38.817596 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812607 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 18:44:38.817596 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812609 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 18:44:38.817596 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812613 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 18:44:38.817596 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812616 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 18:44:38.817596 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812619 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 18:44:38.817596 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812622 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 18:44:38.818109 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812627 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 18:44:38.818109 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812630 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 18:44:38.818109 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812634 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 18:44:38.818109 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812637 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 18:44:38.818109 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812639 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 18:44:38.818109 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812642 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 18:44:38.818109 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812645 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 18:44:38.818109 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812647 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 18:44:38.818109 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812650 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 18:44:38.818109 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812653 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 18:44:38.818109 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812655 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 18:44:38.818109 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812658 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 18:44:38.818109 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812660 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 18:44:38.818109 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812663 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 18:44:38.818109 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812666 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 18:44:38.818109 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812668 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 18:44:38.818109 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812670 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 18:44:38.818109 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812673 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 18:44:38.818109 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812676 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 18:44:38.818109 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812678 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 18:44:38.818601 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812681 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 18:44:38.818601 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812683 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 18:44:38.818601 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812687 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 18:44:38.818601 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812689 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 18:44:38.818601 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812692 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 18:44:38.818601 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812694 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 18:44:38.818601 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812697 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 18:44:38.818601 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812700 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 18:44:38.818601 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812702 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 18:44:38.818601 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812705 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 18:44:38.818601 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812707 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 18:44:38.818601 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812710 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 18:44:38.818601 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812716 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 18:44:38.818601 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812719 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 18:44:38.818601 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812721 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 18:44:38.818601 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812724 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 18:44:38.818601 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.812727 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 18:44:38.818601 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812803 2563 flags.go:64] FLAG: --address="0.0.0.0" Sep 11 18:44:38.818601 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812844 2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 11 18:44:38.818601 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812852 2563 flags.go:64] FLAG: --anonymous-auth="true" Sep 11 18:44:38.819213 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812856 2563 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 11 18:44:38.819213 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812861 2563 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 11 18:44:38.819213 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812864 2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 11 18:44:38.819213 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812869 2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 11 18:44:38.819213 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812873 2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 11 18:44:38.819213 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812876 2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 11 18:44:38.819213 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812880 2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 11 18:44:38.819213 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812883 2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 11 18:44:38.819213 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812886 2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 11 18:44:38.819213 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812889 2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 11 18:44:38.819213 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812892 2563 flags.go:64] FLAG: --cgroup-root="" Sep 11 18:44:38.819213 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812895 2563 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 11 18:44:38.819213 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812901 2563 flags.go:64] FLAG: --client-ca-file="" Sep 11 18:44:38.819213 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812904 2563 flags.go:64] FLAG: --cloud-provider="external" Sep 11 18:44:38.819213 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812907 2563 flags.go:64] FLAG: --cluster-dns="[]" Sep 11 18:44:38.819213 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812911 2563 flags.go:64] FLAG: --cluster-domain="" Sep 11 18:44:38.819213 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812914 2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 11 18:44:38.819213 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812917 2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 11 18:44:38.819213 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812920 2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 11 18:44:38.819213 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812924 2563 flags.go:64] FLAG: --container-log-max-files="5" Sep 11 18:44:38.819213 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812928 2563 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 11 18:44:38.819213 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812931 2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 11 18:44:38.819213 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812934 2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 11 18:44:38.819785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812938 2563 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 11 18:44:38.819785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812943 2563 flags.go:64] FLAG: --contention-profiling="false" Sep 11 18:44:38.819785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812946 2563 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 11 18:44:38.819785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812949 2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 11 18:44:38.819785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812952 2563 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 11 18:44:38.819785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812955 2563 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 11 18:44:38.819785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812959 2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 11 18:44:38.819785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812962 2563 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 11 18:44:38.819785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812965 2563 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 11 18:44:38.819785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812968 2563 flags.go:64] FLAG: --enable-load-reader="false" Sep 11 18:44:38.819785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812971 2563 flags.go:64] FLAG: --enable-server="true" Sep 11 18:44:38.819785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812975 2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 11 18:44:38.819785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812979 2563 flags.go:64] FLAG: --event-burst="100" Sep 11 18:44:38.819785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812983 2563 flags.go:64] FLAG: --event-qps="50" Sep 11 18:44:38.819785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812986 2563 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 11 18:44:38.819785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812989 2563 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 11 18:44:38.819785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812992 2563 flags.go:64] FLAG: --eviction-hard="" Sep 11 18:44:38.819785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812996 2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 11 18:44:38.819785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.812999 2563 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 11 18:44:38.819785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813002 2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 11 18:44:38.819785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813005 2563 flags.go:64] FLAG: --eviction-soft="" Sep 11 18:44:38.819785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813009 2563 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 11 18:44:38.819785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813012 2563 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 11 18:44:38.819785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813015 2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 11 18:44:38.819785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813018 2563 flags.go:64] FLAG: --experimental-mounter-path="" Sep 11 18:44:38.820408 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813021 2563 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 11 18:44:38.820408 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813024 2563 flags.go:64] FLAG: --fail-swap-on="true" Sep 11 18:44:38.820408 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813027 2563 flags.go:64] FLAG: --feature-gates="" Sep 11 18:44:38.820408 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813031 2563 flags.go:64] FLAG: --file-check-frequency="20s" Sep 11 18:44:38.820408 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813034 2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 11 18:44:38.820408 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813037 2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 11 18:44:38.820408 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813040 2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 11 18:44:38.820408 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813043 2563 flags.go:64] FLAG: --healthz-port="10248" Sep 11 18:44:38.820408 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813048 2563 flags.go:64] FLAG: --help="false" Sep 11 18:44:38.820408 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813051 2563 flags.go:64] FLAG: --hostname-override="ip-10-0-134-192.ec2.internal" Sep 11 18:44:38.820408 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813054 2563 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 11 18:44:38.820408 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813057 2563 flags.go:64] FLAG: --http-check-frequency="20s" Sep 11 18:44:38.820408 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813060 2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 11 18:44:38.820408 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813064 2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 11 18:44:38.820408 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813067 2563 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 11 18:44:38.820408 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813070 2563 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 11 18:44:38.820408 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813073 2563 flags.go:64] FLAG: --image-service-endpoint="" Sep 11 18:44:38.820408 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813076 2563 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 11 18:44:38.820408 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813078 2563 flags.go:64] FLAG: --kube-api-burst="100" Sep 11 18:44:38.820408 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813081 2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 11 18:44:38.820408 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813084 2563 flags.go:64] FLAG: --kube-api-qps="50" Sep 11 18:44:38.820408 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813087 2563 flags.go:64] FLAG: --kube-reserved="" Sep 11 18:44:38.820408 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813090 2563 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 11 18:44:38.820408 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813093 2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 11 18:44:38.821036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813096 2563 flags.go:64] FLAG: --kubelet-cgroups="" Sep 11 18:44:38.821036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813099 2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 11 18:44:38.821036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813102 2563 flags.go:64] FLAG: --lock-file="" Sep 11 18:44:38.821036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813105 2563 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 11 18:44:38.821036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813109 2563 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 11 18:44:38.821036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813112 2563 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 11 18:44:38.821036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813118 2563 flags.go:64] FLAG: --log-json-split-stream="false" Sep 11 18:44:38.821036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813120 2563 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 11 18:44:38.821036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813123 2563 flags.go:64] FLAG: --log-text-split-stream="false" Sep 11 18:44:38.821036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813126 2563 flags.go:64] FLAG: --logging-format="text" Sep 11 18:44:38.821036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813129 2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 11 18:44:38.821036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813132 2563 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 11 18:44:38.821036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813135 2563 flags.go:64] FLAG: --manifest-url="" Sep 11 18:44:38.821036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813138 2563 flags.go:64] FLAG: --manifest-url-header="" Sep 11 18:44:38.821036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813142 2563 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 11 18:44:38.821036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813145 2563 flags.go:64] FLAG: --max-open-files="1000000" Sep 11 18:44:38.821036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813150 2563 flags.go:64] FLAG: --max-pods="110" Sep 11 18:44:38.821036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813154 2563 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 11 18:44:38.821036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813157 2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 11 18:44:38.821036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813160 2563 flags.go:64] FLAG: --memory-manager-policy="None" Sep 11 18:44:38.821036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813163 2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 11 18:44:38.821036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813166 2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 11 18:44:38.821036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813169 2563 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 11 18:44:38.821036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813172 2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 11 18:44:38.821036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813206 2563 flags.go:64] FLAG: --node-status-max-images="50" Sep 11 18:44:38.821659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813210 2563 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 11 18:44:38.821659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813213 2563 flags.go:64] FLAG: --oom-score-adj="-999" Sep 11 18:44:38.821659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813216 2563 flags.go:64] FLAG: --pod-cidr="" Sep 11 18:44:38.821659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813219 2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10" Sep 11 18:44:38.821659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813225 2563 flags.go:64] FLAG: --pod-manifest-path="" Sep 11 18:44:38.821659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813228 2563 flags.go:64] FLAG: --pod-max-pids="-1" Sep 11 18:44:38.821659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813231 2563 flags.go:64] FLAG: --pods-per-core="0" Sep 11 18:44:38.821659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813234 2563 flags.go:64] FLAG: --port="10250" Sep 11 18:44:38.821659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813237 2563 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 11 18:44:38.821659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813240 2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b479042eab5b4b2d" Sep 11 18:44:38.821659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813243 2563 flags.go:64] FLAG: --qos-reserved="" Sep 11 18:44:38.821659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813248 2563 flags.go:64] FLAG: --read-only-port="10255" Sep 11 18:44:38.821659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813251 2563 flags.go:64] FLAG: --register-node="true" Sep 11 18:44:38.821659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813254 2563 flags.go:64] FLAG: --register-with-taints="" Sep 11 18:44:38.821659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813258 2563 flags.go:64] FLAG: --registry-burst="10" Sep 11 18:44:38.821659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813261 2563 flags.go:64] FLAG: --registry-qps="5" Sep 11 18:44:38.821659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813263 2563 flags.go:64] FLAG: --reserved-cpus="" Sep 11 18:44:38.821659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813266 2563 flags.go:64] FLAG: --reserved-memory="" Sep 11 18:44:38.821659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813270 2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 11 18:44:38.821659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813273 2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 11 18:44:38.821659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813276 2563 flags.go:64] FLAG: --rotate-certificates="false" Sep 11 18:44:38.821659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813279 2563 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 11 18:44:38.821659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813282 2563 flags.go:64] FLAG: --runonce="false" Sep 11 18:44:38.821659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813284 2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 11 18:44:38.821659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813289 2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 11 18:44:38.822276 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813293 2563 flags.go:64] FLAG: --seccomp-default="false" Sep 11 18:44:38.822276 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813296 2563 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 11 18:44:38.822276 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813299 2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 11 18:44:38.822276 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813302 2563 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 11 18:44:38.822276 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813305 2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 11 18:44:38.822276 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813308 2563 flags.go:64] FLAG: --storage-driver-password="root" Sep 11 18:44:38.822276 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813311 2563 flags.go:64] FLAG: --storage-driver-secure="false" Sep 11 18:44:38.822276 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813314 2563 flags.go:64] FLAG: --storage-driver-table="stats" Sep 11 18:44:38.822276 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813317 2563 flags.go:64] FLAG: --storage-driver-user="root" Sep 11 18:44:38.822276 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813320 2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 11 18:44:38.822276 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813323 2563 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 11 18:44:38.822276 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813325 2563 flags.go:64] FLAG: --system-cgroups="" Sep 11 18:44:38.822276 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813328 2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 11 18:44:38.822276 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813333 2563 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 11 18:44:38.822276 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813336 2563 flags.go:64] FLAG: --tls-cert-file="" Sep 11 18:44:38.822276 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813339 2563 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 11 18:44:38.822276 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813343 2563 flags.go:64] FLAG: --tls-min-version="" Sep 11 18:44:38.822276 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813346 2563 flags.go:64] FLAG: --tls-private-key-file="" Sep 11 18:44:38.822276 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813350 2563 flags.go:64] FLAG: --topology-manager-policy="none" Sep 11 18:44:38.822276 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813353 2563 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 11 18:44:38.822276 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813356 2563 flags.go:64] FLAG: --topology-manager-scope="container" Sep 11 18:44:38.822276 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813359 2563 flags.go:64] FLAG: --v="2" Sep 11 18:44:38.822276 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813363 2563 flags.go:64] FLAG: --version="false" Sep 11 18:44:38.822276 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813367 2563 flags.go:64] FLAG: --vmodule="" Sep 11 18:44:38.822276 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813371 2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 11 18:44:38.822946 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.813374 2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 11 18:44:38.822946 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813493 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 18:44:38.822946 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813497 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 18:44:38.822946 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813514 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 18:44:38.822946 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813518 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 18:44:38.822946 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813522 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 18:44:38.822946 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813526 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 18:44:38.822946 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813530 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 18:44:38.822946 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813534 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 18:44:38.822946 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813537 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 18:44:38.822946 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813540 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 18:44:38.822946 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813543 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 18:44:38.822946 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813553 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 18:44:38.822946 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813557 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 18:44:38.822946 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813560 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 18:44:38.822946 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813563 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 18:44:38.822946 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813565 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 18:44:38.822946 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813568 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 18:44:38.822946 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813571 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 18:44:38.823412 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813574 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 18:44:38.823412 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813576 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 18:44:38.823412 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813579 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 18:44:38.823412 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813581 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 18:44:38.823412 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813584 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 18:44:38.823412 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813587 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 18:44:38.823412 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813592 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 18:44:38.823412 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813596 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 18:44:38.823412 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813599 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 18:44:38.823412 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813601 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 18:44:38.823412 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813604 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 11 18:44:38.823412 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813607 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 18:44:38.823412 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813609 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 18:44:38.823412 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813612 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 18:44:38.823412 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813615 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 18:44:38.823412 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813617 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 18:44:38.823412 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813620 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 18:44:38.823412 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813622 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 18:44:38.823412 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813625 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 18:44:38.823412 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813629 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 18:44:38.823918 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813632 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 18:44:38.823918 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813634 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 18:44:38.823918 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813680 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 18:44:38.823918 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813730 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 18:44:38.823918 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813736 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 18:44:38.823918 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813740 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 18:44:38.823918 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813745 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 18:44:38.823918 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813750 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 18:44:38.823918 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813754 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 18:44:38.823918 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813758 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 18:44:38.823918 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813763 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 18:44:38.823918 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813771 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 18:44:38.823918 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813776 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 18:44:38.823918 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813781 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 18:44:38.823918 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813786 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 18:44:38.823918 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813795 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 18:44:38.823918 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813800 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 18:44:38.823918 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813804 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 18:44:38.823918 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.813810 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 18:44:38.824388 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815679 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 18:44:38.824388 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815697 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 18:44:38.824388 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815703 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 18:44:38.824388 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815708 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 18:44:38.824388 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815713 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 18:44:38.824388 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815718 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 18:44:38.824388 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815723 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 18:44:38.824388 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815728 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 18:44:38.824388 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815733 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 18:44:38.824388 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815761 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 18:44:38.824388 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815767 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 18:44:38.824388 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815862 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 18:44:38.824388 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815868 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 18:44:38.824388 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815873 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 18:44:38.824388 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815878 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 18:44:38.824388 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815881 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 18:44:38.824388 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815884 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 18:44:38.824388 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815887 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 18:44:38.824388 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815890 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 18:44:38.824388 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815892 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 18:44:38.825017 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815895 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 18:44:38.825017 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815898 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 18:44:38.825017 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815901 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 18:44:38.825017 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815903 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 18:44:38.825017 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815906 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 18:44:38.825017 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815909 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 18:44:38.825017 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815911 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 18:44:38.825017 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815915 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 18:44:38.825017 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815918 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 18:44:38.825017 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815921 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 18:44:38.825017 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815923 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 18:44:38.825017 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815926 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 18:44:38.825017 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815929 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 18:44:38.825017 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815932 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 18:44:38.825017 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815935 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 18:44:38.825017 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815938 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 18:44:38.825017 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815940 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 18:44:38.825017 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815943 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 18:44:38.825017 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815946 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 18:44:38.825017 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815949 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 18:44:38.825520 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815951 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 18:44:38.825520 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.815954 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 18:44:38.825520 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.816565 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 11 18:44:38.825520 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.823311 2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 11 18:44:38.825520 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.823327 2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 11 18:44:38.825520 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.824182 2563 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 11 18:44:38.825520 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.824190 2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 11 18:44:38.825520 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824251 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 18:44:38.825520 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824258 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 18:44:38.825520 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824262 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 18:44:38.825520 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824265 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 18:44:38.825520 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824268 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 18:44:38.825520 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824271 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 18:44:38.825520 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824275 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 18:44:38.825520 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824278 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 18:44:38.825906 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824280 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 18:44:38.825906 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824283 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 18:44:38.825906 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824286 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 18:44:38.825906 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824288 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 18:44:38.825906 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824291 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 18:44:38.825906 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824294 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 18:44:38.825906 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824296 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 18:44:38.825906 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824299 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 18:44:38.825906 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824301 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 18:44:38.825906 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824304 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 18:44:38.825906 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824306 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 18:44:38.825906 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824310 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 18:44:38.825906 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824313 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 18:44:38.825906 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824316 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 18:44:38.825906 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824318 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 18:44:38.825906 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824321 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 18:44:38.825906 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824324 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 18:44:38.825906 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824326 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 18:44:38.825906 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824329 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 18:44:38.825906 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824331 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 18:44:38.826435 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824334 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 18:44:38.826435 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824338 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 18:44:38.826435 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824341 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 18:44:38.826435 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824344 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 18:44:38.826435 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824347 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 18:44:38.826435 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824349 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 18:44:38.826435 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824352 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 18:44:38.826435 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824355 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 18:44:38.826435 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824357 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 18:44:38.826435 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824360 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 18:44:38.826435 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824363 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 18:44:38.826435 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824365 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 18:44:38.826435 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824368 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 18:44:38.826435 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824371 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 18:44:38.826435 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824373 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 18:44:38.826435 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824376 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 18:44:38.826435 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824378 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 18:44:38.826435 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824381 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 18:44:38.826435 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824384 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 18:44:38.826435 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824386 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 18:44:38.826929 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824389 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 18:44:38.826929 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824392 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 18:44:38.826929 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824395 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 18:44:38.826929 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824398 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 18:44:38.826929 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824401 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 18:44:38.826929 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824404 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 18:44:38.826929 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824408 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 18:44:38.826929 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824412 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 18:44:38.826929 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824415 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 18:44:38.826929 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824418 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 18:44:38.826929 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824421 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 18:44:38.826929 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824423 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 18:44:38.826929 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824426 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 18:44:38.826929 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824429 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 18:44:38.826929 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824431 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 18:44:38.826929 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824435 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 18:44:38.826929 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824437 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 18:44:38.826929 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824440 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 18:44:38.826929 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824442 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 18:44:38.827394 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824445 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 18:44:38.827394 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824447 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 18:44:38.827394 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824450 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 18:44:38.827394 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824452 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 18:44:38.827394 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824455 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 18:44:38.827394 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824458 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 18:44:38.827394 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824460 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 18:44:38.827394 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824463 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 18:44:38.827394 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824465 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 18:44:38.827394 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824468 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 18:44:38.827394 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824470 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 18:44:38.827394 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824473 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 18:44:38.827394 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824477 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 11 18:44:38.827394 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824480 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 18:44:38.827394 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824482 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 18:44:38.827394 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824485 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 18:44:38.827394 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824487 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 18:44:38.827394 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824490 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 18:44:38.827394 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824493 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 18:44:38.827394 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824495 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 18:44:38.827903 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824498 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 18:44:38.827903 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824518 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 18:44:38.827903 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824524 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 18:44:38.827903 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824528 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 18:44:38.827903 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824531 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 18:44:38.827903 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824535 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 18:44:38.827903 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824538 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 18:44:38.827903 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824541 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 18:44:38.827903 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824544 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 18:44:38.827903 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824547 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 18:44:38.827903 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824551 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 18:44:38.827903 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824553 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 18:44:38.827903 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.824558 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 11 18:44:38.827903 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824681 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 18:44:38.828263 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824687 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 18:44:38.828263 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824689 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 18:44:38.828263 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824692 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 18:44:38.828263 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824695 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 18:44:38.828263 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824698 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 18:44:38.828263 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824701 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 18:44:38.828263 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824703 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 18:44:38.828263 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824706 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 18:44:38.828263 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824709 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 18:44:38.828263 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824711 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 18:44:38.828263 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824715 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 18:44:38.828263 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824719 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 18:44:38.828263 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824722 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 18:44:38.828263 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824725 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 18:44:38.828263 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824728 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 18:44:38.828263 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824731 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 18:44:38.828263 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824734 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 18:44:38.828263 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824737 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 18:44:38.828263 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824740 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 18:44:38.828263 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824743 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 18:44:38.828848 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824746 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 18:44:38.828848 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824748 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 18:44:38.828848 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824751 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 18:44:38.828848 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824754 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 18:44:38.828848 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824756 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 18:44:38.828848 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824759 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 18:44:38.828848 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824762 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 18:44:38.828848 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824765 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 18:44:38.828848 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824768 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 18:44:38.828848 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824770 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 18:44:38.828848 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824773 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 18:44:38.828848 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824775 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 18:44:38.828848 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824778 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 18:44:38.828848 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824781 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 18:44:38.828848 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824783 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 18:44:38.828848 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824786 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 18:44:38.828848 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824788 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 18:44:38.828848 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824791 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 18:44:38.828848 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824794 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 18:44:38.828848 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824796 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 18:44:38.829327 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824799 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 18:44:38.829327 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824801 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 18:44:38.829327 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824803 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 18:44:38.829327 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824806 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 18:44:38.829327 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824810 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 18:44:38.829327 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824814 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 18:44:38.829327 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824817 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 18:44:38.829327 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824820 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 18:44:38.829327 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824823 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 18:44:38.829327 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824826 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 18:44:38.829327 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824828 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 18:44:38.829327 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824831 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 18:44:38.829327 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824833 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 18:44:38.829327 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824836 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 18:44:38.829327 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824838 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 18:44:38.829327 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824841 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 18:44:38.829327 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824844 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 18:44:38.829327 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824847 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 18:44:38.829327 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824850 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 18:44:38.829327 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824852 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 18:44:38.829820 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824855 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 18:44:38.829820 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824858 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 18:44:38.829820 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824861 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 18:44:38.829820 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824863 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 18:44:38.829820 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824866 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 18:44:38.829820 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824869 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 11 18:44:38.829820 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824871 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 18:44:38.829820 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824874 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 18:44:38.829820 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824876 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 18:44:38.829820 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824879 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 18:44:38.829820 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824881 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 18:44:38.829820 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824884 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 18:44:38.829820 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824887 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 18:44:38.829820 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824890 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 18:44:38.829820 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824892 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 18:44:38.829820 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824895 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 18:44:38.829820 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824898 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 18:44:38.829820 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824901 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 18:44:38.829820 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824904 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 18:44:38.830293 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824906 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 18:44:38.830293 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824909 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 18:44:38.830293 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824911 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 18:44:38.830293 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824914 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 18:44:38.830293 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824916 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 18:44:38.830293 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824920 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 18:44:38.830293 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824923 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 18:44:38.830293 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824926 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 18:44:38.830293 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824929 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 18:44:38.830293 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824932 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 18:44:38.830293 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824935 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 18:44:38.830293 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824938 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 18:44:38.830293 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824940 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 18:44:38.830293 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824943 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 18:44:38.830293 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824946 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 18:44:38.830293 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824949 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 18:44:38.830293 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824951 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 18:44:38.830293 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824954 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 18:44:38.830293 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:38.824956 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 18:44:38.830773 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.824961 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 11 18:44:38.830773 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.825110 2563 server.go:962] "Client rotation is on, will bootstrap in background" Sep 11 18:44:38.830773 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.827077 2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 11 18:44:38.830773 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.827938 2563 server.go:1019] "Starting client certificate rotation" Sep 11 18:44:38.830773 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.828043 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 11 18:44:38.830773 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.828073 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 11 18:44:38.830987 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.830970 2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 11 18:44:38.832162 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.832144 2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 11 18:44:38.840723 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.840702 2563 log.go:25] "Validated CRI v1 runtime API" Sep 11 18:44:38.845561 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.845548 2563 log.go:25] "Validated CRI v1 image API" Sep 11 18:44:38.846403 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.846385 2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 11 18:44:38.848609 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.848581 2563 fs.go:135] Filesystem UUIDs: map[3c8c8bb9-0267-4f04-ac07-c07c8757aa70:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 d79df89f-bb8e-493a-96fc-5ee01754a40f:/dev/nvme0n1p3] Sep 11 18:44:38.848704 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.848608 2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 11 18:44:38.855121 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.855001 2563 manager.go:216] Machine: {Timestamp:2026-09-11 18:44:38.853696129 +0000 UTC m=+0.316302139 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097616 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ebe989716dd35f8d1e7c585139339 SystemUUID:ec2ebe98-9716-dd35-f8d1-e7c585139339 BootID:673bb230-4efa-4cae-a169-5a7d2fa4def1 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:fe:90:5b:d9:fb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:fe:90:5b:d9:fb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ce:20:18:24:60:11 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 11 18:44:38.855121 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.855111 2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 11 18:44:38.855269 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.855206 2563 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 11 18:44:38.856616 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.856599 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 11 18:44:38.858402 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.858378 2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 11 18:44:38.858564 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.858406 2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-192.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 11 18:44:38.859268 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.859257 2563 topology_manager.go:138] "Creating topology manager with none policy" Sep 11 18:44:38.859300 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.859271 2563 container_manager_linux.go:309] "Creating device plugin manager" Sep 11 18:44:38.859300 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.859279 2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 11 18:44:38.859821 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.859811 2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 11 18:44:38.860563 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.860553 2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 11 18:44:38.863814 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.863801 2563 state_mem.go:36] "Initialized new in-memory state store" Sep 11 18:44:38.863928 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.863918 2563 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 11 18:44:38.863960 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.863949 2563 kubelet.go:486] "Attempting to sync node with API server" Sep 11 18:44:38.863987 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.863961 2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 11 18:44:38.863987 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.863973 2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 11 18:44:38.863987 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.863982 2563 kubelet.go:398] "Adding apiserver pod source" Sep 11 18:44:38.864099 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.863992 2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 11 18:44:38.864360 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.864345 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 11 18:44:38.864360 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.864365 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 11 18:44:38.866374 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.866359 2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 11 18:44:38.868325 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.868311 2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 11 18:44:38.868382 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.868341 2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 11 18:44:38.869897 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.869886 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 11 18:44:38.869940 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.869902 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 11 18:44:38.869940 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.869908 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 11 18:44:38.869940 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.869913 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 11 18:44:38.869940 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.869922 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 11 18:44:38.869940 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.869928 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 11 18:44:38.870114 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.869945 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 11 18:44:38.870114 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.869952 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 11 18:44:38.870114 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.869959 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 11 18:44:38.870114 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.869964 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 11 18:44:38.870114 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.869980 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 11 18:44:38.870114 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.869985 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 11 18:44:38.871516 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.871493 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 11 18:44:38.871558 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.871518 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 11 18:44:38.872417 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:38.872387 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-192.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 11 18:44:38.872579 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:38.872554 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 11 18:44:38.874718 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.874701 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-192.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 11 18:44:38.875533 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.875520 2563 server.go:1268] "Started kubelet" Sep 11 18:44:38.875618 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.875590 2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 11 18:44:38.875697 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.875647 2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 11 18:44:38.875733 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.875721 2563 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 11 18:44:38.876643 ip-10-0-134-192 systemd[1]: Started Kubernetes Kubelet. Sep 11 18:44:38.877105 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.877085 2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 11 18:44:38.878582 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.878569 2563 server.go:310] "Adding debug handlers to kubelet server" Sep 11 18:44:38.879425 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.879408 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rdmcn" Sep 11 18:44:38.882157 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.882139 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 11 18:44:38.882554 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:38.881547 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-192.ec2.internal.18d45884af73feef default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-192.ec2.internal,UID:ip-10-0-134-192.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-192.ec2.internal,},FirstTimestamp:2026-09-11 18:44:38.875471599 +0000 UTC m=+0.338077608,LastTimestamp:2026-09-11 18:44:38.875471599 +0000 UTC m=+0.338077608,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-192.ec2.internal,}" Sep 11 18:44:38.882683 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.882669 2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 11 18:44:38.882829 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:38.882807 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-192.ec2.internal\" not found" Sep 11 18:44:38.883388 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.883234 2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 11 18:44:38.883388 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.883239 2563 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 11 18:44:38.883388 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.883264 2563 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 11 18:44:38.883388 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.883376 2563 reconstruct.go:97] "Volume reconstruction finished" Sep 11 18:44:38.883388 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.883388 2563 reconciler.go:29] "Reconciler: start to sync state" Sep 11 18:44:38.883964 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.883947 2563 factory.go:55] Registering systemd factory Sep 11 18:44:38.883964 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.883965 2563 factory.go:223] Registration of the systemd container factory successfully Sep 11 18:44:38.884184 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.884166 2563 factory.go:153] Registering CRI-O factory Sep 11 18:44:38.884184 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.884181 2563 factory.go:223] Registration of the crio container factory successfully Sep 11 18:44:38.884314 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.884231 2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 11 18:44:38.884314 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.884251 2563 factory.go:103] Registering Raw factory Sep 11 18:44:38.884314 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.884264 2563 manager.go:1196] Started watching for new ooms in manager Sep 11 18:44:38.884861 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:38.884829 2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 11 18:44:38.885093 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.885078 2563 manager.go:349] Starting recovery of all containers Sep 11 18:44:38.887852 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.887827 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rdmcn" Sep 11 18:44:38.888965 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:38.888934 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 11 18:44:38.889101 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:38.889082 2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-192.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 11 18:44:38.896338 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.896321 2563 manager.go:354] Recovery completed Sep 11 18:44:38.900665 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.900651 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 18:44:38.902312 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.902296 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-192.ec2.internal" event="NodeHasSufficientMemory" Sep 11 18:44:38.902369 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.902323 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-192.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 18:44:38.902369 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.902338 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-192.ec2.internal" event="NodeHasSufficientPID" Sep 11 18:44:38.903706 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.902953 2563 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 11 18:44:38.903706 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.902965 2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 11 18:44:38.903706 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.902954 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 18:44:38.903706 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.902982 2563 state_mem.go:36] "Initialized new in-memory state store" Sep 11 18:44:38.903706 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.903684 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-192.ec2.internal" event="NodeHasSufficientMemory" Sep 11 18:44:38.903895 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.903711 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-192.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 18:44:38.903895 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.903725 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-192.ec2.internal" event="NodeHasSufficientPID" Sep 11 18:44:38.904794 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:38.904704 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-192.ec2.internal.18d45884b10d8dd5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-192.ec2.internal,UID:ip-10-0-134-192.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-192.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-192.ec2.internal,},FirstTimestamp:2026-09-11 18:44:38.902312405 +0000 UTC m=+0.364918414,LastTimestamp:2026-09-11 18:44:38.902312405 +0000 UTC m=+0.364918414,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-192.ec2.internal,}" Sep 11 18:44:38.904892 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.904881 2563 policy_none.go:49] "None policy: Start" Sep 11 18:44:38.904943 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.904893 2563 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 11 18:44:38.904943 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.904907 2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 11 18:44:38.906941 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.906830 2563 policy_none.go:47] "Start" Sep 11 18:44:38.931134 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.931107 2563 manager.go:337] "Starting Device Plugin manager" Sep 11 18:44:38.931259 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:38.931150 2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 11 18:44:38.931259 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.931163 2563 server.go:87] "Starting device plugin registration server" Sep 11 18:44:38.931591 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.931568 2563 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 11 18:44:38.931662 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.931592 2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 11 18:44:38.932058 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.931774 2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 11 18:44:38.932058 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.931865 2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 11 18:44:38.932058 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:38.931874 2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 11 18:44:38.932488 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:38.932465 2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 11 18:44:38.932608 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:38.932531 2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-192.ec2.internal\" not found" Sep 11 18:44:39.032757 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.032723 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 18:44:39.033825 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.033806 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-192.ec2.internal" event="NodeHasSufficientMemory" Sep 11 18:44:39.033934 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.033841 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-192.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 18:44:39.033934 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.033854 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-192.ec2.internal" event="NodeHasSufficientPID" Sep 11 18:44:39.033934 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.033880 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-192.ec2.internal" Sep 11 18:44:39.041440 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.041422 2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-192.ec2.internal" Sep 11 18:44:39.041517 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:39.041448 2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-192.ec2.internal\": node \"ip-10-0-134-192.ec2.internal\" not found" Sep 11 18:44:39.052792 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.052760 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 11 18:44:39.054181 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.054161 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 11 18:44:39.054280 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.054208 2563 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 11 18:44:39.054280 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.054227 2563 kubelet.go:2448] "Starting kubelet main sync loop" Sep 11 18:44:39.054357 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:39.054299 2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 11 18:44:39.056140 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.056124 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 18:44:39.059020 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:39.058960 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-192.ec2.internal\" not found" Sep 11 18:44:39.154445 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.154403 2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-134-192.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-192.ec2.internal"] Sep 11 18:44:39.154635 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.154497 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 18:44:39.156610 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.156591 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-192.ec2.internal" event="NodeHasSufficientMemory" Sep 11 18:44:39.156713 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.156622 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-192.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 18:44:39.156713 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.156633 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-192.ec2.internal" event="NodeHasSufficientPID" Sep 11 18:44:39.158968 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.158954 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 18:44:39.159010 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:39.158983 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-192.ec2.internal\" not found" Sep 11 18:44:39.159075 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.159060 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-192.ec2.internal" Sep 11 18:44:39.159116 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.159096 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 18:44:39.159845 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.159826 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-192.ec2.internal" event="NodeHasSufficientMemory" Sep 11 18:44:39.159845 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.159848 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-192.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 18:44:39.159997 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.159857 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-192.ec2.internal" event="NodeHasSufficientPID" Sep 11 18:44:39.159997 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.159828 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-192.ec2.internal" event="NodeHasSufficientMemory" Sep 11 18:44:39.159997 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.159927 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-192.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 18:44:39.159997 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.159942 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-192.ec2.internal" event="NodeHasSufficientPID" Sep 11 18:44:39.162096 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.162081 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-192.ec2.internal" Sep 11 18:44:39.162161 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.162106 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 18:44:39.162825 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.162809 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-192.ec2.internal" event="NodeHasSufficientMemory" Sep 11 18:44:39.162928 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.162834 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-192.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 18:44:39.162928 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.162846 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-192.ec2.internal" event="NodeHasSufficientPID" Sep 11 18:44:39.182094 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:39.182065 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-192.ec2.internal\" not found" node="ip-10-0-134-192.ec2.internal" Sep 11 18:44:39.184192 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.184168 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7241124c924c97f39a53ae6bf4dfdb02-config\") pod \"kube-apiserver-proxy-ip-10-0-134-192.ec2.internal\" (UID: \"7241124c924c97f39a53ae6bf4dfdb02\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-192.ec2.internal" Sep 11 18:44:39.184301 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.184206 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0430ff9a9376761ce7d088dd1f84ed06-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-192.ec2.internal\" (UID: \"0430ff9a9376761ce7d088dd1f84ed06\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-192.ec2.internal" Sep 11 18:44:39.184301 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.184232 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0430ff9a9376761ce7d088dd1f84ed06-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-192.ec2.internal\" (UID: \"0430ff9a9376761ce7d088dd1f84ed06\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-192.ec2.internal" Sep 11 18:44:39.186602 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:39.186588 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-192.ec2.internal\" not found" node="ip-10-0-134-192.ec2.internal" Sep 11 18:44:39.260072 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:39.260042 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-192.ec2.internal\" not found" Sep 11 18:44:39.284497 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.284455 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0430ff9a9376761ce7d088dd1f84ed06-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-192.ec2.internal\" (UID: \"0430ff9a9376761ce7d088dd1f84ed06\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-192.ec2.internal" Sep 11 18:44:39.284659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.284526 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0430ff9a9376761ce7d088dd1f84ed06-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-192.ec2.internal\" (UID: \"0430ff9a9376761ce7d088dd1f84ed06\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-192.ec2.internal" Sep 11 18:44:39.284659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.284550 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7241124c924c97f39a53ae6bf4dfdb02-config\") pod \"kube-apiserver-proxy-ip-10-0-134-192.ec2.internal\" (UID: \"7241124c924c97f39a53ae6bf4dfdb02\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-192.ec2.internal" Sep 11 18:44:39.284659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.284562 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0430ff9a9376761ce7d088dd1f84ed06-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-192.ec2.internal\" (UID: \"0430ff9a9376761ce7d088dd1f84ed06\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-192.ec2.internal" Sep 11 18:44:39.284659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.284593 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7241124c924c97f39a53ae6bf4dfdb02-config\") pod \"kube-apiserver-proxy-ip-10-0-134-192.ec2.internal\" (UID: \"7241124c924c97f39a53ae6bf4dfdb02\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-192.ec2.internal" Sep 11 18:44:39.284659 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.284624 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0430ff9a9376761ce7d088dd1f84ed06-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-192.ec2.internal\" (UID: \"0430ff9a9376761ce7d088dd1f84ed06\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-192.ec2.internal" Sep 11 18:44:39.360810 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:39.360732 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-192.ec2.internal\" not found" Sep 11 18:44:39.465790 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:39.465750 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-192.ec2.internal\" not found" Sep 11 18:44:39.484406 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.484367 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-192.ec2.internal" Sep 11 18:44:39.489802 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.489422 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-192.ec2.internal" Sep 11 18:44:39.567078 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:39.567033 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-192.ec2.internal\" not found" Sep 11 18:44:39.668244 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:39.668161 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-192.ec2.internal\" not found" Sep 11 18:44:39.769167 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:39.769124 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-192.ec2.internal\" not found" Sep 11 18:44:39.828024 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.827992 2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 11 18:44:39.828579 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:39.828105 2563 request.go:1196] "Unexpected error when reading response body" err="read tcp 10.0.134.192:46210->54.242.52.197:6443: use of closed network connection" Sep 11 18:44:39.828579 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.828121 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 11 18:44:39.828579 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:39.828177 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: unexpected error when reading response body. Please retry. Original error: read tcp 10.0.134.192:46210->54.242.52.197:6443: use of closed network connection" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 11 18:44:39.870305 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:39.870268 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-192.ec2.internal\" not found" Sep 11 18:44:39.893592 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.893551 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 18:39:39 +0000 UTC" deadline="2028-03-02 05:49:46.441971025 +0000 UTC" Sep 11 18:44:39.893592 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:39.893584 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12899h5m6.548391751s" Sep 11 18:44:39.970630 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:39.970572 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-192.ec2.internal\" not found" Sep 11 18:44:40.071495 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:40.071464 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-192.ec2.internal\" not found" Sep 11 18:44:40.108659 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:40.108620 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7241124c924c97f39a53ae6bf4dfdb02.slice/crio-8011e0b3ba4709af6ed302e36ec038922004687c6c9de83595a2d4f07273cee5 WatchSource:0}: Error finding container 8011e0b3ba4709af6ed302e36ec038922004687c6c9de83595a2d4f07273cee5: Status 404 returned error can't find the container with id 8011e0b3ba4709af6ed302e36ec038922004687c6c9de83595a2d4f07273cee5 Sep 11 18:44:40.109667 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:40.109636 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0430ff9a9376761ce7d088dd1f84ed06.slice/crio-7378e3c99cae1ae51055059f83fc40b4d504aba0e813949ca2a39a11d3a65ab9 WatchSource:0}: Error finding container 7378e3c99cae1ae51055059f83fc40b4d504aba0e813949ca2a39a11d3a65ab9: Status 404 returned error can't find the container with id 7378e3c99cae1ae51055059f83fc40b4d504aba0e813949ca2a39a11d3a65ab9 Sep 11 18:44:40.115418 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.115399 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 11 18:44:40.172545 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:40.172490 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-192.ec2.internal\" not found" Sep 11 18:44:40.207814 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.207754 2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 18:44:40.283035 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.283004 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-192.ec2.internal" Sep 11 18:44:40.293855 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.293835 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 11 18:44:40.295721 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.295708 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-192.ec2.internal" Sep 11 18:44:40.307567 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.307551 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 11 18:44:40.444525 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.444494 2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 18:44:40.679475 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.679390 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 18:44:40.865684 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.865651 2563 apiserver.go:52] "Watching apiserver" Sep 11 18:44:40.877600 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.877563 2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 11 18:44:40.878164 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.878129 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-7f246","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-192.ec2.internal","openshift-network-operator/iptables-alerter-nth2m","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6","openshift-image-registry/node-ca-tnjsf","openshift-multus/multus-additional-cni-plugins-9xtl5","openshift-multus/multus-z8mxj","openshift-multus/network-metrics-daemon-lhj2z","openshift-network-diagnostics/network-check-target-rxmht","openshift-ovn-kubernetes/ovnkube-node-8tmtk","kube-system/global-pull-secret-syncer-sxqg6","kube-system/konnectivity-agent-gv5z5","kube-system/kube-apiserver-proxy-ip-10-0-134-192.ec2.internal","openshift-cluster-node-tuning-operator/tuned-fdn85"] Sep 11 18:44:40.881024 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.880996 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.884349 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.883325 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 11 18:44:40.884349 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.883924 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:44:40.884349 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:40.883998 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxqg6" podUID="b799f9cb-ce50-4731-9fbe-48f47807a053" Sep 11 18:44:40.886018 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.885635 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nth2m" Sep 11 18:44:40.888094 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.888076 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" Sep 11 18:44:40.890937 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.890388 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tnjsf" Sep 11 18:44:40.893970 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.893522 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9xtl5" Sep 11 18:44:40.895076 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.895040 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zrxh\" (UniqueName: \"kubernetes.io/projected/96d752b9-bf5e-4519-b2b8-6dbd832cab19-kube-api-access-2zrxh\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.895182 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.895078 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wtv7\" (UniqueName: \"kubernetes.io/projected/876bf078-f416-471c-b683-74781309a51a-kube-api-access-7wtv7\") pod \"iptables-alerter-nth2m\" (UID: \"876bf078-f416-471c-b683-74781309a51a\") " pod="openshift-network-operator/iptables-alerter-nth2m" Sep 11 18:44:40.895182 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.895107 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-multus-cni-dir\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.895182 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.895132 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-multus-socket-dir-parent\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.895182 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.895160 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b799f9cb-ce50-4731-9fbe-48f47807a053-kubelet-config\") pod \"global-pull-secret-syncer-sxqg6\" (UID: \"b799f9cb-ce50-4731-9fbe-48f47807a053\") " pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:44:40.895404 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.895184 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/876bf078-f416-471c-b683-74781309a51a-iptables-alerter-script\") pod \"iptables-alerter-nth2m\" (UID: \"876bf078-f416-471c-b683-74781309a51a\") " pod="openshift-network-operator/iptables-alerter-nth2m" Sep 11 18:44:40.895404 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.895208 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-system-cni-dir\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.895404 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.895232 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-host-run-k8s-cni-cncf-io\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.895404 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.895261 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-host-run-netns\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.895404 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.895286 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-host-var-lib-cni-bin\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.895404 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.895311 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/96d752b9-bf5e-4519-b2b8-6dbd832cab19-multus-daemon-config\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.895404 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.895335 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b799f9cb-ce50-4731-9fbe-48f47807a053-dbus\") pod \"global-pull-secret-syncer-sxqg6\" (UID: \"b799f9cb-ce50-4731-9fbe-48f47807a053\") " pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:44:40.895404 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.895361 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-cnibin\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.895404 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.895386 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-host-var-lib-cni-multus\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.895858 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.895410 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-host-var-lib-kubelet\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.895858 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.895439 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-hostroot\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.895858 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.895465 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-multus-conf-dir\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.895858 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.895529 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-host-run-multus-certs\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.895858 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.895581 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b799f9cb-ce50-4731-9fbe-48f47807a053-original-pull-secret\") pod \"global-pull-secret-syncer-sxqg6\" (UID: \"b799f9cb-ce50-4731-9fbe-48f47807a053\") " pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:44:40.895858 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.895605 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/876bf078-f416-471c-b683-74781309a51a-host-slash\") pod \"iptables-alerter-nth2m\" (UID: \"876bf078-f416-471c-b683-74781309a51a\") " pod="openshift-network-operator/iptables-alerter-nth2m" Sep 11 18:44:40.895858 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.895632 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96d752b9-bf5e-4519-b2b8-6dbd832cab19-cni-binary-copy\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.895858 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.895672 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-etc-kubernetes\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.895858 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.895747 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:44:40.895858 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:40.895818 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxmht" podUID="254d8445-a0f4-4c11-bc96-2cb74c34ab13" Sep 11 18:44:40.896350 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.896087 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 11 18:44:40.896417 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.896400 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 11 18:44:40.896756 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.896577 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 11 18:44:40.896756 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.896681 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 11 18:44:40.896905 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.896684 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-t6v2x\"" Sep 11 18:44:40.898083 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.897725 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:40.900205 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.900178 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:44:40.900299 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:40.900270 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lhj2z" podUID="b644d098-032f-42c9-9839-41ce9eb09844" Sep 11 18:44:40.907535 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.904513 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:40.907535 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.905520 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 11 18:44:40.907535 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.905802 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7f246" Sep 11 18:44:40.907535 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.906113 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 11 18:44:40.907535 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.906406 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 11 18:44:40.907535 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.906669 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-t8xdn\"" Sep 11 18:44:40.907535 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.906807 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 11 18:44:40.907535 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.906844 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 11 18:44:40.907535 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.906891 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 11 18:44:40.907535 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.907033 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 11 18:44:40.907535 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.907443 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 11 18:44:40.908142 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.907823 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5sh79\"" Sep 11 18:44:40.908792 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.908327 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 11 18:44:40.908792 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.908359 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 11 18:44:40.908792 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.908374 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hr8b6\"" Sep 11 18:44:40.908792 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.908516 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-krlxf\"" Sep 11 18:44:40.908792 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.908596 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-jvdn4\"" Sep 11 18:44:40.909092 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.908947 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 11 18:44:40.909092 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.908920 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 11 18:44:40.909886 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.909841 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 11 18:44:40.911662 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.911281 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gv5z5" Sep 11 18:44:40.912168 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.912013 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 11 18:44:40.920153 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.919967 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 11 18:44:40.920153 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.919977 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 11 18:44:40.920153 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.920055 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ct69s\"" Sep 11 18:44:40.931314 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.931293 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-c9fgs\"" Sep 11 18:44:40.931815 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.931792 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-25l7c\"" Sep 11 18:44:40.932762 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.932032 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 11 18:44:40.932762 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.932134 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 11 18:44:40.932762 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.932223 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 11 18:44:40.932762 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.932272 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 11 18:44:40.932762 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.932359 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 11 18:44:40.932762 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.932614 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 11 18:44:40.933841 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.933678 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 11 18:44:40.940599 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.940444 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 11 18:44:40.980388 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.980359 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-b4q7q" Sep 11 18:44:40.984643 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.984619 2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 11 18:44:40.996190 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996150 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b799f9cb-ce50-4731-9fbe-48f47807a053-dbus\") pod \"global-pull-secret-syncer-sxqg6\" (UID: \"b799f9cb-ce50-4731-9fbe-48f47807a053\") " pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:44:40.996342 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996193 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-cnibin\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.996342 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996219 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-hostroot\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.996342 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996244 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-host-kubelet\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:40.996342 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996267 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a688c734-cd8a-40eb-843f-d78e5ec411fd-ovn-node-metrics-cert\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:40.996342 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996291 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a688c734-cd8a-40eb-843f-d78e5ec411fd-ovnkube-script-lib\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:40.996342 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996313 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-etc-kubernetes\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.996676 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996361 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce-socket-dir\") pod \"aws-ebs-csi-driver-node-m4xd6\" (UID: \"93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" Sep 11 18:44:40.996676 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996390 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/254dd733-80bd-4f2a-9fbc-a57622608b32-host\") pod \"node-ca-tnjsf\" (UID: \"254dd733-80bd-4f2a-9fbc-a57622608b32\") " pod="openshift-image-registry/node-ca-tnjsf" Sep 11 18:44:40.996676 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996416 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9819d65-2122-497e-831c-917aa1fabd52-cni-binary-copy\") pod \"multus-additional-cni-plugins-9xtl5\" (UID: \"e9819d65-2122-497e-831c-917aa1fabd52\") " pod="openshift-multus/multus-additional-cni-plugins-9xtl5" Sep 11 18:44:40.996676 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996445 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-var-lib-kubelet\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:40.996676 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996484 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce-device-dir\") pod \"aws-ebs-csi-driver-node-m4xd6\" (UID: \"93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" Sep 11 18:44:40.996676 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996512 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/254dd733-80bd-4f2a-9fbc-a57622608b32-serviceca\") pod \"node-ca-tnjsf\" (UID: \"254dd733-80bd-4f2a-9fbc-a57622608b32\") " pod="openshift-image-registry/node-ca-tnjsf" Sep 11 18:44:40.996676 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996508 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b799f9cb-ce50-4731-9fbe-48f47807a053-dbus\") pod \"global-pull-secret-syncer-sxqg6\" (UID: \"b799f9cb-ce50-4731-9fbe-48f47807a053\") " pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:44:40.996676 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996540 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9819d65-2122-497e-831c-917aa1fabd52-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9xtl5\" (UID: \"e9819d65-2122-497e-831c-917aa1fabd52\") " pod="openshift-multus/multus-additional-cni-plugins-9xtl5" Sep 11 18:44:40.996676 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996568 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/274576da-90e4-47d9-b053-72498cd1ab3e-etc-tuned\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:40.996676 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996599 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-cnibin\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.996676 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996608 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/274576da-90e4-47d9-b053-72498cd1ab3e-tmp\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:40.996676 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996644 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-hostroot\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.997327 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996704 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-systemd-units\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:40.997327 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996746 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-host-run-ovn-kubernetes\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:40.997327 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996773 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgw2d\" (UniqueName: \"kubernetes.io/projected/a688c734-cd8a-40eb-843f-d78e5ec411fd-kube-api-access-bgw2d\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:40.997327 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996800 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zrxh\" (UniqueName: \"kubernetes.io/projected/96d752b9-bf5e-4519-b2b8-6dbd832cab19-kube-api-access-2zrxh\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.997327 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996841 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-host-var-lib-cni-multus\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.997327 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996867 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-host-var-lib-kubelet\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.997327 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996896 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-multus-conf-dir\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.997327 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996947 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-host-run-multus-certs\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.997327 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.996976 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-etc-modprobe-d\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:40.997327 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.997002 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-host\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:40.997327 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.997026 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-host-slash\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:40.997327 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.997052 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96d752b9-bf5e-4519-b2b8-6dbd832cab19-cni-binary-copy\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.997327 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.997077 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m4xd6\" (UID: \"93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" Sep 11 18:44:40.997327 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.997120 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9819d65-2122-497e-831c-917aa1fabd52-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9xtl5\" (UID: \"e9819d65-2122-497e-831c-917aa1fabd52\") " pod="openshift-multus/multus-additional-cni-plugins-9xtl5" Sep 11 18:44:40.997327 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.997146 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-run-openvswitch\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:40.997327 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.997173 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a688c734-cd8a-40eb-843f-d78e5ec411fd-ovnkube-config\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:40.997327 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.997200 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/22ef1e56-f6b2-4ebf-8be0-63669535237b-konnectivity-ca\") pod \"konnectivity-agent-gv5z5\" (UID: \"22ef1e56-f6b2-4ebf-8be0-63669535237b\") " pod="kube-system/konnectivity-agent-gv5z5" Sep 11 18:44:40.998221 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.997228 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-multus-cni-dir\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.998221 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.997257 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-multus-socket-dir-parent\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.998221 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.997284 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dp74\" (UniqueName: \"kubernetes.io/projected/93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce-kube-api-access-8dp74\") pod \"aws-ebs-csi-driver-node-m4xd6\" (UID: \"93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" Sep 11 18:44:40.998221 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.997317 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjqb8\" (UniqueName: \"kubernetes.io/projected/254dd733-80bd-4f2a-9fbc-a57622608b32-kube-api-access-xjqb8\") pod \"node-ca-tnjsf\" (UID: \"254dd733-80bd-4f2a-9fbc-a57622608b32\") " pod="openshift-image-registry/node-ca-tnjsf" Sep 11 18:44:40.998221 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.997355 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s2jb\" (UniqueName: \"kubernetes.io/projected/e9819d65-2122-497e-831c-917aa1fabd52-kube-api-access-5s2jb\") pod \"multus-additional-cni-plugins-9xtl5\" (UID: \"e9819d65-2122-497e-831c-917aa1fabd52\") " pod="openshift-multus/multus-additional-cni-plugins-9xtl5" Sep 11 18:44:40.998221 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.997384 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-etc-sysctl-conf\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:40.998221 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.997448 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/60dd726a-4169-4a95-ade1-c067f1851aac-hosts-file\") pod \"node-resolver-7f246\" (UID: \"60dd726a-4169-4a95-ade1-c067f1851aac\") " pod="openshift-dns/node-resolver-7f246" Sep 11 18:44:40.998221 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.997476 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-system-cni-dir\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.998221 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.997523 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-host-run-netns\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.998221 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.997555 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-etc-kubernetes\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:40.998221 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.997598 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-sys\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:40.998221 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.997627 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6h8l\" (UniqueName: \"kubernetes.io/projected/b644d098-032f-42c9-9839-41ce9eb09844-kube-api-access-x6h8l\") pod \"network-metrics-daemon-lhj2z\" (UID: \"b644d098-032f-42c9-9839-41ce9eb09844\") " pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:44:40.998221 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.997653 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-log-socket\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:40.998221 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.997694 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60dd726a-4169-4a95-ade1-c067f1851aac-tmp-dir\") pod \"node-resolver-7f246\" (UID: \"60dd726a-4169-4a95-ade1-c067f1851aac\") " pod="openshift-dns/node-resolver-7f246" Sep 11 18:44:40.998221 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.997723 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/22ef1e56-f6b2-4ebf-8be0-63669535237b-agent-certs\") pod \"konnectivity-agent-gv5z5\" (UID: \"22ef1e56-f6b2-4ebf-8be0-63669535237b\") " pod="kube-system/konnectivity-agent-gv5z5" Sep 11 18:44:40.998221 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.997751 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9819d65-2122-497e-831c-917aa1fabd52-cnibin\") pod \"multus-additional-cni-plugins-9xtl5\" (UID: \"e9819d65-2122-497e-831c-917aa1fabd52\") " pod="openshift-multus/multus-additional-cni-plugins-9xtl5" Sep 11 18:44:40.998221 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.997909 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-multus-cni-dir\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.999070 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.997979 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-multus-socket-dir-parent\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.999070 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.998185 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-system-cni-dir\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.999070 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.998235 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-host-run-netns\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.999070 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.998350 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96d752b9-bf5e-4519-b2b8-6dbd832cab19-cni-binary-copy\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.999070 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.997722 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-b4q7q" Sep 11 18:44:40.999070 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.998469 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-host-var-lib-cni-multus\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.999070 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.998764 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-etc-kubernetes\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.999070 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.998909 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-host-var-lib-kubelet\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.999070 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.998936 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-run-ovn\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:40.999070 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.998957 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-multus-conf-dir\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.999070 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.999002 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-host-run-multus-certs\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.999070 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.999033 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b799f9cb-ce50-4731-9fbe-48f47807a053-original-pull-secret\") pod \"global-pull-secret-syncer-sxqg6\" (UID: \"b799f9cb-ce50-4731-9fbe-48f47807a053\") " pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:44:40.999070 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.999070 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/876bf078-f416-471c-b683-74781309a51a-host-slash\") pod \"iptables-alerter-nth2m\" (UID: \"876bf078-f416-471c-b683-74781309a51a\") " pod="openshift-network-operator/iptables-alerter-nth2m" Sep 11 18:44:40.999724 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.999098 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:40.999724 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.999130 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wtv7\" (UniqueName: \"kubernetes.io/projected/876bf078-f416-471c-b683-74781309a51a-kube-api-access-7wtv7\") pod \"iptables-alerter-nth2m\" (UID: \"876bf078-f416-471c-b683-74781309a51a\") " pod="openshift-network-operator/iptables-alerter-nth2m" Sep 11 18:44:40.999724 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:40.999142 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 18:44:40.999724 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.999155 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce-etc-selinux\") pod \"aws-ebs-csi-driver-node-m4xd6\" (UID: \"93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" Sep 11 18:44:40.999724 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.999180 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-etc-systemd\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:40.999724 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:40.999204 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b799f9cb-ce50-4731-9fbe-48f47807a053-original-pull-secret podName:b799f9cb-ce50-4731-9fbe-48f47807a053 nodeName:}" failed. No retries permitted until 2026-09-11 18:44:41.499183285 +0000 UTC m=+2.961789289 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b799f9cb-ce50-4731-9fbe-48f47807a053-original-pull-secret") pod "global-pull-secret-syncer-sxqg6" (UID: "b799f9cb-ce50-4731-9fbe-48f47807a053") : object "kube-system"/"original-pull-secret" not registered Sep 11 18:44:40.999724 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.999244 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/876bf078-f416-471c-b683-74781309a51a-host-slash\") pod \"iptables-alerter-nth2m\" (UID: \"876bf078-f416-471c-b683-74781309a51a\") " pod="openshift-network-operator/iptables-alerter-nth2m" Sep 11 18:44:40.999724 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.999389 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a688c734-cd8a-40eb-843f-d78e5ec411fd-env-overrides\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:40.999724 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.999417 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4kf4\" (UniqueName: \"kubernetes.io/projected/60dd726a-4169-4a95-ade1-c067f1851aac-kube-api-access-n4kf4\") pod \"node-resolver-7f246\" (UID: \"60dd726a-4169-4a95-ade1-c067f1851aac\") " pod="openshift-dns/node-resolver-7f246" Sep 11 18:44:40.999724 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.999443 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce-registration-dir\") pod \"aws-ebs-csi-driver-node-m4xd6\" (UID: \"93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" Sep 11 18:44:40.999724 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.999475 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b799f9cb-ce50-4731-9fbe-48f47807a053-kubelet-config\") pod \"global-pull-secret-syncer-sxqg6\" (UID: \"b799f9cb-ce50-4731-9fbe-48f47807a053\") " pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:44:40.999724 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.999500 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/96d752b9-bf5e-4519-b2b8-6dbd832cab19-multus-daemon-config\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:40.999724 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:40.999524 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce-sys-fs\") pod \"aws-ebs-csi-driver-node-m4xd6\" (UID: \"93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" Sep 11 18:44:41.000399 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.000088 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/96d752b9-bf5e-4519-b2b8-6dbd832cab19-multus-daemon-config\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:41.000399 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.000137 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-etc-openvswitch\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.000399 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.000167 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9819d65-2122-497e-831c-917aa1fabd52-system-cni-dir\") pod \"multus-additional-cni-plugins-9xtl5\" (UID: \"e9819d65-2122-497e-831c-917aa1fabd52\") " pod="openshift-multus/multus-additional-cni-plugins-9xtl5" Sep 11 18:44:41.000399 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.000194 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-host-cni-bin\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.000399 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.000241 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkbps\" (UniqueName: \"kubernetes.io/projected/274576da-90e4-47d9-b053-72498cd1ab3e-kube-api-access-nkbps\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.000399 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.000253 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b799f9cb-ce50-4731-9fbe-48f47807a053-kubelet-config\") pod \"global-pull-secret-syncer-sxqg6\" (UID: \"b799f9cb-ce50-4731-9fbe-48f47807a053\") " pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:44:41.000399 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.000281 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-etc-sysconfig\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.000399 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.000312 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-etc-sysctl-d\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.000399 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.000342 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhskz\" (UniqueName: \"kubernetes.io/projected/254d8445-a0f4-4c11-bc96-2cb74c34ab13-kube-api-access-jhskz\") pod \"network-check-target-rxmht\" (UID: \"254d8445-a0f4-4c11-bc96-2cb74c34ab13\") " pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:44:41.000399 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.000383 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b644d098-032f-42c9-9839-41ce9eb09844-metrics-certs\") pod \"network-metrics-daemon-lhj2z\" (UID: \"b644d098-032f-42c9-9839-41ce9eb09844\") " pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:44:41.000963 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.000407 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-host-run-netns\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.000963 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.000472 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-var-lib-openvswitch\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.000963 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.000499 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-host-cni-netd\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.000963 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.000530 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/876bf078-f416-471c-b683-74781309a51a-iptables-alerter-script\") pod \"iptables-alerter-nth2m\" (UID: \"876bf078-f416-471c-b683-74781309a51a\") " pod="openshift-network-operator/iptables-alerter-nth2m" Sep 11 18:44:41.000963 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.000556 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-host-run-k8s-cni-cncf-io\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:41.000963 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.000599 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-host-var-lib-cni-bin\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:41.000963 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.000626 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9819d65-2122-497e-831c-917aa1fabd52-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9xtl5\" (UID: \"e9819d65-2122-497e-831c-917aa1fabd52\") " pod="openshift-multus/multus-additional-cni-plugins-9xtl5" Sep 11 18:44:41.000963 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.000649 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-run\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.000963 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.000733 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-lib-modules\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.000963 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.000760 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-run-systemd\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.000963 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.000783 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-node-log\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.001459 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.001228 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-host-run-k8s-cni-cncf-io\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:41.001459 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.001280 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96d752b9-bf5e-4519-b2b8-6dbd832cab19-host-var-lib-cni-bin\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:41.003093 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.001935 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/876bf078-f416-471c-b683-74781309a51a-iptables-alerter-script\") pod \"iptables-alerter-nth2m\" (UID: \"876bf078-f416-471c-b683-74781309a51a\") " pod="openshift-network-operator/iptables-alerter-nth2m" Sep 11 18:44:41.013797 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.013764 2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 11 18:44:41.018482 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.018447 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zrxh\" (UniqueName: \"kubernetes.io/projected/96d752b9-bf5e-4519-b2b8-6dbd832cab19-kube-api-access-2zrxh\") pod \"multus-z8mxj\" (UID: \"96d752b9-bf5e-4519-b2b8-6dbd832cab19\") " pod="openshift-multus/multus-z8mxj" Sep 11 18:44:41.025199 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.025165 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wtv7\" (UniqueName: \"kubernetes.io/projected/876bf078-f416-471c-b683-74781309a51a-kube-api-access-7wtv7\") pod \"iptables-alerter-nth2m\" (UID: \"876bf078-f416-471c-b683-74781309a51a\") " pod="openshift-network-operator/iptables-alerter-nth2m" Sep 11 18:44:41.060029 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.059959 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-192.ec2.internal" event={"ID":"0430ff9a9376761ce7d088dd1f84ed06","Type":"ContainerStarted","Data":"7378e3c99cae1ae51055059f83fc40b4d504aba0e813949ca2a39a11d3a65ab9"} Sep 11 18:44:41.061547 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.061496 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-192.ec2.internal" event={"ID":"7241124c924c97f39a53ae6bf4dfdb02","Type":"ContainerStarted","Data":"8011e0b3ba4709af6ed302e36ec038922004687c6c9de83595a2d4f07273cee5"} Sep 11 18:44:41.101067 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101029 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s2jb\" (UniqueName: \"kubernetes.io/projected/e9819d65-2122-497e-831c-917aa1fabd52-kube-api-access-5s2jb\") pod \"multus-additional-cni-plugins-9xtl5\" (UID: \"e9819d65-2122-497e-831c-917aa1fabd52\") " pod="openshift-multus/multus-additional-cni-plugins-9xtl5" Sep 11 18:44:41.101239 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101103 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-etc-sysctl-conf\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.101239 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101132 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/60dd726a-4169-4a95-ade1-c067f1851aac-hosts-file\") pod \"node-resolver-7f246\" (UID: \"60dd726a-4169-4a95-ade1-c067f1851aac\") " pod="openshift-dns/node-resolver-7f246" Sep 11 18:44:41.101239 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101159 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-etc-kubernetes\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.101239 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101184 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-sys\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.101239 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101209 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6h8l\" (UniqueName: \"kubernetes.io/projected/b644d098-032f-42c9-9839-41ce9eb09844-kube-api-access-x6h8l\") pod \"network-metrics-daemon-lhj2z\" (UID: \"b644d098-032f-42c9-9839-41ce9eb09844\") " pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:44:41.101239 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101233 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-log-socket\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.101578 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101258 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60dd726a-4169-4a95-ade1-c067f1851aac-tmp-dir\") pod \"node-resolver-7f246\" (UID: \"60dd726a-4169-4a95-ade1-c067f1851aac\") " pod="openshift-dns/node-resolver-7f246" Sep 11 18:44:41.101578 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101255 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/60dd726a-4169-4a95-ade1-c067f1851aac-hosts-file\") pod \"node-resolver-7f246\" (UID: \"60dd726a-4169-4a95-ade1-c067f1851aac\") " pod="openshift-dns/node-resolver-7f246" Sep 11 18:44:41.101578 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101273 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-etc-kubernetes\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.101578 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101283 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/22ef1e56-f6b2-4ebf-8be0-63669535237b-agent-certs\") pod \"konnectivity-agent-gv5z5\" (UID: \"22ef1e56-f6b2-4ebf-8be0-63669535237b\") " pod="kube-system/konnectivity-agent-gv5z5" Sep 11 18:44:41.101578 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101298 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-etc-sysctl-conf\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.101578 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101329 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9819d65-2122-497e-831c-917aa1fabd52-cnibin\") pod \"multus-additional-cni-plugins-9xtl5\" (UID: \"e9819d65-2122-497e-831c-917aa1fabd52\") " pod="openshift-multus/multus-additional-cni-plugins-9xtl5" Sep 11 18:44:41.101578 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101369 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-run-ovn\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.101578 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101425 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.101578 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101449 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-sys\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.101578 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101456 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce-etc-selinux\") pod \"aws-ebs-csi-driver-node-m4xd6\" (UID: \"93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" Sep 11 18:44:41.101578 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101461 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-run-ovn\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.101578 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101424 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9819d65-2122-497e-831c-917aa1fabd52-cnibin\") pod \"multus-additional-cni-plugins-9xtl5\" (UID: \"e9819d65-2122-497e-831c-917aa1fabd52\") " pod="openshift-multus/multus-additional-cni-plugins-9xtl5" Sep 11 18:44:41.101578 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101442 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-log-socket\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.101578 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101492 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.101578 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101574 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-etc-systemd\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.102426 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101596 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce-etc-selinux\") pod \"aws-ebs-csi-driver-node-m4xd6\" (UID: \"93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" Sep 11 18:44:41.102426 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101623 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a688c734-cd8a-40eb-843f-d78e5ec411fd-env-overrides\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.102426 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101633 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-etc-systemd\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.102426 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101658 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4kf4\" (UniqueName: \"kubernetes.io/projected/60dd726a-4169-4a95-ade1-c067f1851aac-kube-api-access-n4kf4\") pod \"node-resolver-7f246\" (UID: \"60dd726a-4169-4a95-ade1-c067f1851aac\") " pod="openshift-dns/node-resolver-7f246" Sep 11 18:44:41.102426 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101685 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce-registration-dir\") pod \"aws-ebs-csi-driver-node-m4xd6\" (UID: \"93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" Sep 11 18:44:41.102426 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101715 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce-sys-fs\") pod \"aws-ebs-csi-driver-node-m4xd6\" (UID: \"93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" Sep 11 18:44:41.102426 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101773 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce-sys-fs\") pod \"aws-ebs-csi-driver-node-m4xd6\" (UID: \"93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" Sep 11 18:44:41.102426 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101847 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce-registration-dir\") pod \"aws-ebs-csi-driver-node-m4xd6\" (UID: \"93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" Sep 11 18:44:41.102426 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101854 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60dd726a-4169-4a95-ade1-c067f1851aac-tmp-dir\") pod \"node-resolver-7f246\" (UID: \"60dd726a-4169-4a95-ade1-c067f1851aac\") " pod="openshift-dns/node-resolver-7f246" Sep 11 18:44:41.102426 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101881 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-etc-openvswitch\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.102426 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101923 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9819d65-2122-497e-831c-917aa1fabd52-system-cni-dir\") pod \"multus-additional-cni-plugins-9xtl5\" (UID: \"e9819d65-2122-497e-831c-917aa1fabd52\") " pod="openshift-multus/multus-additional-cni-plugins-9xtl5" Sep 11 18:44:41.102426 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101977 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-host-cni-bin\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.102426 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.101988 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9819d65-2122-497e-831c-917aa1fabd52-system-cni-dir\") pod \"multus-additional-cni-plugins-9xtl5\" (UID: \"e9819d65-2122-497e-831c-917aa1fabd52\") " pod="openshift-multus/multus-additional-cni-plugins-9xtl5" Sep 11 18:44:41.102426 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102020 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-etc-openvswitch\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.102426 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102038 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a688c734-cd8a-40eb-843f-d78e5ec411fd-env-overrides\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.102426 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102066 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkbps\" (UniqueName: \"kubernetes.io/projected/274576da-90e4-47d9-b053-72498cd1ab3e-kube-api-access-nkbps\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.102426 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102090 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-host-cni-bin\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.103192 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102125 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-etc-sysconfig\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.103192 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102251 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-etc-sysctl-d\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.103192 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102268 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-etc-sysconfig\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.103192 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102303 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhskz\" (UniqueName: \"kubernetes.io/projected/254d8445-a0f4-4c11-bc96-2cb74c34ab13-kube-api-access-jhskz\") pod \"network-check-target-rxmht\" (UID: \"254d8445-a0f4-4c11-bc96-2cb74c34ab13\") " pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:44:41.103192 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102347 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b644d098-032f-42c9-9839-41ce9eb09844-metrics-certs\") pod \"network-metrics-daemon-lhj2z\" (UID: \"b644d098-032f-42c9-9839-41ce9eb09844\") " pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:44:41.103192 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102379 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-host-run-netns\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.103192 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102388 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-etc-sysctl-d\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.103192 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:41.102459 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 18:44:41.103192 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102517 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-var-lib-openvswitch\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.103192 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:41.102540 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b644d098-032f-42c9-9839-41ce9eb09844-metrics-certs podName:b644d098-032f-42c9-9839-41ce9eb09844 nodeName:}" failed. No retries permitted until 2026-09-11 18:44:41.60250031 +0000 UTC m=+3.065106315 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b644d098-032f-42c9-9839-41ce9eb09844-metrics-certs") pod "network-metrics-daemon-lhj2z" (UID: "b644d098-032f-42c9-9839-41ce9eb09844") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 18:44:41.103192 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102574 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-host-cni-netd\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.103192 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102580 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-host-run-netns\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.103192 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102607 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9819d65-2122-497e-831c-917aa1fabd52-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9xtl5\" (UID: \"e9819d65-2122-497e-831c-917aa1fabd52\") " pod="openshift-multus/multus-additional-cni-plugins-9xtl5" Sep 11 18:44:41.103192 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102634 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-run\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.103192 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102655 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-host-cni-netd\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.103192 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102661 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-lib-modules\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.103192 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102658 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-var-lib-openvswitch\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.104120 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102688 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-run-systemd\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.104120 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102714 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-node-log\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.104120 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102739 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-run\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.104120 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102739 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-run-systemd\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.104120 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102744 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-host-kubelet\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.104120 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102781 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-node-log\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.104120 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102781 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-host-kubelet\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.104120 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102795 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a688c734-cd8a-40eb-843f-d78e5ec411fd-ovn-node-metrics-cert\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.104120 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102830 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a688c734-cd8a-40eb-843f-d78e5ec411fd-ovnkube-script-lib\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.104120 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102858 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-lib-modules\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.104120 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102891 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce-socket-dir\") pod \"aws-ebs-csi-driver-node-m4xd6\" (UID: \"93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" Sep 11 18:44:41.104120 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102917 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/254dd733-80bd-4f2a-9fbc-a57622608b32-host\") pod \"node-ca-tnjsf\" (UID: \"254dd733-80bd-4f2a-9fbc-a57622608b32\") " pod="openshift-image-registry/node-ca-tnjsf" Sep 11 18:44:41.104120 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102944 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9819d65-2122-497e-831c-917aa1fabd52-cni-binary-copy\") pod \"multus-additional-cni-plugins-9xtl5\" (UID: \"e9819d65-2122-497e-831c-917aa1fabd52\") " pod="openshift-multus/multus-additional-cni-plugins-9xtl5" Sep 11 18:44:41.104120 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.102972 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-var-lib-kubelet\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.104120 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.103006 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce-device-dir\") pod \"aws-ebs-csi-driver-node-m4xd6\" (UID: \"93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" Sep 11 18:44:41.104120 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.103028 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/254dd733-80bd-4f2a-9fbc-a57622608b32-serviceca\") pod \"node-ca-tnjsf\" (UID: \"254dd733-80bd-4f2a-9fbc-a57622608b32\") " pod="openshift-image-registry/node-ca-tnjsf" Sep 11 18:44:41.104120 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.103055 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9819d65-2122-497e-831c-917aa1fabd52-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9xtl5\" (UID: \"e9819d65-2122-497e-831c-917aa1fabd52\") " pod="openshift-multus/multus-additional-cni-plugins-9xtl5" Sep 11 18:44:41.105012 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.103083 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/274576da-90e4-47d9-b053-72498cd1ab3e-etc-tuned\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.105012 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.103108 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/274576da-90e4-47d9-b053-72498cd1ab3e-tmp\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.105012 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.103150 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-systemd-units\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.105012 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.103175 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-host-run-ovn-kubernetes\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.105012 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.103203 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgw2d\" (UniqueName: \"kubernetes.io/projected/a688c734-cd8a-40eb-843f-d78e5ec411fd-kube-api-access-bgw2d\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.105012 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.103237 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-etc-modprobe-d\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.105012 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.103261 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-host\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.105012 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.103286 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-host-slash\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.105012 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.103312 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m4xd6\" (UID: \"93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" Sep 11 18:44:41.105012 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.103338 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9819d65-2122-497e-831c-917aa1fabd52-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9xtl5\" (UID: \"e9819d65-2122-497e-831c-917aa1fabd52\") " pod="openshift-multus/multus-additional-cni-plugins-9xtl5" Sep 11 18:44:41.105012 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.103371 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-run-openvswitch\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.105012 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.103385 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9819d65-2122-497e-831c-917aa1fabd52-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9xtl5\" (UID: \"e9819d65-2122-497e-831c-917aa1fabd52\") " pod="openshift-multus/multus-additional-cni-plugins-9xtl5" Sep 11 18:44:41.105012 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.103396 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a688c734-cd8a-40eb-843f-d78e5ec411fd-ovnkube-config\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.105012 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.103430 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/22ef1e56-f6b2-4ebf-8be0-63669535237b-konnectivity-ca\") pod \"konnectivity-agent-gv5z5\" (UID: \"22ef1e56-f6b2-4ebf-8be0-63669535237b\") " pod="kube-system/konnectivity-agent-gv5z5" Sep 11 18:44:41.105012 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.103464 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dp74\" (UniqueName: \"kubernetes.io/projected/93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce-kube-api-access-8dp74\") pod \"aws-ebs-csi-driver-node-m4xd6\" (UID: \"93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" Sep 11 18:44:41.105012 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.103491 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjqb8\" (UniqueName: \"kubernetes.io/projected/254dd733-80bd-4f2a-9fbc-a57622608b32-kube-api-access-xjqb8\") pod \"node-ca-tnjsf\" (UID: \"254dd733-80bd-4f2a-9fbc-a57622608b32\") " pod="openshift-image-registry/node-ca-tnjsf" Sep 11 18:44:41.105012 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.103651 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/254dd733-80bd-4f2a-9fbc-a57622608b32-serviceca\") pod \"node-ca-tnjsf\" (UID: \"254dd733-80bd-4f2a-9fbc-a57622608b32\") " pod="openshift-image-registry/node-ca-tnjsf" Sep 11 18:44:41.105767 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.103721 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-var-lib-kubelet\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.105767 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.103778 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce-device-dir\") pod \"aws-ebs-csi-driver-node-m4xd6\" (UID: \"93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" Sep 11 18:44:41.105767 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.103910 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a688c734-cd8a-40eb-843f-d78e5ec411fd-ovnkube-config\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.105767 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.103926 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9819d65-2122-497e-831c-917aa1fabd52-cni-binary-copy\") pod \"multus-additional-cni-plugins-9xtl5\" (UID: \"e9819d65-2122-497e-831c-917aa1fabd52\") " pod="openshift-multus/multus-additional-cni-plugins-9xtl5" Sep 11 18:44:41.105767 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.104025 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-etc-modprobe-d\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.105767 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.104043 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce-socket-dir\") pod \"aws-ebs-csi-driver-node-m4xd6\" (UID: \"93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" Sep 11 18:44:41.105767 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.104078 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/274576da-90e4-47d9-b053-72498cd1ab3e-host\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.105767 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.104088 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/254dd733-80bd-4f2a-9fbc-a57622608b32-host\") pod \"node-ca-tnjsf\" (UID: \"254dd733-80bd-4f2a-9fbc-a57622608b32\") " pod="openshift-image-registry/node-ca-tnjsf" Sep 11 18:44:41.105767 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.103431 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a688c734-cd8a-40eb-843f-d78e5ec411fd-ovnkube-script-lib\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.105767 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.104125 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-host-slash\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.105767 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.104178 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m4xd6\" (UID: \"93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" Sep 11 18:44:41.105767 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.104341 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9819d65-2122-497e-831c-917aa1fabd52-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9xtl5\" (UID: \"e9819d65-2122-497e-831c-917aa1fabd52\") " pod="openshift-multus/multus-additional-cni-plugins-9xtl5" Sep 11 18:44:41.105767 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.104394 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-run-openvswitch\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.105767 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.104399 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/22ef1e56-f6b2-4ebf-8be0-63669535237b-agent-certs\") pod \"konnectivity-agent-gv5z5\" (UID: \"22ef1e56-f6b2-4ebf-8be0-63669535237b\") " pod="kube-system/konnectivity-agent-gv5z5" Sep 11 18:44:41.105767 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.104445 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-systemd-units\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.105767 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.104487 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a688c734-cd8a-40eb-843f-d78e5ec411fd-host-run-ovn-kubernetes\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.105767 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.104622 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9819d65-2122-497e-831c-917aa1fabd52-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9xtl5\" (UID: \"e9819d65-2122-497e-831c-917aa1fabd52\") " pod="openshift-multus/multus-additional-cni-plugins-9xtl5" Sep 11 18:44:41.106496 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.105014 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/22ef1e56-f6b2-4ebf-8be0-63669535237b-konnectivity-ca\") pod \"konnectivity-agent-gv5z5\" (UID: \"22ef1e56-f6b2-4ebf-8be0-63669535237b\") " pod="kube-system/konnectivity-agent-gv5z5" Sep 11 18:44:41.106879 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.106855 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a688c734-cd8a-40eb-843f-d78e5ec411fd-ovn-node-metrics-cert\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.106988 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.106934 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/274576da-90e4-47d9-b053-72498cd1ab3e-etc-tuned\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.107181 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.107157 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/274576da-90e4-47d9-b053-72498cd1ab3e-tmp\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.119857 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.119813 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s2jb\" (UniqueName: \"kubernetes.io/projected/e9819d65-2122-497e-831c-917aa1fabd52-kube-api-access-5s2jb\") pod \"multus-additional-cni-plugins-9xtl5\" (UID: \"e9819d65-2122-497e-831c-917aa1fabd52\") " pod="openshift-multus/multus-additional-cni-plugins-9xtl5" Sep 11 18:44:41.120316 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.120203 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6h8l\" (UniqueName: \"kubernetes.io/projected/b644d098-032f-42c9-9839-41ce9eb09844-kube-api-access-x6h8l\") pod \"network-metrics-daemon-lhj2z\" (UID: \"b644d098-032f-42c9-9839-41ce9eb09844\") " pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:44:41.120636 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:41.120615 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 18:44:41.120722 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:41.120639 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 18:44:41.120722 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:41.120650 2563 projected.go:196] Error preparing data for projected volume kube-api-access-jhskz for pod openshift-network-diagnostics/network-check-target-rxmht: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 18:44:41.120821 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:41.120727 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/254d8445-a0f4-4c11-bc96-2cb74c34ab13-kube-api-access-jhskz podName:254d8445-a0f4-4c11-bc96-2cb74c34ab13 nodeName:}" failed. No retries permitted until 2026-09-11 18:44:41.620706344 +0000 UTC m=+3.083312347 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jhskz" (UniqueName: "kubernetes.io/projected/254d8445-a0f4-4c11-bc96-2cb74c34ab13-kube-api-access-jhskz") pod "network-check-target-rxmht" (UID: "254d8445-a0f4-4c11-bc96-2cb74c34ab13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 18:44:41.138051 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.138009 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4kf4\" (UniqueName: \"kubernetes.io/projected/60dd726a-4169-4a95-ade1-c067f1851aac-kube-api-access-n4kf4\") pod \"node-resolver-7f246\" (UID: \"60dd726a-4169-4a95-ade1-c067f1851aac\") " pod="openshift-dns/node-resolver-7f246" Sep 11 18:44:41.140104 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.140079 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjqb8\" (UniqueName: \"kubernetes.io/projected/254dd733-80bd-4f2a-9fbc-a57622608b32-kube-api-access-xjqb8\") pod \"node-ca-tnjsf\" (UID: \"254dd733-80bd-4f2a-9fbc-a57622608b32\") " pod="openshift-image-registry/node-ca-tnjsf" Sep 11 18:44:41.144727 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.144702 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgw2d\" (UniqueName: \"kubernetes.io/projected/a688c734-cd8a-40eb-843f-d78e5ec411fd-kube-api-access-bgw2d\") pod \"ovnkube-node-8tmtk\" (UID: \"a688c734-cd8a-40eb-843f-d78e5ec411fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.150160 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.150134 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkbps\" (UniqueName: \"kubernetes.io/projected/274576da-90e4-47d9-b053-72498cd1ab3e-kube-api-access-nkbps\") pod \"tuned-fdn85\" (UID: \"274576da-90e4-47d9-b053-72498cd1ab3e\") " pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.150359 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.150340 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dp74\" (UniqueName: \"kubernetes.io/projected/93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce-kube-api-access-8dp74\") pod \"aws-ebs-csi-driver-node-m4xd6\" (UID: \"93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" Sep 11 18:44:41.197324 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.197243 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-z8mxj" Sep 11 18:44:41.207805 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.207768 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nth2m" Sep 11 18:44:41.220486 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.220458 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" Sep 11 18:44:41.226881 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.226860 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tnjsf" Sep 11 18:44:41.235716 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.235691 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9xtl5" Sep 11 18:44:41.243935 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.243912 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fdn85" Sep 11 18:44:41.254655 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.254625 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:44:41.265241 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.265208 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7f246" Sep 11 18:44:41.272032 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.272008 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gv5z5" Sep 11 18:44:41.507116 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.507027 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b799f9cb-ce50-4731-9fbe-48f47807a053-original-pull-secret\") pod \"global-pull-secret-syncer-sxqg6\" (UID: \"b799f9cb-ce50-4731-9fbe-48f47807a053\") " pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:44:41.507282 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:41.507151 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 18:44:41.507282 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:41.507226 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b799f9cb-ce50-4731-9fbe-48f47807a053-original-pull-secret podName:b799f9cb-ce50-4731-9fbe-48f47807a053 nodeName:}" failed. No retries permitted until 2026-09-11 18:44:42.507204706 +0000 UTC m=+3.969810764 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b799f9cb-ce50-4731-9fbe-48f47807a053-original-pull-secret") pod "global-pull-secret-syncer-sxqg6" (UID: "b799f9cb-ce50-4731-9fbe-48f47807a053") : object "kube-system"/"original-pull-secret" not registered Sep 11 18:44:41.608139 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.608106 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b644d098-032f-42c9-9839-41ce9eb09844-metrics-certs\") pod \"network-metrics-daemon-lhj2z\" (UID: \"b644d098-032f-42c9-9839-41ce9eb09844\") " pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:44:41.608380 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:41.608225 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 18:44:41.608380 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:41.608297 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b644d098-032f-42c9-9839-41ce9eb09844-metrics-certs podName:b644d098-032f-42c9-9839-41ce9eb09844 nodeName:}" failed. No retries permitted until 2026-09-11 18:44:42.608276123 +0000 UTC m=+4.070882151 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b644d098-032f-42c9-9839-41ce9eb09844-metrics-certs") pod "network-metrics-daemon-lhj2z" (UID: "b644d098-032f-42c9-9839-41ce9eb09844") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 18:44:41.613569 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.613355 2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 18:44:41.709007 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:41.708970 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhskz\" (UniqueName: \"kubernetes.io/projected/254d8445-a0f4-4c11-bc96-2cb74c34ab13-kube-api-access-jhskz\") pod \"network-check-target-rxmht\" (UID: \"254d8445-a0f4-4c11-bc96-2cb74c34ab13\") " pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:44:41.709193 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:41.709153 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 18:44:41.709193 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:41.709174 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 18:44:41.709193 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:41.709189 2563 projected.go:196] Error preparing data for projected volume kube-api-access-jhskz for pod openshift-network-diagnostics/network-check-target-rxmht: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 18:44:41.709368 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:41.709254 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/254d8445-a0f4-4c11-bc96-2cb74c34ab13-kube-api-access-jhskz podName:254d8445-a0f4-4c11-bc96-2cb74c34ab13 nodeName:}" failed. No retries permitted until 2026-09-11 18:44:42.709233518 +0000 UTC m=+4.171839515 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jhskz" (UniqueName: "kubernetes.io/projected/254d8445-a0f4-4c11-bc96-2cb74c34ab13-kube-api-access-jhskz") pod "network-check-target-rxmht" (UID: "254d8445-a0f4-4c11-bc96-2cb74c34ab13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 18:44:42.005095 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:42.004958 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 18:39:41 +0000 UTC" deadline="2028-05-23 10:48:13.484484926 +0000 UTC" Sep 11 18:44:42.005095 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:42.005033 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14872h3m31.479455392s" Sep 11 18:44:42.219639 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:42.219543 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod254dd733_80bd_4f2a_9fbc_a57622608b32.slice/crio-7e92b85a244e31da47ffc4d1e0f4b82182ebcfd255cfe327e67a9ae3f57c2b0d WatchSource:0}: Error finding container 7e92b85a244e31da47ffc4d1e0f4b82182ebcfd255cfe327e67a9ae3f57c2b0d: Status 404 returned error can't find the container with id 7e92b85a244e31da47ffc4d1e0f4b82182ebcfd255cfe327e67a9ae3f57c2b0d Sep 11 18:44:42.221488 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:42.221449 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60dd726a_4169_4a95_ade1_c067f1851aac.slice/crio-0df113d817bc9843c0fa9c0547f083671dfef3c3f9765a515b4b2d41436136a0 WatchSource:0}: Error finding container 0df113d817bc9843c0fa9c0547f083671dfef3c3f9765a515b4b2d41436136a0: Status 404 returned error can't find the container with id 0df113d817bc9843c0fa9c0547f083671dfef3c3f9765a515b4b2d41436136a0 Sep 11 18:44:42.225106 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:42.225082 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda688c734_cd8a_40eb_843f_d78e5ec411fd.slice/crio-511be6a52549a6b7d95c9687de94d3723f80e72b8586288d32cb1fe80dc4cff2 WatchSource:0}: Error finding container 511be6a52549a6b7d95c9687de94d3723f80e72b8586288d32cb1fe80dc4cff2: Status 404 returned error can't find the container with id 511be6a52549a6b7d95c9687de94d3723f80e72b8586288d32cb1fe80dc4cff2 Sep 11 18:44:42.247584 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:42.247554 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22ef1e56_f6b2_4ebf_8be0_63669535237b.slice/crio-5e1ed3425c451fadf8374caea684a0f6e04d51bf683557a90f72fd3627566d38 WatchSource:0}: Error finding container 5e1ed3425c451fadf8374caea684a0f6e04d51bf683557a90f72fd3627566d38: Status 404 returned error can't find the container with id 5e1ed3425c451fadf8374caea684a0f6e04d51bf683557a90f72fd3627566d38 Sep 11 18:44:42.248718 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:42.248695 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9819d65_2122_497e_831c_917aa1fabd52.slice/crio-18b0ae75df2da97fd37aad57bd4f664d5de1f35b61405124bf1ccf3d4350e6fc WatchSource:0}: Error finding container 18b0ae75df2da97fd37aad57bd4f664d5de1f35b61405124bf1ccf3d4350e6fc: Status 404 returned error can't find the container with id 18b0ae75df2da97fd37aad57bd4f664d5de1f35b61405124bf1ccf3d4350e6fc Sep 11 18:44:42.250141 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:42.250114 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96d752b9_bf5e_4519_b2b8_6dbd832cab19.slice/crio-e6efe595592d5c3167b0e40bb9fcdf17751055a6a0ecf0ccdc10e8c0931495b1 WatchSource:0}: Error finding container e6efe595592d5c3167b0e40bb9fcdf17751055a6a0ecf0ccdc10e8c0931495b1: Status 404 returned error can't find the container with id e6efe595592d5c3167b0e40bb9fcdf17751055a6a0ecf0ccdc10e8c0931495b1 Sep 11 18:44:42.251564 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:42.251482 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod876bf078_f416_471c_b683_74781309a51a.slice/crio-e856ffee4c816a1a7d434dd252e1ce3a648919e1b834e6eaa4d29e523303d002 WatchSource:0}: Error finding container e856ffee4c816a1a7d434dd252e1ce3a648919e1b834e6eaa4d29e523303d002: Status 404 returned error can't find the container with id e856ffee4c816a1a7d434dd252e1ce3a648919e1b834e6eaa4d29e523303d002 Sep 11 18:44:42.252629 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:42.252586 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93e0fb88_ab2b_4842_8c8a_fc4bd3e0e8ce.slice/crio-50ba6c992e81be31f8862c95ee6d4568828145da60eac66d2d49be42096bfdb5 WatchSource:0}: Error finding container 50ba6c992e81be31f8862c95ee6d4568828145da60eac66d2d49be42096bfdb5: Status 404 returned error can't find the container with id 50ba6c992e81be31f8862c95ee6d4568828145da60eac66d2d49be42096bfdb5 Sep 11 18:44:42.253361 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:44:42.253334 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod274576da_90e4_47d9_b053_72498cd1ab3e.slice/crio-6ebd0c005c4ef795b629d33f03bfd29eb90801a751615d975dd75ac087e17ab1 WatchSource:0}: Error finding container 6ebd0c005c4ef795b629d33f03bfd29eb90801a751615d975dd75ac087e17ab1: Status 404 returned error can't find the container with id 6ebd0c005c4ef795b629d33f03bfd29eb90801a751615d975dd75ac087e17ab1 Sep 11 18:44:42.515567 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:42.515412 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b799f9cb-ce50-4731-9fbe-48f47807a053-original-pull-secret\") pod \"global-pull-secret-syncer-sxqg6\" (UID: \"b799f9cb-ce50-4731-9fbe-48f47807a053\") " pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:44:42.515696 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:42.515579 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 18:44:42.515696 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:42.515657 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b799f9cb-ce50-4731-9fbe-48f47807a053-original-pull-secret podName:b799f9cb-ce50-4731-9fbe-48f47807a053 nodeName:}" failed. No retries permitted until 2026-09-11 18:44:44.515630552 +0000 UTC m=+5.978236566 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b799f9cb-ce50-4731-9fbe-48f47807a053-original-pull-secret") pod "global-pull-secret-syncer-sxqg6" (UID: "b799f9cb-ce50-4731-9fbe-48f47807a053") : object "kube-system"/"original-pull-secret" not registered Sep 11 18:44:42.615896 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:42.615849 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b644d098-032f-42c9-9839-41ce9eb09844-metrics-certs\") pod \"network-metrics-daemon-lhj2z\" (UID: \"b644d098-032f-42c9-9839-41ce9eb09844\") " pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:44:42.616077 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:42.616014 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 18:44:42.616128 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:42.616098 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b644d098-032f-42c9-9839-41ce9eb09844-metrics-certs podName:b644d098-032f-42c9-9839-41ce9eb09844 nodeName:}" failed. No retries permitted until 2026-09-11 18:44:44.616081597 +0000 UTC m=+6.078687599 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b644d098-032f-42c9-9839-41ce9eb09844-metrics-certs") pod "network-metrics-daemon-lhj2z" (UID: "b644d098-032f-42c9-9839-41ce9eb09844") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 18:44:42.716946 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:42.716909 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhskz\" (UniqueName: \"kubernetes.io/projected/254d8445-a0f4-4c11-bc96-2cb74c34ab13-kube-api-access-jhskz\") pod \"network-check-target-rxmht\" (UID: \"254d8445-a0f4-4c11-bc96-2cb74c34ab13\") " pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:44:42.717187 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:42.717116 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 18:44:42.717187 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:42.717148 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 18:44:42.717187 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:42.717163 2563 projected.go:196] Error preparing data for projected volume kube-api-access-jhskz for pod openshift-network-diagnostics/network-check-target-rxmht: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 18:44:42.717343 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:42.717235 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/254d8445-a0f4-4c11-bc96-2cb74c34ab13-kube-api-access-jhskz podName:254d8445-a0f4-4c11-bc96-2cb74c34ab13 nodeName:}" failed. No retries permitted until 2026-09-11 18:44:44.717214224 +0000 UTC m=+6.179820228 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jhskz" (UniqueName: "kubernetes.io/projected/254d8445-a0f4-4c11-bc96-2cb74c34ab13-kube-api-access-jhskz") pod "network-check-target-rxmht" (UID: "254d8445-a0f4-4c11-bc96-2cb74c34ab13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 18:44:43.005288 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:43.005213 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 18:39:41 +0000 UTC" deadline="2028-02-25 06:48:26.953353525 +0000 UTC" Sep 11 18:44:43.005288 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:43.005256 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12756h3m43.948102519s" Sep 11 18:44:43.057101 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:43.057067 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:44:43.057272 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:43.057198 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxqg6" podUID="b799f9cb-ce50-4731-9fbe-48f47807a053" Sep 11 18:44:43.057751 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:43.057624 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:44:43.057751 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:43.057727 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxmht" podUID="254d8445-a0f4-4c11-bc96-2cb74c34ab13" Sep 11 18:44:43.057909 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:43.057795 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:44:43.057963 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:43.057939 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lhj2z" podUID="b644d098-032f-42c9-9839-41ce9eb09844" Sep 11 18:44:43.096453 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:43.095789 2563 generic.go:356] "Generic (PLEG): container finished" podID="0430ff9a9376761ce7d088dd1f84ed06" containerID="9a7615625f40b59a6bcc1742b16f66d8560fd9a7a892b46280c0dee13048a459" exitCode=0 Sep 11 18:44:43.096453 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:43.095928 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-192.ec2.internal" event={"ID":"0430ff9a9376761ce7d088dd1f84ed06","Type":"ContainerDied","Data":"9a7615625f40b59a6bcc1742b16f66d8560fd9a7a892b46280c0dee13048a459"} Sep 11 18:44:43.123926 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:43.123892 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-192.ec2.internal" event={"ID":"7241124c924c97f39a53ae6bf4dfdb02","Type":"ContainerStarted","Data":"3fb5bfb43425076125e78ffa7f84e373525f9997b77cee8aeba681982af5468f"} Sep 11 18:44:43.138364 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:43.138325 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fdn85" event={"ID":"274576da-90e4-47d9-b053-72498cd1ab3e","Type":"ContainerStarted","Data":"6ebd0c005c4ef795b629d33f03bfd29eb90801a751615d975dd75ac087e17ab1"} Sep 11 18:44:43.144097 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:43.144061 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nth2m" event={"ID":"876bf078-f416-471c-b683-74781309a51a","Type":"ContainerStarted","Data":"e856ffee4c816a1a7d434dd252e1ce3a648919e1b834e6eaa4d29e523303d002"} Sep 11 18:44:43.156806 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:43.154896 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" event={"ID":"93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce","Type":"ContainerStarted","Data":"50ba6c992e81be31f8862c95ee6d4568828145da60eac66d2d49be42096bfdb5"} Sep 11 18:44:43.158222 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:43.158194 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-z8mxj" event={"ID":"96d752b9-bf5e-4519-b2b8-6dbd832cab19","Type":"ContainerStarted","Data":"e6efe595592d5c3167b0e40bb9fcdf17751055a6a0ecf0ccdc10e8c0931495b1"} Sep 11 18:44:43.164817 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:43.164787 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9xtl5" event={"ID":"e9819d65-2122-497e-831c-917aa1fabd52","Type":"ContainerStarted","Data":"18b0ae75df2da97fd37aad57bd4f664d5de1f35b61405124bf1ccf3d4350e6fc"} Sep 11 18:44:43.202866 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:43.202693 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" event={"ID":"a688c734-cd8a-40eb-843f-d78e5ec411fd","Type":"ContainerStarted","Data":"511be6a52549a6b7d95c9687de94d3723f80e72b8586288d32cb1fe80dc4cff2"} Sep 11 18:44:43.224678 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:43.224522 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gv5z5" event={"ID":"22ef1e56-f6b2-4ebf-8be0-63669535237b","Type":"ContainerStarted","Data":"5e1ed3425c451fadf8374caea684a0f6e04d51bf683557a90f72fd3627566d38"} Sep 11 18:44:43.244330 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:43.244290 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7f246" event={"ID":"60dd726a-4169-4a95-ade1-c067f1851aac","Type":"ContainerStarted","Data":"0df113d817bc9843c0fa9c0547f083671dfef3c3f9765a515b4b2d41436136a0"} Sep 11 18:44:43.250290 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:43.250110 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tnjsf" event={"ID":"254dd733-80bd-4f2a-9fbc-a57622608b32","Type":"ContainerStarted","Data":"7e92b85a244e31da47ffc4d1e0f4b82182ebcfd255cfe327e67a9ae3f57c2b0d"} Sep 11 18:44:44.266749 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:44.266663 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-192.ec2.internal" event={"ID":"0430ff9a9376761ce7d088dd1f84ed06","Type":"ContainerStarted","Data":"d5084d14d64500d93eaf8cb078935df98d75a488976a669749906c7574cf15d9"} Sep 11 18:44:44.285248 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:44.284294 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-192.ec2.internal" podStartSLOduration=4.284272449 podStartE2EDuration="4.284272449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:44:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:44:43.141233706 +0000 UTC m=+4.603839728" watchObservedRunningTime="2026-09-11 18:44:44.284272449 +0000 UTC m=+5.746878471" Sep 11 18:44:44.532254 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:44.532154 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b799f9cb-ce50-4731-9fbe-48f47807a053-original-pull-secret\") pod \"global-pull-secret-syncer-sxqg6\" (UID: \"b799f9cb-ce50-4731-9fbe-48f47807a053\") " pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:44:44.532443 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:44.532388 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 18:44:44.532503 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:44.532458 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b799f9cb-ce50-4731-9fbe-48f47807a053-original-pull-secret podName:b799f9cb-ce50-4731-9fbe-48f47807a053 nodeName:}" failed. No retries permitted until 2026-09-11 18:44:48.532438547 +0000 UTC m=+9.995044573 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b799f9cb-ce50-4731-9fbe-48f47807a053-original-pull-secret") pod "global-pull-secret-syncer-sxqg6" (UID: "b799f9cb-ce50-4731-9fbe-48f47807a053") : object "kube-system"/"original-pull-secret" not registered Sep 11 18:44:44.632950 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:44.632908 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b644d098-032f-42c9-9839-41ce9eb09844-metrics-certs\") pod \"network-metrics-daemon-lhj2z\" (UID: \"b644d098-032f-42c9-9839-41ce9eb09844\") " pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:44:44.633156 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:44.633136 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 18:44:44.633225 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:44.633209 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b644d098-032f-42c9-9839-41ce9eb09844-metrics-certs podName:b644d098-032f-42c9-9839-41ce9eb09844 nodeName:}" failed. No retries permitted until 2026-09-11 18:44:48.63318762 +0000 UTC m=+10.095793618 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b644d098-032f-42c9-9839-41ce9eb09844-metrics-certs") pod "network-metrics-daemon-lhj2z" (UID: "b644d098-032f-42c9-9839-41ce9eb09844") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 18:44:44.734249 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:44.733910 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhskz\" (UniqueName: \"kubernetes.io/projected/254d8445-a0f4-4c11-bc96-2cb74c34ab13-kube-api-access-jhskz\") pod \"network-check-target-rxmht\" (UID: \"254d8445-a0f4-4c11-bc96-2cb74c34ab13\") " pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:44:44.734249 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:44.734107 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 18:44:44.734249 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:44.734127 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 18:44:44.734249 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:44.734141 2563 projected.go:196] Error preparing data for projected volume kube-api-access-jhskz for pod openshift-network-diagnostics/network-check-target-rxmht: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 18:44:44.734249 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:44.734203 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/254d8445-a0f4-4c11-bc96-2cb74c34ab13-kube-api-access-jhskz podName:254d8445-a0f4-4c11-bc96-2cb74c34ab13 nodeName:}" failed. No retries permitted until 2026-09-11 18:44:48.734185337 +0000 UTC m=+10.196791338 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jhskz" (UniqueName: "kubernetes.io/projected/254d8445-a0f4-4c11-bc96-2cb74c34ab13-kube-api-access-jhskz") pod "network-check-target-rxmht" (UID: "254d8445-a0f4-4c11-bc96-2cb74c34ab13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 18:44:45.054727 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:45.054692 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:44:45.054918 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:45.054818 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxqg6" podUID="b799f9cb-ce50-4731-9fbe-48f47807a053" Sep 11 18:44:45.055254 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:45.055231 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:44:45.055368 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:45.055349 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxmht" podUID="254d8445-a0f4-4c11-bc96-2cb74c34ab13" Sep 11 18:44:45.055436 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:45.055421 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:44:45.055531 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:45.055511 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lhj2z" podUID="b644d098-032f-42c9-9839-41ce9eb09844" Sep 11 18:44:47.056026 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:47.054837 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:44:47.056026 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:47.054979 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lhj2z" podUID="b644d098-032f-42c9-9839-41ce9eb09844" Sep 11 18:44:47.056026 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:47.055352 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:44:47.056026 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:47.055449 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxqg6" podUID="b799f9cb-ce50-4731-9fbe-48f47807a053" Sep 11 18:44:47.056026 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:47.055494 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:44:47.056026 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:47.055576 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxmht" podUID="254d8445-a0f4-4c11-bc96-2cb74c34ab13" Sep 11 18:44:48.565766 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:48.565729 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b799f9cb-ce50-4731-9fbe-48f47807a053-original-pull-secret\") pod \"global-pull-secret-syncer-sxqg6\" (UID: \"b799f9cb-ce50-4731-9fbe-48f47807a053\") " pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:44:48.566218 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:48.565871 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 18:44:48.566218 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:48.565920 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b799f9cb-ce50-4731-9fbe-48f47807a053-original-pull-secret podName:b799f9cb-ce50-4731-9fbe-48f47807a053 nodeName:}" failed. No retries permitted until 2026-09-11 18:44:56.565906176 +0000 UTC m=+18.028512173 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b799f9cb-ce50-4731-9fbe-48f47807a053-original-pull-secret") pod "global-pull-secret-syncer-sxqg6" (UID: "b799f9cb-ce50-4731-9fbe-48f47807a053") : object "kube-system"/"original-pull-secret" not registered Sep 11 18:44:48.666842 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:48.666806 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b644d098-032f-42c9-9839-41ce9eb09844-metrics-certs\") pod \"network-metrics-daemon-lhj2z\" (UID: \"b644d098-032f-42c9-9839-41ce9eb09844\") " pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:44:48.667036 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:48.666974 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 18:44:48.667101 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:48.667056 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b644d098-032f-42c9-9839-41ce9eb09844-metrics-certs podName:b644d098-032f-42c9-9839-41ce9eb09844 nodeName:}" failed. No retries permitted until 2026-09-11 18:44:56.667034702 +0000 UTC m=+18.129640702 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b644d098-032f-42c9-9839-41ce9eb09844-metrics-certs") pod "network-metrics-daemon-lhj2z" (UID: "b644d098-032f-42c9-9839-41ce9eb09844") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 18:44:48.768480 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:48.767894 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhskz\" (UniqueName: \"kubernetes.io/projected/254d8445-a0f4-4c11-bc96-2cb74c34ab13-kube-api-access-jhskz\") pod \"network-check-target-rxmht\" (UID: \"254d8445-a0f4-4c11-bc96-2cb74c34ab13\") " pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:44:48.768480 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:48.768055 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 18:44:48.768480 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:48.768073 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 18:44:48.768480 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:48.768085 2563 projected.go:196] Error preparing data for projected volume kube-api-access-jhskz for pod openshift-network-diagnostics/network-check-target-rxmht: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 18:44:48.768480 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:48.768143 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/254d8445-a0f4-4c11-bc96-2cb74c34ab13-kube-api-access-jhskz podName:254d8445-a0f4-4c11-bc96-2cb74c34ab13 nodeName:}" failed. No retries permitted until 2026-09-11 18:44:56.76812482 +0000 UTC m=+18.230730832 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jhskz" (UniqueName: "kubernetes.io/projected/254d8445-a0f4-4c11-bc96-2cb74c34ab13-kube-api-access-jhskz") pod "network-check-target-rxmht" (UID: "254d8445-a0f4-4c11-bc96-2cb74c34ab13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 18:44:49.054607 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:49.054494 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:44:49.054607 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:49.054514 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:44:49.054607 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:49.054586 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:44:49.055885 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:49.055468 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lhj2z" podUID="b644d098-032f-42c9-9839-41ce9eb09844" Sep 11 18:44:49.055885 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:49.055565 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxqg6" podUID="b799f9cb-ce50-4731-9fbe-48f47807a053" Sep 11 18:44:49.055885 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:49.055648 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxmht" podUID="254d8445-a0f4-4c11-bc96-2cb74c34ab13" Sep 11 18:44:51.059615 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:51.058814 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:44:51.059615 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:51.058950 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lhj2z" podUID="b644d098-032f-42c9-9839-41ce9eb09844" Sep 11 18:44:51.059615 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:51.059342 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:44:51.059615 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:51.059428 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxqg6" podUID="b799f9cb-ce50-4731-9fbe-48f47807a053" Sep 11 18:44:51.059615 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:51.059488 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:44:51.059615 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:51.059581 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxmht" podUID="254d8445-a0f4-4c11-bc96-2cb74c34ab13" Sep 11 18:44:53.054454 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:53.054426 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:44:53.054806 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:53.054430 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:44:53.054806 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:53.054548 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxqg6" podUID="b799f9cb-ce50-4731-9fbe-48f47807a053" Sep 11 18:44:53.054806 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:53.054436 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:44:53.054806 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:53.054606 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lhj2z" podUID="b644d098-032f-42c9-9839-41ce9eb09844" Sep 11 18:44:53.054806 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:53.054703 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxmht" podUID="254d8445-a0f4-4c11-bc96-2cb74c34ab13" Sep 11 18:44:55.058826 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:55.058616 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:44:55.059289 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:55.058674 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:44:55.059289 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:55.058930 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lhj2z" podUID="b644d098-032f-42c9-9839-41ce9eb09844" Sep 11 18:44:55.059289 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:55.058674 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:44:55.059289 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:55.058992 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxmht" podUID="254d8445-a0f4-4c11-bc96-2cb74c34ab13" Sep 11 18:44:55.059289 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:55.059074 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxqg6" podUID="b799f9cb-ce50-4731-9fbe-48f47807a053" Sep 11 18:44:56.626491 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:56.626449 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b799f9cb-ce50-4731-9fbe-48f47807a053-original-pull-secret\") pod \"global-pull-secret-syncer-sxqg6\" (UID: \"b799f9cb-ce50-4731-9fbe-48f47807a053\") " pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:44:56.626979 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:56.626617 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 18:44:56.626979 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:56.626693 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b799f9cb-ce50-4731-9fbe-48f47807a053-original-pull-secret podName:b799f9cb-ce50-4731-9fbe-48f47807a053 nodeName:}" failed. No retries permitted until 2026-09-11 18:45:12.626671008 +0000 UTC m=+34.089277004 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b799f9cb-ce50-4731-9fbe-48f47807a053-original-pull-secret") pod "global-pull-secret-syncer-sxqg6" (UID: "b799f9cb-ce50-4731-9fbe-48f47807a053") : object "kube-system"/"original-pull-secret" not registered Sep 11 18:44:56.727053 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:56.727018 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b644d098-032f-42c9-9839-41ce9eb09844-metrics-certs\") pod \"network-metrics-daemon-lhj2z\" (UID: \"b644d098-032f-42c9-9839-41ce9eb09844\") " pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:44:56.727233 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:56.727195 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 18:44:56.727286 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:56.727278 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b644d098-032f-42c9-9839-41ce9eb09844-metrics-certs podName:b644d098-032f-42c9-9839-41ce9eb09844 nodeName:}" failed. No retries permitted until 2026-09-11 18:45:12.727257578 +0000 UTC m=+34.189863590 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b644d098-032f-42c9-9839-41ce9eb09844-metrics-certs") pod "network-metrics-daemon-lhj2z" (UID: "b644d098-032f-42c9-9839-41ce9eb09844") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 18:44:56.828192 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:56.828156 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhskz\" (UniqueName: \"kubernetes.io/projected/254d8445-a0f4-4c11-bc96-2cb74c34ab13-kube-api-access-jhskz\") pod \"network-check-target-rxmht\" (UID: \"254d8445-a0f4-4c11-bc96-2cb74c34ab13\") " pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:44:56.828388 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:56.828331 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 18:44:56.828388 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:56.828354 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 18:44:56.828388 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:56.828364 2563 projected.go:196] Error preparing data for projected volume kube-api-access-jhskz for pod openshift-network-diagnostics/network-check-target-rxmht: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 18:44:56.828572 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:56.828417 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/254d8445-a0f4-4c11-bc96-2cb74c34ab13-kube-api-access-jhskz podName:254d8445-a0f4-4c11-bc96-2cb74c34ab13 nodeName:}" failed. No retries permitted until 2026-09-11 18:45:12.828401101 +0000 UTC m=+34.291007099 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jhskz" (UniqueName: "kubernetes.io/projected/254d8445-a0f4-4c11-bc96-2cb74c34ab13-kube-api-access-jhskz") pod "network-check-target-rxmht" (UID: "254d8445-a0f4-4c11-bc96-2cb74c34ab13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 18:44:57.054569 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:57.054537 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:44:57.054753 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:57.054582 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:44:57.054753 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:57.054666 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxqg6" podUID="b799f9cb-ce50-4731-9fbe-48f47807a053" Sep 11 18:44:57.054753 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:57.054539 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:44:57.054917 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:57.054770 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lhj2z" podUID="b644d098-032f-42c9-9839-41ce9eb09844" Sep 11 18:44:57.054917 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:57.054849 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxmht" podUID="254d8445-a0f4-4c11-bc96-2cb74c34ab13" Sep 11 18:44:59.055780 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:59.055741 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:44:59.056261 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:59.055859 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxqg6" podUID="b799f9cb-ce50-4731-9fbe-48f47807a053" Sep 11 18:44:59.056261 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:59.055888 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:44:59.056261 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:59.055991 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxmht" podUID="254d8445-a0f4-4c11-bc96-2cb74c34ab13" Sep 11 18:44:59.056261 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:44:59.056046 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:44:59.056261 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:44:59.056168 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lhj2z" podUID="b644d098-032f-42c9-9839-41ce9eb09844" Sep 11 18:45:01.054801 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.054642 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:45:01.055474 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.054637 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:45:01.055474 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.054682 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:45:01.055474 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:01.054871 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxmht" podUID="254d8445-a0f4-4c11-bc96-2cb74c34ab13" Sep 11 18:45:01.055474 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:01.055003 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxqg6" podUID="b799f9cb-ce50-4731-9fbe-48f47807a053" Sep 11 18:45:01.055474 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:01.055067 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lhj2z" podUID="b644d098-032f-42c9-9839-41ce9eb09844" Sep 11 18:45:01.308174 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.308092 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7f246" event={"ID":"60dd726a-4169-4a95-ade1-c067f1851aac","Type":"ContainerStarted","Data":"fda6ae240103ee72f22f5e2ac994a0bacd2eab0a3ccbed297671797203ed30b1"} Sep 11 18:45:01.309652 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.309628 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tnjsf" event={"ID":"254dd733-80bd-4f2a-9fbc-a57622608b32","Type":"ContainerStarted","Data":"71223ce0e240b017c5be4538d431b64ac13ccf0df88a5337325015bc4da319cd"} Sep 11 18:45:01.313886 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.313857 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fdn85" event={"ID":"274576da-90e4-47d9-b053-72498cd1ab3e","Type":"ContainerStarted","Data":"5cfa405b07e9b4ca59d15bd79c70142ba94eef172c060e53055489cd3118d6df"} Sep 11 18:45:01.315223 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.315198 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" event={"ID":"93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce","Type":"ContainerStarted","Data":"4f5317c61d57ccb1705efce7dcfa1640877fc467e26376dbbe30bf20beeee5bf"} Sep 11 18:45:01.316511 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.316470 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-z8mxj" event={"ID":"96d752b9-bf5e-4519-b2b8-6dbd832cab19","Type":"ContainerStarted","Data":"5da1cdcd7a403fe2c3f8a5a64fb5cb432c9aff04742b81b46c3ae5e82f20de2f"} Sep 11 18:45:01.317887 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.317865 2563 generic.go:356] "Generic (PLEG): container finished" podID="e9819d65-2122-497e-831c-917aa1fabd52" containerID="2a896ce6be264c67ecd0728531b9ae1832206119dd4f7fe8ab6bfae6e3d2f283" exitCode=0 Sep 11 18:45:01.317957 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.317927 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9xtl5" event={"ID":"e9819d65-2122-497e-831c-917aa1fabd52","Type":"ContainerDied","Data":"2a896ce6be264c67ecd0728531b9ae1832206119dd4f7fe8ab6bfae6e3d2f283"} Sep 11 18:45:01.322422 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.322395 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" event={"ID":"a688c734-cd8a-40eb-843f-d78e5ec411fd","Type":"ContainerStarted","Data":"bdd6075b22b18a238c9e483ef541a9683fd5a99a4a883224b655a69f4e2ae1bc"} Sep 11 18:45:01.322535 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.322481 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" event={"ID":"a688c734-cd8a-40eb-843f-d78e5ec411fd","Type":"ContainerStarted","Data":"38d859407b7647fb9c44e264883d3744ddbd26d176ba186ce23b86a451e33d5c"} Sep 11 18:45:01.322602 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.322543 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" event={"ID":"a688c734-cd8a-40eb-843f-d78e5ec411fd","Type":"ContainerStarted","Data":"bd9ce83be91e676ada15c733cf6bea2048cd222423f1b0c840a40c466e557f7a"} Sep 11 18:45:01.322602 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.322586 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" event={"ID":"a688c734-cd8a-40eb-843f-d78e5ec411fd","Type":"ContainerStarted","Data":"02a0e8f40acbede65648aa599bad13df938ef175bc612114f590eecdf28e9b94"} Sep 11 18:45:01.322710 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.322631 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" event={"ID":"a688c734-cd8a-40eb-843f-d78e5ec411fd","Type":"ContainerStarted","Data":"f3cee08c6b7864ac5aa866fc4ddf1827159c43493b4f1194dce26a6b9fb5e5d3"} Sep 11 18:45:01.322710 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.322660 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" event={"ID":"a688c734-cd8a-40eb-843f-d78e5ec411fd","Type":"ContainerStarted","Data":"dcfc92f1ef64cc0bfec1c102669215ea741b4429ccb7e9285570ea3a3f27b2e8"} Sep 11 18:45:01.324490 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.324425 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gv5z5" event={"ID":"22ef1e56-f6b2-4ebf-8be0-63669535237b","Type":"ContainerStarted","Data":"3f8d3b0a18c30f70113685ede8725a9785275f8a2a414844c6e7102513cb7032"} Sep 11 18:45:01.326852 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.326810 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7f246" podStartSLOduration=4.216798219 podStartE2EDuration="22.326796321s" totalImagesPullingTime="18.109998102s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:44:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:45:01.326262095 +0000 UTC m=+22.788868114" watchObservedRunningTime="2026-09-11 18:45:01.326796321 +0000 UTC m=+22.789402339" Sep 11 18:45:01.326958 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.326883 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-192.ec2.internal" podStartSLOduration=21.326877098 podStartE2EDuration="21.326877098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:44:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:44:44.285229406 +0000 UTC m=+5.747835430" watchObservedRunningTime="2026-09-11 18:45:01.326877098 +0000 UTC m=+22.789483147" Sep 11 18:45:01.345909 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.345862 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-z8mxj" podStartSLOduration=4.202555624 podStartE2EDuration="22.345849498s" totalImagesPullingTime="18.143293874s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:44:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:45:01.345536986 +0000 UTC m=+22.808143003" watchObservedRunningTime="2026-09-11 18:45:01.345849498 +0000 UTC m=+22.808455516" Sep 11 18:45:01.388554 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.388454 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fdn85" podStartSLOduration=4.297749567 podStartE2EDuration="22.38843902s" totalImagesPullingTime="18.090689453s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:44:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:45:01.370636356 +0000 UTC m=+22.833242371" watchObservedRunningTime="2026-09-11 18:45:01.38843902 +0000 UTC m=+22.851045040" Sep 11 18:45:01.388835 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.388808 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-tnjsf" podStartSLOduration=4.276107274 podStartE2EDuration="22.388796739s" totalImagesPullingTime="18.112689465s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:44:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:45:01.388566952 +0000 UTC m=+22.851172972" watchObservedRunningTime="2026-09-11 18:45:01.388796739 +0000 UTC m=+22.851402757" Sep 11 18:45:01.543856 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.543824 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:01.610928 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.610855 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:01.860826 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.860636 2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 11 18:45:01.944413 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.944304 2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T18:45:01.860819652Z","UUID":"b4b39c4e-8d12-4cb5-bbed-352a67b46c9f","Handler":null,"Name":"","Endpoint":""} Sep 11 18:45:01.946036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.946013 2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 11 18:45:01.946176 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:01.946046 2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 11 18:45:02.042667 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:02.042617 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-gv5z5" Sep 11 18:45:02.328865 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:02.328701 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nth2m" event={"ID":"876bf078-f416-471c-b683-74781309a51a","Type":"ContainerStarted","Data":"549080d5eaf86d8c65e6bce8993cf6ca273a66fd4a0aea7fa4e2b43f1e423dc4"} Sep 11 18:45:02.330761 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:02.330734 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" event={"ID":"93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce","Type":"ContainerStarted","Data":"6f201f756c199cebfd93a0b58a3c2ad48ef2f124b4ed85ea8793bee232186394"} Sep 11 18:45:02.345384 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:02.345336 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-gv5z5" podStartSLOduration=8.680872587 podStartE2EDuration="23.345315794s" totalImagesPullingTime="14.664443207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:44:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:45:01.47312385 +0000 UTC m=+22.935729868" watchObservedRunningTime="2026-09-11 18:45:02.345315794 +0000 UTC m=+23.807921815" Sep 11 18:45:02.345672 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:02.345641 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nth2m" podStartSLOduration=5.265126132 podStartE2EDuration="23.34563133s" totalImagesPullingTime="18.080505198s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:44:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:45:02.344929528 +0000 UTC m=+23.807535547" watchObservedRunningTime="2026-09-11 18:45:02.34563133 +0000 UTC m=+23.808237349" Sep 11 18:45:02.468204 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:02.468170 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:02.545380 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:02.545352 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:03.054465 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:03.054436 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:45:03.054465 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:03.054450 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:45:03.054785 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:03.054575 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxqg6" podUID="b799f9cb-ce50-4731-9fbe-48f47807a053" Sep 11 18:45:03.054785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:03.054631 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:45:03.054785 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:03.054751 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxmht" podUID="254d8445-a0f4-4c11-bc96-2cb74c34ab13" Sep 11 18:45:03.055254 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:03.054844 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lhj2z" podUID="b644d098-032f-42c9-9839-41ce9eb09844" Sep 11 18:45:03.336962 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:03.336878 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" event={"ID":"a688c734-cd8a-40eb-843f-d78e5ec411fd","Type":"ContainerStarted","Data":"104623c2d466dace25b040342c24a259a547cd0f1c3fe9411c673a30df6c84a1"} Sep 11 18:45:04.283520 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:04.283467 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-gv5z5" Sep 11 18:45:04.284211 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:04.284192 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-gv5z5" Sep 11 18:45:04.340266 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:04.340236 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-gv5z5" Sep 11 18:45:04.430748 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:04.430709 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:05.054849 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:05.054818 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:45:05.055015 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:05.054819 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:45:05.055015 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:05.054935 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxmht" podUID="254d8445-a0f4-4c11-bc96-2cb74c34ab13" Sep 11 18:45:05.055015 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:05.054823 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:45:05.055015 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:05.055007 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxqg6" podUID="b799f9cb-ce50-4731-9fbe-48f47807a053" Sep 11 18:45:05.055198 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:05.055087 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lhj2z" podUID="b644d098-032f-42c9-9839-41ce9eb09844" Sep 11 18:45:05.446724 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:05.446695 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:06.347038 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:06.346836 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" event={"ID":"93e0fb88-ab2b-4842-8c8a-fc4bd3e0e8ce","Type":"ContainerStarted","Data":"805c61b46ea0f9751d9064d6ec0116a72f121d624c97669e73f96dfb31cc1c42"} Sep 11 18:45:06.348675 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:06.348650 2563 generic.go:356] "Generic (PLEG): container finished" podID="e9819d65-2122-497e-831c-917aa1fabd52" containerID="2c72d3be1108d8aabd0f2033411271dc758c9c572dfe6a8f5989cb7c4d1707ca" exitCode=0 Sep 11 18:45:06.348785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:06.348691 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9xtl5" event={"ID":"e9819d65-2122-497e-831c-917aa1fabd52","Type":"ContainerDied","Data":"2c72d3be1108d8aabd0f2033411271dc758c9c572dfe6a8f5989cb7c4d1707ca"} Sep 11 18:45:06.352683 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:06.352660 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" event={"ID":"a688c734-cd8a-40eb-843f-d78e5ec411fd","Type":"ContainerStarted","Data":"2d4456816dbba36c5fd4e742cecaf9ceb4f9f2e392543d94f89b1a86dc6c5b64"} Sep 11 18:45:06.366229 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:06.366191 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4xd6" podStartSLOduration=4.597389954 podStartE2EDuration="27.366178906s" totalImagesPullingTime="22.768788952s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:44:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:45:06.365704037 +0000 UTC m=+27.828310054" watchObservedRunningTime="2026-09-11 18:45:06.366178906 +0000 UTC m=+27.828784956" Sep 11 18:45:06.414766 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:06.414714 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" podStartSLOduration=9.260778431 podStartE2EDuration="27.414699058s" totalImagesPullingTime="18.153920627s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:44:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:45:06.413047773 +0000 UTC m=+27.875653791" watchObservedRunningTime="2026-09-11 18:45:06.414699058 +0000 UTC m=+27.877305075" Sep 11 18:45:07.055197 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:07.055158 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:45:07.056110 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:07.055279 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxqg6" podUID="b799f9cb-ce50-4731-9fbe-48f47807a053" Sep 11 18:45:07.056110 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:07.055173 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:45:07.056110 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:07.055355 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lhj2z" podUID="b644d098-032f-42c9-9839-41ce9eb09844" Sep 11 18:45:07.056110 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:07.055158 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:45:07.056110 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:07.055410 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxmht" podUID="254d8445-a0f4-4c11-bc96-2cb74c34ab13" Sep 11 18:45:07.356299 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:07.355961 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:45:07.356299 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:07.356201 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:45:07.356299 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:07.356248 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:45:07.370491 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:07.370459 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sxqg6"] Sep 11 18:45:07.370639 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:07.370604 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:45:07.370775 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:07.370750 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxqg6" podUID="b799f9cb-ce50-4731-9fbe-48f47807a053" Sep 11 18:45:07.371361 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:07.371336 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rxmht"] Sep 11 18:45:07.371461 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:07.371432 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:45:07.371554 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:07.371537 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxmht" podUID="254d8445-a0f4-4c11-bc96-2cb74c34ab13" Sep 11 18:45:07.372921 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:07.372846 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lhj2z"] Sep 11 18:45:07.373040 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:07.372944 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:45:07.373095 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:07.373039 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lhj2z" podUID="b644d098-032f-42c9-9839-41ce9eb09844" Sep 11 18:45:07.374209 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:07.374186 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:45:07.374372 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:07.374352 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:45:07.701993 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:07.701861 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:07.779161 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:07.779127 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:08.359324 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:08.359295 2563 generic.go:356] "Generic (PLEG): container finished" podID="e9819d65-2122-497e-831c-917aa1fabd52" containerID="bf4950ff1817c7357c63d2e6aa999ae400efa0e5ea1bb21e4dbb1ae22a5a15bf" exitCode=0 Sep 11 18:45:08.359876 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:08.359384 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9xtl5" event={"ID":"e9819d65-2122-497e-831c-917aa1fabd52","Type":"ContainerDied","Data":"bf4950ff1817c7357c63d2e6aa999ae400efa0e5ea1bb21e4dbb1ae22a5a15bf"} Sep 11 18:45:09.055440 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:09.055399 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:45:09.055652 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:09.055482 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxmht" podUID="254d8445-a0f4-4c11-bc96-2cb74c34ab13" Sep 11 18:45:09.055652 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:09.055572 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:45:09.055779 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:09.055663 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lhj2z" podUID="b644d098-032f-42c9-9839-41ce9eb09844" Sep 11 18:45:09.055779 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:09.055706 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:45:09.055779 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:09.055758 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxqg6" podUID="b799f9cb-ce50-4731-9fbe-48f47807a053" Sep 11 18:45:09.477661 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:09.477586 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:09.576953 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:09.576926 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:09.926367 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:09.926286 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:10.027166 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:10.027135 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:10.366919 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:10.366832 2563 generic.go:356] "Generic (PLEG): container finished" podID="e9819d65-2122-497e-831c-917aa1fabd52" containerID="a46313fcc557ce1aab8ec4bb47823813bfa083264c3e737330fbac3523a392ba" exitCode=0 Sep 11 18:45:10.366919 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:10.366879 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9xtl5" event={"ID":"e9819d65-2122-497e-831c-917aa1fabd52","Type":"ContainerDied","Data":"a46313fcc557ce1aab8ec4bb47823813bfa083264c3e737330fbac3523a392ba"} Sep 11 18:45:10.395943 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:10.395911 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:10.452786 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:10.452752 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:11.055283 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:11.055191 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:45:11.055283 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:11.055235 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:45:11.055827 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:11.055327 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxmht" podUID="254d8445-a0f4-4c11-bc96-2cb74c34ab13" Sep 11 18:45:11.055827 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:11.055776 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxqg6" podUID="b799f9cb-ce50-4731-9fbe-48f47807a053" Sep 11 18:45:11.055941 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:11.055848 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:45:11.055990 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:11.055958 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lhj2z" podUID="b644d098-032f-42c9-9839-41ce9eb09844" Sep 11 18:45:12.646057 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:12.646023 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b799f9cb-ce50-4731-9fbe-48f47807a053-original-pull-secret\") pod \"global-pull-secret-syncer-sxqg6\" (UID: \"b799f9cb-ce50-4731-9fbe-48f47807a053\") " pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:45:12.646486 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:12.646157 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 18:45:12.646486 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:12.646227 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b799f9cb-ce50-4731-9fbe-48f47807a053-original-pull-secret podName:b799f9cb-ce50-4731-9fbe-48f47807a053 nodeName:}" failed. No retries permitted until 2026-09-11 18:45:44.6462074 +0000 UTC m=+66.108813402 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b799f9cb-ce50-4731-9fbe-48f47807a053-original-pull-secret") pod "global-pull-secret-syncer-sxqg6" (UID: "b799f9cb-ce50-4731-9fbe-48f47807a053") : object "kube-system"/"original-pull-secret" not registered Sep 11 18:45:12.746719 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:12.746679 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b644d098-032f-42c9-9839-41ce9eb09844-metrics-certs\") pod \"network-metrics-daemon-lhj2z\" (UID: \"b644d098-032f-42c9-9839-41ce9eb09844\") " pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:45:12.746891 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:12.746831 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 18:45:12.746936 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:12.746898 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b644d098-032f-42c9-9839-41ce9eb09844-metrics-certs podName:b644d098-032f-42c9-9839-41ce9eb09844 nodeName:}" failed. No retries permitted until 2026-09-11 18:45:44.746882879 +0000 UTC m=+66.209488876 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b644d098-032f-42c9-9839-41ce9eb09844-metrics-certs") pod "network-metrics-daemon-lhj2z" (UID: "b644d098-032f-42c9-9839-41ce9eb09844") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 18:45:12.847143 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:12.847104 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhskz\" (UniqueName: \"kubernetes.io/projected/254d8445-a0f4-4c11-bc96-2cb74c34ab13-kube-api-access-jhskz\") pod \"network-check-target-rxmht\" (UID: \"254d8445-a0f4-4c11-bc96-2cb74c34ab13\") " pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:45:12.847313 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:12.847259 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 18:45:12.847313 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:12.847273 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 18:45:12.847313 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:12.847282 2563 projected.go:196] Error preparing data for projected volume kube-api-access-jhskz for pod openshift-network-diagnostics/network-check-target-rxmht: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 18:45:12.847419 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:12.847328 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/254d8445-a0f4-4c11-bc96-2cb74c34ab13-kube-api-access-jhskz podName:254d8445-a0f4-4c11-bc96-2cb74c34ab13 nodeName:}" failed. No retries permitted until 2026-09-11 18:45:44.847315216 +0000 UTC m=+66.309921212 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jhskz" (UniqueName: "kubernetes.io/projected/254d8445-a0f4-4c11-bc96-2cb74c34ab13-kube-api-access-jhskz") pod "network-check-target-rxmht" (UID: "254d8445-a0f4-4c11-bc96-2cb74c34ab13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 18:45:13.055675 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.055395 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:45:13.055845 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.055395 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:45:13.055845 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:13.055727 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxqg6" podUID="b799f9cb-ce50-4731-9fbe-48f47807a053" Sep 11 18:45:13.055845 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:13.055783 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxmht" podUID="254d8445-a0f4-4c11-bc96-2cb74c34ab13" Sep 11 18:45:13.055845 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.055479 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:45:13.056054 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:13.055872 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lhj2z" podUID="b644d098-032f-42c9-9839-41ce9eb09844" Sep 11 18:45:13.264004 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.263973 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-192.ec2.internal" event="NodeReady" Sep 11 18:45:13.264147 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.264120 2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 11 18:45:13.314837 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.314753 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dzb2p"] Sep 11 18:45:13.342492 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.342456 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7h9dc"] Sep 11 18:45:13.342673 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.342562 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dzb2p" Sep 11 18:45:13.345420 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.345391 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 11 18:45:13.345953 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.345933 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 11 18:45:13.346088 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.345995 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bsbkc\"" Sep 11 18:45:13.356154 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.356132 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7h9dc"] Sep 11 18:45:13.356262 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.356167 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dzb2p"] Sep 11 18:45:13.356262 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.356239 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7h9dc" Sep 11 18:45:13.359471 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.359431 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 11 18:45:13.359593 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.359534 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 11 18:45:13.360224 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.360074 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-czgt4\"" Sep 11 18:45:13.360224 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.360092 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 11 18:45:13.443142 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.443106 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:13.451063 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.451035 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smtwp\" (UniqueName: \"kubernetes.io/projected/7a322a44-c645-482e-811b-cae2ea119a2f-kube-api-access-smtwp\") pod \"ingress-canary-7h9dc\" (UID: \"7a322a44-c645-482e-811b-cae2ea119a2f\") " pod="openshift-ingress-canary/ingress-canary-7h9dc" Sep 11 18:45:13.451232 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.451075 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0a1958ce-2c55-4c46-a2af-6b48ab70f9df-config-volume\") pod \"dns-default-dzb2p\" (UID: \"0a1958ce-2c55-4c46-a2af-6b48ab70f9df\") " pod="openshift-dns/dns-default-dzb2p" Sep 11 18:45:13.451232 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.451176 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a322a44-c645-482e-811b-cae2ea119a2f-cert\") pod \"ingress-canary-7h9dc\" (UID: \"7a322a44-c645-482e-811b-cae2ea119a2f\") " pod="openshift-ingress-canary/ingress-canary-7h9dc" Sep 11 18:45:13.451352 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.451263 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a1958ce-2c55-4c46-a2af-6b48ab70f9df-metrics-tls\") pod \"dns-default-dzb2p\" (UID: \"0a1958ce-2c55-4c46-a2af-6b48ab70f9df\") " pod="openshift-dns/dns-default-dzb2p" Sep 11 18:45:13.451352 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.451299 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a1958ce-2c55-4c46-a2af-6b48ab70f9df-tmp-dir\") pod \"dns-default-dzb2p\" (UID: \"0a1958ce-2c55-4c46-a2af-6b48ab70f9df\") " pod="openshift-dns/dns-default-dzb2p" Sep 11 18:45:13.451352 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.451328 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7778q\" (UniqueName: \"kubernetes.io/projected/0a1958ce-2c55-4c46-a2af-6b48ab70f9df-kube-api-access-7778q\") pod \"dns-default-dzb2p\" (UID: \"0a1958ce-2c55-4c46-a2af-6b48ab70f9df\") " pod="openshift-dns/dns-default-dzb2p" Sep 11 18:45:13.499420 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.499388 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:13.551918 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.551878 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smtwp\" (UniqueName: \"kubernetes.io/projected/7a322a44-c645-482e-811b-cae2ea119a2f-kube-api-access-smtwp\") pod \"ingress-canary-7h9dc\" (UID: \"7a322a44-c645-482e-811b-cae2ea119a2f\") " pod="openshift-ingress-canary/ingress-canary-7h9dc" Sep 11 18:45:13.551918 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.551927 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0a1958ce-2c55-4c46-a2af-6b48ab70f9df-config-volume\") pod \"dns-default-dzb2p\" (UID: \"0a1958ce-2c55-4c46-a2af-6b48ab70f9df\") " pod="openshift-dns/dns-default-dzb2p" Sep 11 18:45:13.552161 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.551955 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a322a44-c645-482e-811b-cae2ea119a2f-cert\") pod \"ingress-canary-7h9dc\" (UID: \"7a322a44-c645-482e-811b-cae2ea119a2f\") " pod="openshift-ingress-canary/ingress-canary-7h9dc" Sep 11 18:45:13.552161 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.551994 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a1958ce-2c55-4c46-a2af-6b48ab70f9df-metrics-tls\") pod \"dns-default-dzb2p\" (UID: \"0a1958ce-2c55-4c46-a2af-6b48ab70f9df\") " pod="openshift-dns/dns-default-dzb2p" Sep 11 18:45:13.552161 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:13.552082 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 18:45:13.552161 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:13.552136 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a1958ce-2c55-4c46-a2af-6b48ab70f9df-metrics-tls podName:0a1958ce-2c55-4c46-a2af-6b48ab70f9df nodeName:}" failed. No retries permitted until 2026-09-11 18:45:14.052121713 +0000 UTC m=+35.514727732 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a1958ce-2c55-4c46-a2af-6b48ab70f9df-metrics-tls") pod "dns-default-dzb2p" (UID: "0a1958ce-2c55-4c46-a2af-6b48ab70f9df") : secret "dns-default-metrics-tls" not found Sep 11 18:45:13.552161 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.552149 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a1958ce-2c55-4c46-a2af-6b48ab70f9df-tmp-dir\") pod \"dns-default-dzb2p\" (UID: \"0a1958ce-2c55-4c46-a2af-6b48ab70f9df\") " pod="openshift-dns/dns-default-dzb2p" Sep 11 18:45:13.552388 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.552168 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7778q\" (UniqueName: \"kubernetes.io/projected/0a1958ce-2c55-4c46-a2af-6b48ab70f9df-kube-api-access-7778q\") pod \"dns-default-dzb2p\" (UID: \"0a1958ce-2c55-4c46-a2af-6b48ab70f9df\") " pod="openshift-dns/dns-default-dzb2p" Sep 11 18:45:13.552761 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.552735 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a1958ce-2c55-4c46-a2af-6b48ab70f9df-tmp-dir\") pod \"dns-default-dzb2p\" (UID: \"0a1958ce-2c55-4c46-a2af-6b48ab70f9df\") " pod="openshift-dns/dns-default-dzb2p" Sep 11 18:45:13.552945 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.552917 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0a1958ce-2c55-4c46-a2af-6b48ab70f9df-config-volume\") pod \"dns-default-dzb2p\" (UID: \"0a1958ce-2c55-4c46-a2af-6b48ab70f9df\") " pod="openshift-dns/dns-default-dzb2p" Sep 11 18:45:13.557016 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.556995 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a322a44-c645-482e-811b-cae2ea119a2f-cert\") pod \"ingress-canary-7h9dc\" (UID: \"7a322a44-c645-482e-811b-cae2ea119a2f\") " pod="openshift-ingress-canary/ingress-canary-7h9dc" Sep 11 18:45:13.563258 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.563209 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7778q\" (UniqueName: \"kubernetes.io/projected/0a1958ce-2c55-4c46-a2af-6b48ab70f9df-kube-api-access-7778q\") pod \"dns-default-dzb2p\" (UID: \"0a1958ce-2c55-4c46-a2af-6b48ab70f9df\") " pod="openshift-dns/dns-default-dzb2p" Sep 11 18:45:13.563400 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.563339 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smtwp\" (UniqueName: \"kubernetes.io/projected/7a322a44-c645-482e-811b-cae2ea119a2f-kube-api-access-smtwp\") pod \"ingress-canary-7h9dc\" (UID: \"7a322a44-c645-482e-811b-cae2ea119a2f\") " pod="openshift-ingress-canary/ingress-canary-7h9dc" Sep 11 18:45:13.665584 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.665483 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7h9dc" Sep 11 18:45:13.839523 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.839480 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-65929"] Sep 11 18:45:13.840047 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.839985 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:13.859775 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.859747 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-65929"] Sep 11 18:45:13.859944 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.859885 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:13.862701 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.862677 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 11 18:45:13.862701 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.862693 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 11 18:45:13.862881 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.862817 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-c5jnz\"" Sep 11 18:45:13.862944 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.862917 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 11 18:45:13.863431 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.863411 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 11 18:45:13.918734 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.918659 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:13.955601 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.955563 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-data-volume\") pod \"insights-runtime-extractor-65929\" (UID: \"1c275a87-26b0-4fd7-a4fc-8f12b496a04e\") " pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:13.955601 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.955600 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-65929\" (UID: \"1c275a87-26b0-4fd7-a4fc-8f12b496a04e\") " pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:13.955846 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.955621 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krnbl\" (UniqueName: \"kubernetes.io/projected/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-kube-api-access-krnbl\") pod \"insights-runtime-extractor-65929\" (UID: \"1c275a87-26b0-4fd7-a4fc-8f12b496a04e\") " pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:13.955846 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.955736 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-crio-socket\") pod \"insights-runtime-extractor-65929\" (UID: \"1c275a87-26b0-4fd7-a4fc-8f12b496a04e\") " pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:13.955846 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:13.955771 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-65929\" (UID: \"1c275a87-26b0-4fd7-a4fc-8f12b496a04e\") " pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:14.056244 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:14.056207 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a1958ce-2c55-4c46-a2af-6b48ab70f9df-metrics-tls\") pod \"dns-default-dzb2p\" (UID: \"0a1958ce-2c55-4c46-a2af-6b48ab70f9df\") " pod="openshift-dns/dns-default-dzb2p" Sep 11 18:45:14.056421 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:14.056248 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-crio-socket\") pod \"insights-runtime-extractor-65929\" (UID: \"1c275a87-26b0-4fd7-a4fc-8f12b496a04e\") " pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:14.056421 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:14.056281 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-65929\" (UID: \"1c275a87-26b0-4fd7-a4fc-8f12b496a04e\") " pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:14.056421 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:14.056374 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 18:45:14.056602 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:14.056422 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-data-volume\") pod \"insights-runtime-extractor-65929\" (UID: \"1c275a87-26b0-4fd7-a4fc-8f12b496a04e\") " pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:14.056602 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:14.056436 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-insights-runtime-extractor-tls podName:1c275a87-26b0-4fd7-a4fc-8f12b496a04e nodeName:}" failed. No retries permitted until 2026-09-11 18:45:14.556417812 +0000 UTC m=+36.019023830 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-65929" (UID: "1c275a87-26b0-4fd7-a4fc-8f12b496a04e") : secret "insights-runtime-extractor-tls" not found Sep 11 18:45:14.056602 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:14.056460 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-65929\" (UID: \"1c275a87-26b0-4fd7-a4fc-8f12b496a04e\") " pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:14.056602 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:14.056486 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krnbl\" (UniqueName: \"kubernetes.io/projected/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-kube-api-access-krnbl\") pod \"insights-runtime-extractor-65929\" (UID: \"1c275a87-26b0-4fd7-a4fc-8f12b496a04e\") " pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:14.056602 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:14.056538 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-crio-socket\") pod \"insights-runtime-extractor-65929\" (UID: \"1c275a87-26b0-4fd7-a4fc-8f12b496a04e\") " pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:14.056602 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:14.056374 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 18:45:14.056878 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:14.056626 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 11 18:45:14.056878 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:14.056642 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a1958ce-2c55-4c46-a2af-6b48ab70f9df-metrics-tls podName:0a1958ce-2c55-4c46-a2af-6b48ab70f9df nodeName:}" failed. No retries permitted until 2026-09-11 18:45:15.056624061 +0000 UTC m=+36.519230064 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a1958ce-2c55-4c46-a2af-6b48ab70f9df-metrics-tls") pod "dns-default-dzb2p" (UID: "0a1958ce-2c55-4c46-a2af-6b48ab70f9df") : secret "dns-default-metrics-tls" not found Sep 11 18:45:14.056878 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:14.056674 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-kube-rbac-proxy-cm podName:1c275a87-26b0-4fd7-a4fc-8f12b496a04e nodeName:}" failed. No retries permitted until 2026-09-11 18:45:14.556661162 +0000 UTC m=+36.019267158 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-65929" (UID: "1c275a87-26b0-4fd7-a4fc-8f12b496a04e") : configmap "kube-rbac-proxy" not found Sep 11 18:45:14.056878 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:14.056819 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-data-volume\") pod \"insights-runtime-extractor-65929\" (UID: \"1c275a87-26b0-4fd7-a4fc-8f12b496a04e\") " pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:14.065592 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:14.065563 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krnbl\" (UniqueName: \"kubernetes.io/projected/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-kube-api-access-krnbl\") pod \"insights-runtime-extractor-65929\" (UID: \"1c275a87-26b0-4fd7-a4fc-8f12b496a04e\") " pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:14.560497 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:14.560463 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-65929\" (UID: \"1c275a87-26b0-4fd7-a4fc-8f12b496a04e\") " pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:14.560670 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:14.560553 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-65929\" (UID: \"1c275a87-26b0-4fd7-a4fc-8f12b496a04e\") " pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:14.560670 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:14.560607 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 11 18:45:14.560670 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:14.560655 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 18:45:14.560802 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:14.560687 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-kube-rbac-proxy-cm podName:1c275a87-26b0-4fd7-a4fc-8f12b496a04e nodeName:}" failed. No retries permitted until 2026-09-11 18:45:15.560666336 +0000 UTC m=+37.023272354 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-65929" (UID: "1c275a87-26b0-4fd7-a4fc-8f12b496a04e") : configmap "kube-rbac-proxy" not found Sep 11 18:45:14.560802 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:14.560706 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-insights-runtime-extractor-tls podName:1c275a87-26b0-4fd7-a4fc-8f12b496a04e nodeName:}" failed. No retries permitted until 2026-09-11 18:45:15.560696749 +0000 UTC m=+37.023302748 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-65929" (UID: "1c275a87-26b0-4fd7-a4fc-8f12b496a04e") : secret "insights-runtime-extractor-tls" not found Sep 11 18:45:14.658200 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:14.658120 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:14.683787 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:14.683758 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7h9dc"] Sep 11 18:45:14.687172 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:45:14.687144 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a322a44_c645_482e_811b_cae2ea119a2f.slice/crio-31b86026a3de1729bc767c7da932bb60ce6556778ea248925b7b352321266e99 WatchSource:0}: Error finding container 31b86026a3de1729bc767c7da932bb60ce6556778ea248925b7b352321266e99: Status 404 returned error can't find the container with id 31b86026a3de1729bc767c7da932bb60ce6556778ea248925b7b352321266e99 Sep 11 18:45:14.687555 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:14.687534 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-p5gd9"] Sep 11 18:45:14.707795 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:14.707764 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-p5gd9"] Sep 11 18:45:14.707901 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:14.707885 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-p5gd9" Sep 11 18:45:14.710467 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:14.710441 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Sep 11 18:45:14.710597 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:14.710529 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Sep 11 18:45:14.710779 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:14.710765 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-8bxzs\"" Sep 11 18:45:14.737178 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:14.737130 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:14.761801 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:14.761771 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8tkt\" (UniqueName: \"kubernetes.io/projected/c39ed765-6eaa-420f-81e3-f47d5e972ff9-kube-api-access-x8tkt\") pod \"migrator-8b4d697c5-p5gd9\" (UID: \"c39ed765-6eaa-420f-81e3-f47d5e972ff9\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-p5gd9" Sep 11 18:45:14.816766 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:14.816737 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:14.862377 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:14.862342 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8tkt\" (UniqueName: \"kubernetes.io/projected/c39ed765-6eaa-420f-81e3-f47d5e972ff9-kube-api-access-x8tkt\") pod \"migrator-8b4d697c5-p5gd9\" (UID: \"c39ed765-6eaa-420f-81e3-f47d5e972ff9\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-p5gd9" Sep 11 18:45:14.873764 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:14.873741 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8tkt\" (UniqueName: \"kubernetes.io/projected/c39ed765-6eaa-420f-81e3-f47d5e972ff9-kube-api-access-x8tkt\") pod \"migrator-8b4d697c5-p5gd9\" (UID: \"c39ed765-6eaa-420f-81e3-f47d5e972ff9\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-p5gd9" Sep 11 18:45:14.965141 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:14.965115 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:15.017546 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:15.017517 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-p5gd9" Sep 11 18:45:15.022054 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:15.022025 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:15.054968 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:15.054936 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:45:15.055105 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:15.054982 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:45:15.055105 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:15.054937 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:45:15.058345 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:15.057939 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 11 18:45:15.058345 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:15.058078 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 11 18:45:15.058345 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:15.058185 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 11 18:45:15.058345 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:15.058218 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-whrnn\"" Sep 11 18:45:15.058345 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:15.058277 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-p4phg\"" Sep 11 18:45:15.058673 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:15.058411 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 11 18:45:15.063702 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:15.063678 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a1958ce-2c55-4c46-a2af-6b48ab70f9df-metrics-tls\") pod \"dns-default-dzb2p\" (UID: \"0a1958ce-2c55-4c46-a2af-6b48ab70f9df\") " pod="openshift-dns/dns-default-dzb2p" Sep 11 18:45:15.063837 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:15.063823 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 18:45:15.063905 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:15.063891 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a1958ce-2c55-4c46-a2af-6b48ab70f9df-metrics-tls podName:0a1958ce-2c55-4c46-a2af-6b48ab70f9df nodeName:}" failed. No retries permitted until 2026-09-11 18:45:17.063870389 +0000 UTC m=+38.526476386 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a1958ce-2c55-4c46-a2af-6b48ab70f9df-metrics-tls") pod "dns-default-dzb2p" (UID: "0a1958ce-2c55-4c46-a2af-6b48ab70f9df") : secret "dns-default-metrics-tls" not found Sep 11 18:45:15.298774 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:15.298665 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-p5gd9"] Sep 11 18:45:15.303086 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:45:15.303057 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc39ed765_6eaa_420f_81e3_f47d5e972ff9.slice/crio-7222bb547a971009ef78bc01e6dbcd6fc66cf9d5676c7ecbb75efbaa34d3356c WatchSource:0}: Error finding container 7222bb547a971009ef78bc01e6dbcd6fc66cf9d5676c7ecbb75efbaa34d3356c: Status 404 returned error can't find the container with id 7222bb547a971009ef78bc01e6dbcd6fc66cf9d5676c7ecbb75efbaa34d3356c Sep 11 18:45:15.379270 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:15.379235 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-p5gd9" event={"ID":"c39ed765-6eaa-420f-81e3-f47d5e972ff9","Type":"ContainerStarted","Data":"7222bb547a971009ef78bc01e6dbcd6fc66cf9d5676c7ecbb75efbaa34d3356c"} Sep 11 18:45:15.380454 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:15.380423 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7h9dc" event={"ID":"7a322a44-c645-482e-811b-cae2ea119a2f","Type":"ContainerStarted","Data":"31b86026a3de1729bc767c7da932bb60ce6556778ea248925b7b352321266e99"} Sep 11 18:45:15.382946 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:15.382917 2563 generic.go:356] "Generic (PLEG): container finished" podID="e9819d65-2122-497e-831c-917aa1fabd52" containerID="5048fcd164d9af15557f31ff3cc46f07744210d6704872abee8a39a4737c8376" exitCode=0 Sep 11 18:45:15.383043 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:15.382966 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9xtl5" event={"ID":"e9819d65-2122-497e-831c-917aa1fabd52","Type":"ContainerDied","Data":"5048fcd164d9af15557f31ff3cc46f07744210d6704872abee8a39a4737c8376"} Sep 11 18:45:15.567451 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:15.567406 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-65929\" (UID: \"1c275a87-26b0-4fd7-a4fc-8f12b496a04e\") " pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:15.567599 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:15.567496 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-65929\" (UID: \"1c275a87-26b0-4fd7-a4fc-8f12b496a04e\") " pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:15.567599 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:15.567579 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 18:45:15.567734 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:15.567581 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 11 18:45:15.567734 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:15.567648 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-insights-runtime-extractor-tls podName:1c275a87-26b0-4fd7-a4fc-8f12b496a04e nodeName:}" failed. No retries permitted until 2026-09-11 18:45:17.567627321 +0000 UTC m=+39.030233323 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-65929" (UID: "1c275a87-26b0-4fd7-a4fc-8f12b496a04e") : secret "insights-runtime-extractor-tls" not found Sep 11 18:45:15.567734 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:15.567714 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-kube-rbac-proxy-cm podName:1c275a87-26b0-4fd7-a4fc-8f12b496a04e nodeName:}" failed. No retries permitted until 2026-09-11 18:45:17.567695742 +0000 UTC m=+39.030301741 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-65929" (UID: "1c275a87-26b0-4fd7-a4fc-8f12b496a04e") : configmap "kube-rbac-proxy" not found Sep 11 18:45:15.572682 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:15.572650 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:15.586862 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:15.586843 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7f246_60dd726a-4169-4a95-ade1-c067f1851aac/dns-node-resolver/0.log" Sep 11 18:45:15.674610 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:15.674581 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:16.008562 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:16.008475 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:16.108858 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:16.108815 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:16.286607 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:16.286476 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:16.383674 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:16.383644 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:16.390700 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:16.390658 2563 generic.go:356] "Generic (PLEG): container finished" podID="e9819d65-2122-497e-831c-917aa1fabd52" containerID="9bf000bba747eb07347a36b8c0ed815c93029482778d8f8b6b6d7e224d67f01c" exitCode=0 Sep 11 18:45:16.390872 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:16.390713 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9xtl5" event={"ID":"e9819d65-2122-497e-831c-917aa1fabd52","Type":"ContainerDied","Data":"9bf000bba747eb07347a36b8c0ed815c93029482778d8f8b6b6d7e224d67f01c"} Sep 11 18:45:16.787298 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:16.787268 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tnjsf_254dd733-80bd-4f2a-9fbc-a57622608b32/node-ca/0.log" Sep 11 18:45:17.081813 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:17.081778 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a1958ce-2c55-4c46-a2af-6b48ab70f9df-metrics-tls\") pod \"dns-default-dzb2p\" (UID: \"0a1958ce-2c55-4c46-a2af-6b48ab70f9df\") " pod="openshift-dns/dns-default-dzb2p" Sep 11 18:45:17.082247 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:17.081967 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 18:45:17.082247 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:17.082054 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a1958ce-2c55-4c46-a2af-6b48ab70f9df-metrics-tls podName:0a1958ce-2c55-4c46-a2af-6b48ab70f9df nodeName:}" failed. No retries permitted until 2026-09-11 18:45:21.082033332 +0000 UTC m=+42.544639333 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a1958ce-2c55-4c46-a2af-6b48ab70f9df-metrics-tls") pod "dns-default-dzb2p" (UID: "0a1958ce-2c55-4c46-a2af-6b48ab70f9df") : secret "dns-default-metrics-tls" not found Sep 11 18:45:17.585364 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:17.585284 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-65929\" (UID: \"1c275a87-26b0-4fd7-a4fc-8f12b496a04e\") " pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:17.585538 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:17.585362 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-65929\" (UID: \"1c275a87-26b0-4fd7-a4fc-8f12b496a04e\") " pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:17.585538 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:17.585412 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 18:45:17.585538 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:17.585456 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 11 18:45:17.585538 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:17.585480 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-insights-runtime-extractor-tls podName:1c275a87-26b0-4fd7-a4fc-8f12b496a04e nodeName:}" failed. No retries permitted until 2026-09-11 18:45:21.585465291 +0000 UTC m=+43.048071287 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-65929" (UID: "1c275a87-26b0-4fd7-a4fc-8f12b496a04e") : secret "insights-runtime-extractor-tls" not found Sep 11 18:45:17.585538 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:17.585522 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-kube-rbac-proxy-cm podName:1c275a87-26b0-4fd7-a4fc-8f12b496a04e nodeName:}" failed. No retries permitted until 2026-09-11 18:45:21.585489243 +0000 UTC m=+43.048095242 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-65929" (UID: "1c275a87-26b0-4fd7-a4fc-8f12b496a04e") : configmap "kube-rbac-proxy" not found Sep 11 18:45:17.895187 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:17.895026 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-vjp6p"] Sep 11 18:45:17.909777 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:17.909753 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-vjp6p"] Sep 11 18:45:17.909884 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:17.909845 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-vjp6p" Sep 11 18:45:17.912947 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:17.912922 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-466th\"" Sep 11 18:45:17.914138 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:17.914109 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 11 18:45:17.914138 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:17.914129 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 11 18:45:17.914305 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:17.914182 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 11 18:45:17.914305 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:17.914225 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 11 18:45:17.987954 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:17.987911 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ddbe2737-ad5d-4fd6-a3b5-7215b5bb5c57-signing-key\") pod \"service-ca-568869fdc5-vjp6p\" (UID: \"ddbe2737-ad5d-4fd6-a3b5-7215b5bb5c57\") " pod="openshift-service-ca/service-ca-568869fdc5-vjp6p" Sep 11 18:45:17.988120 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:17.988070 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ddbe2737-ad5d-4fd6-a3b5-7215b5bb5c57-signing-cabundle\") pod \"service-ca-568869fdc5-vjp6p\" (UID: \"ddbe2737-ad5d-4fd6-a3b5-7215b5bb5c57\") " pod="openshift-service-ca/service-ca-568869fdc5-vjp6p" Sep 11 18:45:17.988120 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:17.988110 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g66pj\" (UniqueName: \"kubernetes.io/projected/ddbe2737-ad5d-4fd6-a3b5-7215b5bb5c57-kube-api-access-g66pj\") pod \"service-ca-568869fdc5-vjp6p\" (UID: \"ddbe2737-ad5d-4fd6-a3b5-7215b5bb5c57\") " pod="openshift-service-ca/service-ca-568869fdc5-vjp6p" Sep 11 18:45:18.003478 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:18.003446 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:18.088605 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:18.088572 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ddbe2737-ad5d-4fd6-a3b5-7215b5bb5c57-signing-cabundle\") pod \"service-ca-568869fdc5-vjp6p\" (UID: \"ddbe2737-ad5d-4fd6-a3b5-7215b5bb5c57\") " pod="openshift-service-ca/service-ca-568869fdc5-vjp6p" Sep 11 18:45:18.088965 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:18.088688 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g66pj\" (UniqueName: \"kubernetes.io/projected/ddbe2737-ad5d-4fd6-a3b5-7215b5bb5c57-kube-api-access-g66pj\") pod \"service-ca-568869fdc5-vjp6p\" (UID: \"ddbe2737-ad5d-4fd6-a3b5-7215b5bb5c57\") " pod="openshift-service-ca/service-ca-568869fdc5-vjp6p" Sep 11 18:45:18.088965 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:18.088726 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ddbe2737-ad5d-4fd6-a3b5-7215b5bb5c57-signing-key\") pod \"service-ca-568869fdc5-vjp6p\" (UID: \"ddbe2737-ad5d-4fd6-a3b5-7215b5bb5c57\") " pod="openshift-service-ca/service-ca-568869fdc5-vjp6p" Sep 11 18:45:18.089338 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:18.089294 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ddbe2737-ad5d-4fd6-a3b5-7215b5bb5c57-signing-cabundle\") pod \"service-ca-568869fdc5-vjp6p\" (UID: \"ddbe2737-ad5d-4fd6-a3b5-7215b5bb5c57\") " pod="openshift-service-ca/service-ca-568869fdc5-vjp6p" Sep 11 18:45:18.092801 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:18.092779 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ddbe2737-ad5d-4fd6-a3b5-7215b5bb5c57-signing-key\") pod \"service-ca-568869fdc5-vjp6p\" (UID: \"ddbe2737-ad5d-4fd6-a3b5-7215b5bb5c57\") " pod="openshift-service-ca/service-ca-568869fdc5-vjp6p" Sep 11 18:45:18.096089 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:18.096049 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:18.099440 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:18.099417 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g66pj\" (UniqueName: \"kubernetes.io/projected/ddbe2737-ad5d-4fd6-a3b5-7215b5bb5c57-kube-api-access-g66pj\") pod \"service-ca-568869fdc5-vjp6p\" (UID: \"ddbe2737-ad5d-4fd6-a3b5-7215b5bb5c57\") " pod="openshift-service-ca/service-ca-568869fdc5-vjp6p" Sep 11 18:45:18.220785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:18.220702 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-vjp6p" Sep 11 18:45:18.398469 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:18.398437 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9xtl5" event={"ID":"e9819d65-2122-497e-831c-917aa1fabd52","Type":"ContainerStarted","Data":"e656edd4c8f264a99728400a74ef29bc035fcb3054a51faddf4c757ece27ee59"} Sep 11 18:45:18.400201 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:18.400176 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-p5gd9" event={"ID":"c39ed765-6eaa-420f-81e3-f47d5e972ff9","Type":"ContainerStarted","Data":"9e3637fa593e6c06a6050ede19097f52260c5e1d4194f7091be773d51d2abc40"} Sep 11 18:45:18.400319 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:18.400220 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-p5gd9" event={"ID":"c39ed765-6eaa-420f-81e3-f47d5e972ff9","Type":"ContainerStarted","Data":"41b3143d0242a876d1fbce6d5e2d4aca7c493d97c5dc399c2e7099eabde7dfde"} Sep 11 18:45:18.401594 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:18.401576 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7h9dc" event={"ID":"7a322a44-c645-482e-811b-cae2ea119a2f","Type":"ContainerStarted","Data":"da805335eb936e80f0157af9254f939f4ce40613aaed99638db4740856ed70c2"} Sep 11 18:45:18.435191 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:18.435145 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9xtl5" podStartSLOduration=10.72628068 podStartE2EDuration="39.435129514s" totalImagesPullingTime="28.708848834s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:44:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:45:18.43507995 +0000 UTC m=+39.897685968" watchObservedRunningTime="2026-09-11 18:45:18.435129514 +0000 UTC m=+39.897735532" Sep 11 18:45:18.472022 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:18.471867 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-p5gd9" podStartSLOduration=2.152775129 podStartE2EDuration="4.471850335s" totalImagesPullingTime="2.319075206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:45:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:45:18.470809501 +0000 UTC m=+39.933415518" watchObservedRunningTime="2026-09-11 18:45:18.471850335 +0000 UTC m=+39.934456353" Sep 11 18:45:18.491842 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:18.491794 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7h9dc" podStartSLOduration=2.5571514730000002 podStartE2EDuration="5.491776054s" totalImagesPullingTime="2.934624581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:45:18.491703143 +0000 UTC m=+39.954309161" watchObservedRunningTime="2026-09-11 18:45:18.491776054 +0000 UTC m=+39.954382072" Sep 11 18:45:18.521115 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:18.521085 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-vjp6p"] Sep 11 18:45:18.525086 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:45:18.525052 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddbe2737_ad5d_4fd6_a3b5_7215b5bb5c57.slice/crio-fcd0787897f56fb331575151f116de3e546c5b98dc35b5558ec7d9c340f28f96 WatchSource:0}: Error finding container fcd0787897f56fb331575151f116de3e546c5b98dc35b5558ec7d9c340f28f96: Status 404 returned error can't find the container with id fcd0787897f56fb331575151f116de3e546c5b98dc35b5558ec7d9c340f28f96 Sep 11 18:45:18.602845 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:18.602814 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:18.744998 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:18.744912 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:18.831136 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:18.831057 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:19.274777 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:19.274682 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:19.405723 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:19.405687 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-vjp6p" event={"ID":"ddbe2737-ad5d-4fd6-a3b5-7215b5bb5c57","Type":"ContainerStarted","Data":"fcd0787897f56fb331575151f116de3e546c5b98dc35b5558ec7d9c340f28f96"} Sep 11 18:45:19.641420 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:19.641332 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:19.739927 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:19.739885 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:21.114694 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:21.114587 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a1958ce-2c55-4c46-a2af-6b48ab70f9df-metrics-tls\") pod \"dns-default-dzb2p\" (UID: \"0a1958ce-2c55-4c46-a2af-6b48ab70f9df\") " pod="openshift-dns/dns-default-dzb2p" Sep 11 18:45:21.115066 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:21.114752 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 18:45:21.115066 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:21.114822 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a1958ce-2c55-4c46-a2af-6b48ab70f9df-metrics-tls podName:0a1958ce-2c55-4c46-a2af-6b48ab70f9df nodeName:}" failed. No retries permitted until 2026-09-11 18:45:29.114806319 +0000 UTC m=+50.577412316 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a1958ce-2c55-4c46-a2af-6b48ab70f9df-metrics-tls") pod "dns-default-dzb2p" (UID: "0a1958ce-2c55-4c46-a2af-6b48ab70f9df") : secret "dns-default-metrics-tls" not found Sep 11 18:45:21.412164 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:21.412071 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-vjp6p" event={"ID":"ddbe2737-ad5d-4fd6-a3b5-7215b5bb5c57","Type":"ContainerStarted","Data":"5d0f40261adfa1266d6338acf619724a6eea968005e8297e4abf0873ef764073"} Sep 11 18:45:21.429934 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:21.429873 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-vjp6p" podStartSLOduration=2.13557561 podStartE2EDuration="4.429854762s" totalImagesPullingTime="2.294279152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:45:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:45:21.429288875 +0000 UTC m=+42.891894892" watchObservedRunningTime="2026-09-11 18:45:21.429854762 +0000 UTC m=+42.892460780" Sep 11 18:45:21.619557 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:21.619486 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-65929\" (UID: \"1c275a87-26b0-4fd7-a4fc-8f12b496a04e\") " pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:21.619738 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:21.619627 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-65929\" (UID: \"1c275a87-26b0-4fd7-a4fc-8f12b496a04e\") " pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:21.619738 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:21.619638 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 18:45:21.619808 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:21.619783 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-insights-runtime-extractor-tls podName:1c275a87-26b0-4fd7-a4fc-8f12b496a04e nodeName:}" failed. No retries permitted until 2026-09-11 18:45:29.619765542 +0000 UTC m=+51.082371538 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-65929" (UID: "1c275a87-26b0-4fd7-a4fc-8f12b496a04e") : secret "insights-runtime-extractor-tls" not found Sep 11 18:45:21.619808 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:21.619670 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 11 18:45:21.619879 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:21.619820 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-kube-rbac-proxy-cm podName:1c275a87-26b0-4fd7-a4fc-8f12b496a04e nodeName:}" failed. No retries permitted until 2026-09-11 18:45:29.619813896 +0000 UTC m=+51.082419892 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-65929" (UID: "1c275a87-26b0-4fd7-a4fc-8f12b496a04e") : configmap "kube-rbac-proxy" not found Sep 11 18:45:22.669781 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:22.669745 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:22.855116 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:22.855076 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:22.955415 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:22.955322 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:23.069647 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:23.069613 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:23.131351 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:23.131318 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:23.245051 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:23.244971 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:23.343969 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:23.343935 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:23.447004 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:23.446966 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:23.998800 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:23.998766 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:24.584047 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:24.584013 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:24.970425 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:24.970340 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:25.370029 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:25.369993 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:25.568283 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:25.568250 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:25.983383 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:25.983347 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:26.337409 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:26.337373 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:26.436782 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:26.436755 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:26.584606 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:26.584573 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:27.004800 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:27.004773 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:27.183382 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:27.183346 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:27.271861 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:27.271831 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:27.382361 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:27.382280 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:27.610923 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:27.610815 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:27.982457 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:27.982322 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:28.772827 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:28.772798 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:28.871545 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:28.871465 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:29.177188 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:29.177082 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:29.182335 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:29.182314 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a1958ce-2c55-4c46-a2af-6b48ab70f9df-metrics-tls\") pod \"dns-default-dzb2p\" (UID: \"0a1958ce-2c55-4c46-a2af-6b48ab70f9df\") " pod="openshift-dns/dns-default-dzb2p" Sep 11 18:45:29.185250 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:29.185227 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a1958ce-2c55-4c46-a2af-6b48ab70f9df-metrics-tls\") pod \"dns-default-dzb2p\" (UID: \"0a1958ce-2c55-4c46-a2af-6b48ab70f9df\") " pod="openshift-dns/dns-default-dzb2p" Sep 11 18:45:29.255008 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:29.254963 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dzb2p" Sep 11 18:45:29.374871 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:29.374812 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:29.533182 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:29.533150 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dzb2p"] Sep 11 18:45:29.537534 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:45:29.537475 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a1958ce_2c55_4c46_a2af_6b48ab70f9df.slice/crio-b77a9b1cd115a80b15f1ba901e46be986fdf7951c6c10466bd78b4db07b72bde WatchSource:0}: Error finding container b77a9b1cd115a80b15f1ba901e46be986fdf7951c6c10466bd78b4db07b72bde: Status 404 returned error can't find the container with id b77a9b1cd115a80b15f1ba901e46be986fdf7951c6c10466bd78b4db07b72bde Sep 11 18:45:29.566552 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:29.566514 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:29.686571 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:29.686533 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-65929\" (UID: \"1c275a87-26b0-4fd7-a4fc-8f12b496a04e\") " pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:29.686857 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:29.686601 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-65929\" (UID: \"1c275a87-26b0-4fd7-a4fc-8f12b496a04e\") " pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:29.686857 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:29.686712 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 18:45:29.686857 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:29.686798 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-insights-runtime-extractor-tls podName:1c275a87-26b0-4fd7-a4fc-8f12b496a04e nodeName:}" failed. No retries permitted until 2026-09-11 18:45:45.686776053 +0000 UTC m=+67.149382069 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-65929" (UID: "1c275a87-26b0-4fd7-a4fc-8f12b496a04e") : secret "insights-runtime-extractor-tls" not found Sep 11 18:45:29.687139 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:29.687114 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-65929\" (UID: \"1c275a87-26b0-4fd7-a4fc-8f12b496a04e\") " pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:29.972873 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:29.972784 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:30.394552 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:30.394498 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:30.444105 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:30.444065 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dzb2p" event={"ID":"0a1958ce-2c55-4c46-a2af-6b48ab70f9df","Type":"ContainerStarted","Data":"b77a9b1cd115a80b15f1ba901e46be986fdf7951c6c10466bd78b4db07b72bde"} Sep 11 18:45:30.767641 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:30.767613 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:30.866918 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:30.866871 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:30.967313 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:30.967278 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:31.031051 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:31.030951 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:31.176115 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:31.176082 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:31.274048 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:31.274016 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:31.435248 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:31.435222 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:31.447969 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:31.447943 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dzb2p" event={"ID":"0a1958ce-2c55-4c46-a2af-6b48ab70f9df","Type":"ContainerStarted","Data":"515bca989791208f66e136ffc298c6cd49becde6c66b0739ba6e1bcf9d7a3cea"} Sep 11 18:45:31.534004 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:31.533973 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:31.634946 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:31.634861 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:32.379497 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:32.379467 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:32.453083 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:32.453046 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dzb2p" event={"ID":"0a1958ce-2c55-4c46-a2af-6b48ab70f9df","Type":"ContainerStarted","Data":"1faaf9cbb00f9c3b68f58d25b33e25556552b9bd70431645a627237c633b320b"} Sep 11 18:45:32.453241 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:32.453164 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dzb2p" Sep 11 18:45:32.472099 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:32.472056 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dzb2p" podStartSLOduration=17.723737018 podStartE2EDuration="19.472041195s" totalImagesPullingTime="1.748304177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:45:32.470956189 +0000 UTC m=+53.933562207" watchObservedRunningTime="2026-09-11 18:45:32.472041195 +0000 UTC m=+53.934647213" Sep 11 18:45:32.475139 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:32.475114 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:32.534554 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:32.534521 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:32.667055 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:32.666968 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:32.773192 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:32.773160 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:32.872873 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:32.872843 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:32.976644 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:32.976563 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:33.176287 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:33.176258 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:33.370288 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:33.370245 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:33.776116 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:33.776075 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:36.658521 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:36.658473 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:39.373350 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:39.373317 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8tmtk" Sep 11 18:45:39.598802 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:39.598769 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:39.701674 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:39.701600 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:39.923808 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:39.923772 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7845949556-l5b5w"] Sep 11 18:45:39.926359 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:39.926342 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7845949556-l5b5w" Sep 11 18:45:39.929222 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:39.929192 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d"] Sep 11 18:45:39.930246 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:39.930224 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 11 18:45:39.930367 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:39.930225 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 11 18:45:39.930687 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:39.930665 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-gsg79\"" Sep 11 18:45:39.930687 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:39.930683 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 11 18:45:39.931451 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:39.931433 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d" Sep 11 18:45:39.931662 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:39.931641 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Sep 11 18:45:39.934168 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:39.934129 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Sep 11 18:45:39.934168 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:39.934134 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Sep 11 18:45:39.935046 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:39.935028 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Sep 11 18:45:39.935260 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:39.935245 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Sep 11 18:45:39.946872 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:39.946851 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7845949556-l5b5w"] Sep 11 18:45:39.953621 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:39.953554 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d"] Sep 11 18:45:40.058355 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.058323 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:40.063375 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.063352 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f729e82-ee53-4758-b347-fa7b5958c5b9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-848464499b-sqb2d\" (UID: \"3f729e82-ee53-4758-b347-fa7b5958c5b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d" Sep 11 18:45:40.063524 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.063385 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flz6m\" (UniqueName: \"kubernetes.io/projected/9831dca9-75dc-46d7-8da0-759c72029482-kube-api-access-flz6m\") pod \"managed-serviceaccount-addon-agent-7845949556-l5b5w\" (UID: \"9831dca9-75dc-46d7-8da0-759c72029482\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7845949556-l5b5w" Sep 11 18:45:40.063524 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.063439 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3f729e82-ee53-4758-b347-fa7b5958c5b9-hub\") pod \"cluster-proxy-proxy-agent-848464499b-sqb2d\" (UID: \"3f729e82-ee53-4758-b347-fa7b5958c5b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d" Sep 11 18:45:40.063631 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.063549 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3f729e82-ee53-4758-b347-fa7b5958c5b9-ca\") pod \"cluster-proxy-proxy-agent-848464499b-sqb2d\" (UID: \"3f729e82-ee53-4758-b347-fa7b5958c5b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d" Sep 11 18:45:40.063631 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.063576 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hqhc\" (UniqueName: \"kubernetes.io/projected/3f729e82-ee53-4758-b347-fa7b5958c5b9-kube-api-access-9hqhc\") pod \"cluster-proxy-proxy-agent-848464499b-sqb2d\" (UID: \"3f729e82-ee53-4758-b347-fa7b5958c5b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d" Sep 11 18:45:40.063631 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.063595 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9831dca9-75dc-46d7-8da0-759c72029482-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7845949556-l5b5w\" (UID: \"9831dca9-75dc-46d7-8da0-759c72029482\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7845949556-l5b5w" Sep 11 18:45:40.063631 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.063616 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3f729e82-ee53-4758-b347-fa7b5958c5b9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-848464499b-sqb2d\" (UID: \"3f729e82-ee53-4758-b347-fa7b5958c5b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d" Sep 11 18:45:40.063766 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.063640 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3f729e82-ee53-4758-b347-fa7b5958c5b9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-848464499b-sqb2d\" (UID: \"3f729e82-ee53-4758-b347-fa7b5958c5b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d" Sep 11 18:45:40.163966 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.163937 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f729e82-ee53-4758-b347-fa7b5958c5b9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-848464499b-sqb2d\" (UID: \"3f729e82-ee53-4758-b347-fa7b5958c5b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d" Sep 11 18:45:40.163966 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.163975 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flz6m\" (UniqueName: \"kubernetes.io/projected/9831dca9-75dc-46d7-8da0-759c72029482-kube-api-access-flz6m\") pod \"managed-serviceaccount-addon-agent-7845949556-l5b5w\" (UID: \"9831dca9-75dc-46d7-8da0-759c72029482\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7845949556-l5b5w" Sep 11 18:45:40.164218 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.164021 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3f729e82-ee53-4758-b347-fa7b5958c5b9-hub\") pod \"cluster-proxy-proxy-agent-848464499b-sqb2d\" (UID: \"3f729e82-ee53-4758-b347-fa7b5958c5b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d" Sep 11 18:45:40.164218 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.164082 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3f729e82-ee53-4758-b347-fa7b5958c5b9-ca\") pod \"cluster-proxy-proxy-agent-848464499b-sqb2d\" (UID: \"3f729e82-ee53-4758-b347-fa7b5958c5b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d" Sep 11 18:45:40.164218 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.164131 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hqhc\" (UniqueName: \"kubernetes.io/projected/3f729e82-ee53-4758-b347-fa7b5958c5b9-kube-api-access-9hqhc\") pod \"cluster-proxy-proxy-agent-848464499b-sqb2d\" (UID: \"3f729e82-ee53-4758-b347-fa7b5958c5b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d" Sep 11 18:45:40.164218 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.164183 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9831dca9-75dc-46d7-8da0-759c72029482-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7845949556-l5b5w\" (UID: \"9831dca9-75dc-46d7-8da0-759c72029482\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7845949556-l5b5w" Sep 11 18:45:40.164412 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.164230 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3f729e82-ee53-4758-b347-fa7b5958c5b9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-848464499b-sqb2d\" (UID: \"3f729e82-ee53-4758-b347-fa7b5958c5b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d" Sep 11 18:45:40.164412 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.164283 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3f729e82-ee53-4758-b347-fa7b5958c5b9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-848464499b-sqb2d\" (UID: \"3f729e82-ee53-4758-b347-fa7b5958c5b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d" Sep 11 18:45:40.164882 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.164856 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f729e82-ee53-4758-b347-fa7b5958c5b9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-848464499b-sqb2d\" (UID: \"3f729e82-ee53-4758-b347-fa7b5958c5b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d" Sep 11 18:45:40.167111 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.167080 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3f729e82-ee53-4758-b347-fa7b5958c5b9-hub\") pod \"cluster-proxy-proxy-agent-848464499b-sqb2d\" (UID: \"3f729e82-ee53-4758-b347-fa7b5958c5b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d" Sep 11 18:45:40.167718 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.167695 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3f729e82-ee53-4758-b347-fa7b5958c5b9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-848464499b-sqb2d\" (UID: \"3f729e82-ee53-4758-b347-fa7b5958c5b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d" Sep 11 18:45:40.167806 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.167773 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3f729e82-ee53-4758-b347-fa7b5958c5b9-ca\") pod \"cluster-proxy-proxy-agent-848464499b-sqb2d\" (UID: \"3f729e82-ee53-4758-b347-fa7b5958c5b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d" Sep 11 18:45:40.167849 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.167772 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:40.167849 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.167828 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9831dca9-75dc-46d7-8da0-759c72029482-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7845949556-l5b5w\" (UID: \"9831dca9-75dc-46d7-8da0-759c72029482\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7845949556-l5b5w" Sep 11 18:45:40.167960 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.167946 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3f729e82-ee53-4758-b347-fa7b5958c5b9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-848464499b-sqb2d\" (UID: \"3f729e82-ee53-4758-b347-fa7b5958c5b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d" Sep 11 18:45:40.175861 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.175840 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hqhc\" (UniqueName: \"kubernetes.io/projected/3f729e82-ee53-4758-b347-fa7b5958c5b9-kube-api-access-9hqhc\") pod \"cluster-proxy-proxy-agent-848464499b-sqb2d\" (UID: \"3f729e82-ee53-4758-b347-fa7b5958c5b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d" Sep 11 18:45:40.177248 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.177228 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flz6m\" (UniqueName: \"kubernetes.io/projected/9831dca9-75dc-46d7-8da0-759c72029482-kube-api-access-flz6m\") pod \"managed-serviceaccount-addon-agent-7845949556-l5b5w\" (UID: \"9831dca9-75dc-46d7-8da0-759c72029482\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7845949556-l5b5w" Sep 11 18:45:40.253169 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.253076 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7845949556-l5b5w" Sep 11 18:45:40.259772 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.259542 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d" Sep 11 18:45:40.270745 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.270720 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:40.525108 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.525077 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7845949556-l5b5w"] Sep 11 18:45:40.527809 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:45:40.527782 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9831dca9_75dc_46d7_8da0_759c72029482.slice/crio-b942c5c6d8622f66acb053bb07f2a94a84ac6790cb328f84a66f4a7fd46148f9 WatchSource:0}: Error finding container b942c5c6d8622f66acb053bb07f2a94a84ac6790cb328f84a66f4a7fd46148f9: Status 404 returned error can't find the container with id b942c5c6d8622f66acb053bb07f2a94a84ac6790cb328f84a66f4a7fd46148f9 Sep 11 18:45:40.530034 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:40.530008 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d"] Sep 11 18:45:40.532541 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:45:40.532517 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f729e82_ee53_4758_b347_fa7b5958c5b9.slice/crio-df86d39f9c9654135add9a5de3163b26655c1e21a9120ee9bc52341bf03bdad9 WatchSource:0}: Error finding container df86d39f9c9654135add9a5de3163b26655c1e21a9120ee9bc52341bf03bdad9: Status 404 returned error can't find the container with id df86d39f9c9654135add9a5de3163b26655c1e21a9120ee9bc52341bf03bdad9 Sep 11 18:45:41.060123 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:41.060078 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:41.124013 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:41.123980 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:41.438054 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:41.437962 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:41.486023 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:41.485961 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7845949556-l5b5w" event={"ID":"9831dca9-75dc-46d7-8da0-759c72029482","Type":"ContainerStarted","Data":"b942c5c6d8622f66acb053bb07f2a94a84ac6790cb328f84a66f4a7fd46148f9"} Sep 11 18:45:41.487319 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:41.487290 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d" event={"ID":"3f729e82-ee53-4758-b347-fa7b5958c5b9","Type":"ContainerStarted","Data":"df86d39f9c9654135add9a5de3163b26655c1e21a9120ee9bc52341bf03bdad9"} Sep 11 18:45:41.764041 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:41.764005 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:41.823885 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:41.823839 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:42.462238 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:42.462202 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dzb2p" Sep 11 18:45:42.896753 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:42.896689 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:42.997261 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:42.997229 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:43.092708 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:43.092675 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:43.404496 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:43.404460 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:43.497470 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:43.497433 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7845949556-l5b5w" event={"ID":"9831dca9-75dc-46d7-8da0-759c72029482","Type":"ContainerStarted","Data":"efa6f310f4df5ba001512fea5717306130798824c384b8f3968b9c72e9de3424"} Sep 11 18:45:43.515487 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:43.515427 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7845949556-l5b5w" podStartSLOduration=1.636387718 podStartE2EDuration="4.515409681s" totalImagesPullingTime="2.879021963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:45:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:45:43.51464478 +0000 UTC m=+64.977250798" watchObservedRunningTime="2026-09-11 18:45:43.515409681 +0000 UTC m=+64.978015698" Sep 11 18:45:43.875290 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:43.875250 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:43.957368 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:43.957291 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:44.702199 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:44.702175 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b799f9cb-ce50-4731-9fbe-48f47807a053-original-pull-secret\") pod \"global-pull-secret-syncer-sxqg6\" (UID: \"b799f9cb-ce50-4731-9fbe-48f47807a053\") " pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:45:44.704983 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:44.704963 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 11 18:45:44.714607 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:44.714581 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b799f9cb-ce50-4731-9fbe-48f47807a053-original-pull-secret\") pod \"global-pull-secret-syncer-sxqg6\" (UID: \"b799f9cb-ce50-4731-9fbe-48f47807a053\") " pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:45:44.773523 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:44.773476 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxqg6" Sep 11 18:45:44.803439 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:44.803349 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b644d098-032f-42c9-9839-41ce9eb09844-metrics-certs\") pod \"network-metrics-daemon-lhj2z\" (UID: \"b644d098-032f-42c9-9839-41ce9eb09844\") " pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:45:44.806151 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:44.806125 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 11 18:45:44.816481 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:44.816456 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b644d098-032f-42c9-9839-41ce9eb09844-metrics-certs\") pod \"network-metrics-daemon-lhj2z\" (UID: \"b644d098-032f-42c9-9839-41ce9eb09844\") " pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:45:44.904582 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:44.904469 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhskz\" (UniqueName: \"kubernetes.io/projected/254d8445-a0f4-4c11-bc96-2cb74c34ab13-kube-api-access-jhskz\") pod \"network-check-target-rxmht\" (UID: \"254d8445-a0f4-4c11-bc96-2cb74c34ab13\") " pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:45:44.907621 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:44.907602 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 11 18:45:44.917387 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:44.917363 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 11 18:45:44.928249 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:44.928218 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhskz\" (UniqueName: \"kubernetes.io/projected/254d8445-a0f4-4c11-bc96-2cb74c34ab13-kube-api-access-jhskz\") pod \"network-check-target-rxmht\" (UID: \"254d8445-a0f4-4c11-bc96-2cb74c34ab13\") " pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:45:45.042850 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:45.042818 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sxqg6"] Sep 11 18:45:45.045889 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:45:45.045865 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb799f9cb_ce50_4731_9fbe_48f47807a053.slice/crio-fdd7b25dcf2382e209cbd92fa8f5e0817fccbbd4d8248334e50866dbeb899048 WatchSource:0}: Error finding container fdd7b25dcf2382e209cbd92fa8f5e0817fccbbd4d8248334e50866dbeb899048: Status 404 returned error can't find the container with id fdd7b25dcf2382e209cbd92fa8f5e0817fccbbd4d8248334e50866dbeb899048 Sep 11 18:45:45.070421 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:45.070391 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-whrnn\"" Sep 11 18:45:45.077668 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:45.077647 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lhj2z" Sep 11 18:45:45.082431 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:45.082411 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-p4phg\"" Sep 11 18:45:45.090756 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:45.090731 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:45:45.345725 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:45.345693 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lhj2z"] Sep 11 18:45:45.348555 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:45:45.348530 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb644d098_032f_42c9_9839_41ce9eb09844.slice/crio-aad79bb7858fef7c76a52de017949b9aa25bc39f3c5691478d613db6dcc57f39 WatchSource:0}: Error finding container aad79bb7858fef7c76a52de017949b9aa25bc39f3c5691478d613db6dcc57f39: Status 404 returned error can't find the container with id aad79bb7858fef7c76a52de017949b9aa25bc39f3c5691478d613db6dcc57f39 Sep 11 18:45:45.368719 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:45.368688 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rxmht"] Sep 11 18:45:45.371316 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:45:45.371295 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod254d8445_a0f4_4c11_bc96_2cb74c34ab13.slice/crio-63010fef8fe87dd6da6e1541511493b00c14304d604f892c93c124fde8d821dd WatchSource:0}: Error finding container 63010fef8fe87dd6da6e1541511493b00c14304d604f892c93c124fde8d821dd: Status 404 returned error can't find the container with id 63010fef8fe87dd6da6e1541511493b00c14304d604f892c93c124fde8d821dd Sep 11 18:45:45.506150 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:45.506112 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d" event={"ID":"3f729e82-ee53-4758-b347-fa7b5958c5b9","Type":"ContainerStarted","Data":"0a5745d16341feca6a010ea80c8b69b159129e4a3bf654f9b0feb5ec908b9c2b"} Sep 11 18:45:45.506438 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:45.506159 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d" event={"ID":"3f729e82-ee53-4758-b347-fa7b5958c5b9","Type":"ContainerStarted","Data":"81e91f033f4099c7465f7330b847ecbdf4ae4a53630181196a88dd690f479332"} Sep 11 18:45:45.506438 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:45.506177 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d" event={"ID":"3f729e82-ee53-4758-b347-fa7b5958c5b9","Type":"ContainerStarted","Data":"f87e2f5f30ea2cb875df6e022ce2a78ce97a81b297bf07d0e2ca44f1826d2f2c"} Sep 11 18:45:45.507425 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:45.507395 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rxmht" event={"ID":"254d8445-a0f4-4c11-bc96-2cb74c34ab13","Type":"ContainerStarted","Data":"63010fef8fe87dd6da6e1541511493b00c14304d604f892c93c124fde8d821dd"} Sep 11 18:45:45.508843 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:45.508792 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lhj2z" event={"ID":"b644d098-032f-42c9-9839-41ce9eb09844","Type":"ContainerStarted","Data":"aad79bb7858fef7c76a52de017949b9aa25bc39f3c5691478d613db6dcc57f39"} Sep 11 18:45:45.509965 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:45.509944 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sxqg6" event={"ID":"b799f9cb-ce50-4731-9fbe-48f47807a053","Type":"ContainerStarted","Data":"fdd7b25dcf2382e209cbd92fa8f5e0817fccbbd4d8248334e50866dbeb899048"} Sep 11 18:45:45.547107 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:45.547053 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848464499b-sqb2d" podStartSLOduration=2.489454626 podStartE2EDuration="6.547036395s" totalImagesPullingTime="4.057581769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:45:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:45:45.545565407 +0000 UTC m=+67.008171426" watchObservedRunningTime="2026-09-11 18:45:45.547036395 +0000 UTC m=+67.009642412" Sep 11 18:45:45.663149 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:45.663026 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:45.709445 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:45.709411 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-65929\" (UID: \"1c275a87-26b0-4fd7-a4fc-8f12b496a04e\") " pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:45.711755 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:45.711738 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1c275a87-26b0-4fd7-a4fc-8f12b496a04e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-65929\" (UID: \"1c275a87-26b0-4fd7-a4fc-8f12b496a04e\") " pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:45.719997 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:45.719954 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:45.929809 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:45.929688 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:45.973197 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:45.973166 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-c5jnz\"" Sep 11 18:45:45.980952 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:45.980913 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-65929" Sep 11 18:45:46.031079 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:46.031024 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:46.284820 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:46.284782 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-65929"] Sep 11 18:45:46.439704 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:45:46.439664 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c275a87_26b0_4fd7_a4fc_8f12b496a04e.slice/crio-9f548a945c319a30333f93e00ee5ad0442cc7208a1df28f526c2bdb7a7dc7119 WatchSource:0}: Error finding container 9f548a945c319a30333f93e00ee5ad0442cc7208a1df28f526c2bdb7a7dc7119: Status 404 returned error can't find the container with id 9f548a945c319a30333f93e00ee5ad0442cc7208a1df28f526c2bdb7a7dc7119 Sep 11 18:45:46.518996 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:46.518754 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-65929" event={"ID":"1c275a87-26b0-4fd7-a4fc-8f12b496a04e","Type":"ContainerStarted","Data":"9f548a945c319a30333f93e00ee5ad0442cc7208a1df28f526c2bdb7a7dc7119"} Sep 11 18:45:47.116530 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:47.116383 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:47.199303 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:47.199225 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:47.528019 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:47.527572 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lhj2z" event={"ID":"b644d098-032f-42c9-9839-41ce9eb09844","Type":"ContainerStarted","Data":"bbacc26b29786dad3b5c8fed02a50b26b31f635f9ec2f2db55759b01f4ba1c20"} Sep 11 18:45:47.528019 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:47.527640 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lhj2z" event={"ID":"b644d098-032f-42c9-9839-41ce9eb09844","Type":"ContainerStarted","Data":"4fe443ea7a0bc81be0db661b33d5bdeab689f9895904c9e98f32ffa5fea7ea8c"} Sep 11 18:45:47.530110 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:47.530080 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-65929" event={"ID":"1c275a87-26b0-4fd7-a4fc-8f12b496a04e","Type":"ContainerStarted","Data":"737ebbd1e04ea4a9b0122ed704cf775c3ee9ce68067e0259588014f727351745"} Sep 11 18:45:47.549765 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:47.549710 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lhj2z" podStartSLOduration=67.432665619 podStartE2EDuration="1m8.54968954s" totalImagesPullingTime="1.117023921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:44:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:45:47.547531517 +0000 UTC m=+69.010137535" watchObservedRunningTime="2026-09-11 18:45:47.54968954 +0000 UTC m=+69.012295559" Sep 11 18:45:47.648301 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:47.648262 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:47.707700 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:47.707638 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:49.674388 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:49.674353 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:49.774432 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:49.774391 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:50.269364 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:50.269336 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:51.547383 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:51.547161 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rxmht" event={"ID":"254d8445-a0f4-4c11-bc96-2cb74c34ab13","Type":"ContainerStarted","Data":"aa4ddce2969a76f53d3f8a3f78fae6509812dcd3002da45ab205ba04fcca22f6"} Sep 11 18:45:51.547383 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:51.547275 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:45:51.548967 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:51.548919 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sxqg6" event={"ID":"b799f9cb-ce50-4731-9fbe-48f47807a053","Type":"ContainerStarted","Data":"6f5f46ec676fe004de47bfd58322b0351b7bae220c1ed6df8b1147a0cf419198"} Sep 11 18:45:51.551003 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:51.550982 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-65929" event={"ID":"1c275a87-26b0-4fd7-a4fc-8f12b496a04e","Type":"ContainerStarted","Data":"c6a37197e1f0bff5ba90e85554e6924e6f900b39c5c33e2a2fe8b8fc0e3054d0"} Sep 11 18:45:51.567932 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:51.567873 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rxmht" podStartSLOduration=67.219355945 podStartE2EDuration="1m12.567851074s" totalImagesPullingTime="5.348495129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:44:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:45:51.567277955 +0000 UTC m=+73.029883973" watchObservedRunningTime="2026-09-11 18:45:51.567851074 +0000 UTC m=+73.030457094" Sep 11 18:45:51.588652 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:51.588590 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-sxqg6" podStartSLOduration=66.914546246 podStartE2EDuration="1m12.588568673s" totalImagesPullingTime="5.674022427s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:44:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:45:51.588111268 +0000 UTC m=+73.050717286" watchObservedRunningTime="2026-09-11 18:45:51.588568673 +0000 UTC m=+73.051174691" Sep 11 18:45:51.609795 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:51.609769 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:52.249123 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.249088 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:52.367109 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.367082 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:52.475039 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.474987 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:52.501618 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.501526 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jm77n"] Sep 11 18:45:52.503929 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.503905 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:52.507292 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.507024 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-g7pgw\"" Sep 11 18:45:52.507292 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.507059 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 11 18:45:52.507292 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.507187 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 11 18:45:52.508087 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.508068 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 11 18:45:52.508847 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.508824 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 11 18:45:52.510332 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.509712 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 11 18:45:52.513805 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.510686 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 11 18:45:52.555892 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.555853 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/078f13cc-8c84-4ebc-9452-d235511c38b1-node-exporter-accelerators-collector-config\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:52.556342 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.555904 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/078f13cc-8c84-4ebc-9452-d235511c38b1-node-exporter-wtmp\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:52.556342 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.555928 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/078f13cc-8c84-4ebc-9452-d235511c38b1-node-exporter-textfile\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:52.556342 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.556001 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/078f13cc-8c84-4ebc-9452-d235511c38b1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:52.556342 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.556039 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/078f13cc-8c84-4ebc-9452-d235511c38b1-root\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:52.556342 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.556084 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/078f13cc-8c84-4ebc-9452-d235511c38b1-node-exporter-tls\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:52.556342 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.556112 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/078f13cc-8c84-4ebc-9452-d235511c38b1-metrics-client-ca\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:52.556342 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.556155 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmn66\" (UniqueName: \"kubernetes.io/projected/078f13cc-8c84-4ebc-9452-d235511c38b1-kube-api-access-cmn66\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:52.556342 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.556218 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/078f13cc-8c84-4ebc-9452-d235511c38b1-sys\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:52.556805 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.556781 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-65929" event={"ID":"1c275a87-26b0-4fd7-a4fc-8f12b496a04e","Type":"ContainerStarted","Data":"b25b96528e2c2a95eab7210f92f74d7c948de56b173e49293dabacb0f9e04d0c"} Sep 11 18:45:52.571019 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.570985 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:52.584689 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.583732 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-65929" podStartSLOduration=33.911381514 podStartE2EDuration="39.58371172s" totalImagesPullingTime="5.672330206s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:45:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:45:52.583381192 +0000 UTC m=+74.045987211" watchObservedRunningTime="2026-09-11 18:45:52.58371172 +0000 UTC m=+74.046317740" Sep 11 18:45:52.656754 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.656715 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/078f13cc-8c84-4ebc-9452-d235511c38b1-node-exporter-accelerators-collector-config\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:52.656754 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.656753 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/078f13cc-8c84-4ebc-9452-d235511c38b1-node-exporter-wtmp\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:52.657015 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.656771 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/078f13cc-8c84-4ebc-9452-d235511c38b1-node-exporter-textfile\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:52.657015 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.656844 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/078f13cc-8c84-4ebc-9452-d235511c38b1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:52.657015 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.656868 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/078f13cc-8c84-4ebc-9452-d235511c38b1-root\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:52.657151 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.657034 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/078f13cc-8c84-4ebc-9452-d235511c38b1-node-exporter-wtmp\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:52.657151 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.657109 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/078f13cc-8c84-4ebc-9452-d235511c38b1-node-exporter-tls\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:52.657242 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.657161 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/078f13cc-8c84-4ebc-9452-d235511c38b1-metrics-client-ca\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:52.657242 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.657187 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmn66\" (UniqueName: \"kubernetes.io/projected/078f13cc-8c84-4ebc-9452-d235511c38b1-kube-api-access-cmn66\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:52.657335 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.657258 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/078f13cc-8c84-4ebc-9452-d235511c38b1-sys\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:52.657390 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.657353 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/078f13cc-8c84-4ebc-9452-d235511c38b1-sys\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:52.657390 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.657368 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/078f13cc-8c84-4ebc-9452-d235511c38b1-root\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:52.657390 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.657377 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/078f13cc-8c84-4ebc-9452-d235511c38b1-node-exporter-accelerators-collector-config\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:52.657545 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:52.657452 2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 11 18:45:52.657598 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:45:52.657552 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/078f13cc-8c84-4ebc-9452-d235511c38b1-node-exporter-tls podName:078f13cc-8c84-4ebc-9452-d235511c38b1 nodeName:}" failed. No retries permitted until 2026-09-11 18:45:53.15753154 +0000 UTC m=+74.620137551 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/078f13cc-8c84-4ebc-9452-d235511c38b1-node-exporter-tls") pod "node-exporter-jm77n" (UID: "078f13cc-8c84-4ebc-9452-d235511c38b1") : secret "node-exporter-tls" not found Sep 11 18:45:52.657598 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.657566 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/078f13cc-8c84-4ebc-9452-d235511c38b1-node-exporter-textfile\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:52.657970 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.657948 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/078f13cc-8c84-4ebc-9452-d235511c38b1-metrics-client-ca\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:52.659712 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.659682 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/078f13cc-8c84-4ebc-9452-d235511c38b1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:52.676184 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.676150 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmn66\" (UniqueName: \"kubernetes.io/projected/078f13cc-8c84-4ebc-9452-d235511c38b1-kube-api-access-cmn66\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:52.682756 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.682727 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:52.780295 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.780264 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:52.883859 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.883824 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:52.983633 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:52.983599 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:53.088041 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:53.087964 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:53.160482 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:53.160441 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/078f13cc-8c84-4ebc-9452-d235511c38b1-node-exporter-tls\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:53.162732 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:53.162712 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/078f13cc-8c84-4ebc-9452-d235511c38b1-node-exporter-tls\") pod \"node-exporter-jm77n\" (UID: \"078f13cc-8c84-4ebc-9452-d235511c38b1\") " pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:53.418323 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:53.418238 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jm77n" Sep 11 18:45:53.427816 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:45:53.427774 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod078f13cc_8c84_4ebc_9452_d235511c38b1.slice/crio-fe5174d7d672b9e60501aa15254264e45530bd01c188a8d39ab201abe3ffdc75 WatchSource:0}: Error finding container fe5174d7d672b9e60501aa15254264e45530bd01c188a8d39ab201abe3ffdc75: Status 404 returned error can't find the container with id fe5174d7d672b9e60501aa15254264e45530bd01c188a8d39ab201abe3ffdc75 Sep 11 18:45:53.519767 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:53.519735 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:53.560932 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:53.560890 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jm77n" event={"ID":"078f13cc-8c84-4ebc-9452-d235511c38b1","Type":"ContainerStarted","Data":"fe5174d7d672b9e60501aa15254264e45530bd01c188a8d39ab201abe3ffdc75"} Sep 11 18:45:53.607885 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:53.607848 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:53.693188 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:53.693102 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:54.565613 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:54.565572 2563 generic.go:356] "Generic (PLEG): container finished" podID="078f13cc-8c84-4ebc-9452-d235511c38b1" containerID="49a0d5b394433a6a35c4fe9112a600ddc72a61c4856ccbbd61ee33403f64076a" exitCode=0 Sep 11 18:45:54.566022 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:54.565662 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jm77n" event={"ID":"078f13cc-8c84-4ebc-9452-d235511c38b1","Type":"ContainerDied","Data":"49a0d5b394433a6a35c4fe9112a600ddc72a61c4856ccbbd61ee33403f64076a"} Sep 11 18:45:55.035917 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:55.035880 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:55.136236 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:55.136203 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:55.480185 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:55.480069 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:55.572661 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:55.572605 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jm77n" event={"ID":"078f13cc-8c84-4ebc-9452-d235511c38b1","Type":"ContainerStarted","Data":"237205bfd00077d89a00ec8d0d34dd61de0337501cc94a27fe9d8afaac9c920a"} Sep 11 18:45:55.573018 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:55.572678 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jm77n" event={"ID":"078f13cc-8c84-4ebc-9452-d235511c38b1","Type":"ContainerStarted","Data":"80c8c818e4c5f640bc925c7b0115db2b4ee4cdecb18dc8e36ae221f8b4bea80b"} Sep 11 18:45:55.576899 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:55.576870 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:55.596516 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:55.596443 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jm77n" podStartSLOduration=2.7622156049999997 podStartE2EDuration="3.596420241s" totalImagesPullingTime="771.673861ms" totalInitContainerRuntime="62.530775ms" isStatefulPod=true podCreationTimestamp="2026-09-11 18:45:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:45:55.594583166 +0000 UTC m=+77.057189187" watchObservedRunningTime="2026-09-11 18:45:55.596420241 +0000 UTC m=+77.059026261" Sep 11 18:45:55.702395 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:55.702362 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:55.801162 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:55.801128 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:55.859766 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:55.859728 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:56.248699 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:56.248616 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:56.891367 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:56.891330 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:56.956586 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:56.956553 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:57.277928 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:57.277892 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:57.402903 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:57.402871 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:57.462253 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:57.462220 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:58.902231 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:58.902108 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:45:58.903949 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:58.903927 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 11 18:45:58.906406 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:58.906390 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:58.909176 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:58.909153 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 11 18:45:58.909283 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:58.909154 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 11 18:45:58.909647 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:58.909625 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-f84vmbqctpnh0\"" Sep 11 18:45:58.909761 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:58.909703 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 11 18:45:58.910498 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:58.910482 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 11 18:45:58.910733 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:58.910719 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-ddrh7\"" Sep 11 18:45:58.910803 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:58.910790 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 11 18:45:58.911203 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:58.911181 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 11 18:45:58.911282 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:58.911201 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 11 18:45:58.911282 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:58.911236 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 11 18:45:58.911282 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:58.911205 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 11 18:45:58.911445 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:58.911185 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 11 18:45:58.911832 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:58.911816 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 11 18:45:58.913275 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:58.913254 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 11 18:45:58.913387 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:58.913265 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 11 18:45:58.914199 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:58.914178 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 11 18:45:58.917723 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:58.917706 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 11 18:45:58.924941 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:58.924920 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 11 18:45:59.005952 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.005914 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.005952 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.005947 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.006190 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.005965 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.006190 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.005984 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3ec6e82a-526d-47b6-941a-107d82a48f95-config-out\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.006190 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.006005 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.006190 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.006043 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.006190 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.006077 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.006190 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.006104 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.006190 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.006119 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.006190 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.006139 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3ec6e82a-526d-47b6-941a-107d82a48f95-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.006608 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.006209 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-config\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.006608 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.006288 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.006608 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.006319 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.006608 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.006347 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-web-config\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.006608 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.006398 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.006608 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.006426 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4nmp\" (UniqueName: \"kubernetes.io/projected/3ec6e82a-526d-47b6-941a-107d82a48f95-kube-api-access-q4nmp\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.006608 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.006451 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.006608 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.006479 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.006608 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.006562 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.006608 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.006594 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.107489 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.107442 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.107489 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.107489 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.107741 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.107538 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.107741 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.107564 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.107741 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.107588 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3ec6e82a-526d-47b6-941a-107d82a48f95-config-out\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.107885 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.107823 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.107885 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.107866 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.108249 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.108223 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.108478 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.108432 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.108478 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.108467 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.108676 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.108523 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.108676 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.108562 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3ec6e82a-526d-47b6-941a-107d82a48f95-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.108676 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.108605 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-config\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.108676 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.108639 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.108676 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.108667 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.108936 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.108691 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-web-config\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.108936 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.108728 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.108936 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.108754 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4nmp\" (UniqueName: \"kubernetes.io/projected/3ec6e82a-526d-47b6-941a-107d82a48f95-kube-api-access-q4nmp\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.108936 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.108781 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.108936 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.108808 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.108936 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.108840 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.110737 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.109439 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.110737 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.109564 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.110737 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.108412 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.110737 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.110240 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.110737 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.110354 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3ec6e82a-526d-47b6-941a-107d82a48f95-config-out\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.110737 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.110731 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.111120 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.111089 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.111120 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.111106 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.111238 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.111152 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.111387 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.111331 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.112269 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.111563 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.112269 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.111977 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.112269 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.112052 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.113214 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.113192 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-config\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.113571 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.113553 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-web-config\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.113858 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.113838 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.113963 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.113942 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3ec6e82a-526d-47b6-941a-107d82a48f95-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.114008 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.113982 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.119050 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.119033 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4nmp\" (UniqueName: \"kubernetes.io/projected/3ec6e82a-526d-47b6-941a-107d82a48f95-kube-api-access-q4nmp\") pod \"prometheus-k8s-0\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.216363 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.216269 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:45:59.496695 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.496612 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 11 18:45:59.499688 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:45:59.499663 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ec6e82a_526d_47b6_941a_107d82a48f95.slice/crio-e32450ebda5d4e10bc3003e894ff35578105fdd61c29316d2aa1199f64f6beec WatchSource:0}: Error finding container e32450ebda5d4e10bc3003e894ff35578105fdd61c29316d2aa1199f64f6beec: Status 404 returned error can't find the container with id e32450ebda5d4e10bc3003e894ff35578105fdd61c29316d2aa1199f64f6beec Sep 11 18:45:59.587585 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:45:59.587547 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ec6e82a-526d-47b6-941a-107d82a48f95","Type":"ContainerStarted","Data":"e32450ebda5d4e10bc3003e894ff35578105fdd61c29316d2aa1199f64f6beec"} Sep 11 18:46:01.069095 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:01.069061 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:46:01.169730 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:01.169620 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:46:01.597821 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:01.597779 2563 generic.go:356] "Generic (PLEG): container finished" podID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerID="ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2" exitCode=0 Sep 11 18:46:01.598019 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:01.597855 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ec6e82a-526d-47b6-941a-107d82a48f95","Type":"ContainerDied","Data":"ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2"} Sep 11 18:46:02.059917 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:02.059879 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:46:02.150005 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:02.149945 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:46:02.248875 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:02.248839 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:46:04.615449 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:04.615253 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ec6e82a-526d-47b6-941a-107d82a48f95","Type":"ContainerStarted","Data":"fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c"} Sep 11 18:46:04.615449 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:04.615364 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ec6e82a-526d-47b6-941a-107d82a48f95","Type":"ContainerStarted","Data":"4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c"} Sep 11 18:46:06.626491 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:06.626402 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ec6e82a-526d-47b6-941a-107d82a48f95","Type":"ContainerStarted","Data":"30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b"} Sep 11 18:46:06.626866 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:06.626533 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ec6e82a-526d-47b6-941a-107d82a48f95","Type":"ContainerStarted","Data":"e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a"} Sep 11 18:46:06.626866 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:06.626578 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ec6e82a-526d-47b6-941a-107d82a48f95","Type":"ContainerStarted","Data":"5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a"} Sep 11 18:46:06.626866 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:06.626613 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ec6e82a-526d-47b6-941a-107d82a48f95","Type":"ContainerStarted","Data":"dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787"} Sep 11 18:46:06.656362 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:06.656307 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.995551246 podStartE2EDuration="8.656287438s" totalImagesPullingTime="5.640827163s" totalInitContainerRuntime="19.909029ms" isStatefulPod=false podCreationTimestamp="2026-09-11 18:45:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:46:06.653879304 +0000 UTC m=+88.116485324" watchObservedRunningTime="2026-09-11 18:46:06.656287438 +0000 UTC m=+88.118893458" Sep 11 18:46:08.625905 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:08.625872 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:46:08.725084 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:08.725049 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:46:08.825802 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:08.825771 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:46:09.216838 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:09.216800 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:46:09.481533 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:09.481424 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:46:09.538876 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:09.538843 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:46:11.130671 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:11.130639 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:46:11.226940 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:11.226913 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:46:16.721848 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:16.721810 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:46:16.822930 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:16.822894 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:46:20.565296 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:20.565257 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:46:20.623656 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:20.623624 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:46:22.538376 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:22.538341 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:46:22.560025 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:22.559999 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rxmht" Sep 11 18:46:30.691524 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:30.691469 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:46:30.752059 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:30.752026 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:46:30.822490 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:30.822459 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:46:39.088694 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:39.088656 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:46:46.369261 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:46.369226 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:46:46.470490 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:46.470458 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:46:46.527825 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:46.527800 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:46:49.483060 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:49.483020 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:46:59.216994 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:59.216948 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:46:59.235726 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:59.235692 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:46:59.827451 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:46:59.827421 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:11.873651 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:11.873614 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:47:12.071616 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:12.071586 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:47:17.283869 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.283832 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 11 18:47:17.284341 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.284208 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerName="prometheus" containerID="cri-o://4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c" gracePeriod=600 Sep 11 18:47:17.284341 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.284218 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerName="kube-rbac-proxy" containerID="cri-o://e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a" gracePeriod=600 Sep 11 18:47:17.284341 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.284263 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerName="config-reloader" containerID="cri-o://fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c" gracePeriod=600 Sep 11 18:47:17.284341 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.284275 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerName="thanos-sidecar" containerID="cri-o://dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787" gracePeriod=600 Sep 11 18:47:17.284585 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.284340 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerName="kube-rbac-proxy-web" containerID="cri-o://5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a" gracePeriod=600 Sep 11 18:47:17.284585 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.284272 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerName="kube-rbac-proxy-thanos" containerID="cri-o://30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b" gracePeriod=600 Sep 11 18:47:17.617429 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.617404 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:17.796092 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.796054 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-k8s-rulefiles-0\") pod \"3ec6e82a-526d-47b6-941a-107d82a48f95\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " Sep 11 18:47:17.796092 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.796096 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-k8s-rulefiles-2\") pod \"3ec6e82a-526d-47b6-941a-107d82a48f95\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " Sep 11 18:47:17.796345 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.796124 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-prometheus-k8s-tls\") pod \"3ec6e82a-526d-47b6-941a-107d82a48f95\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " Sep 11 18:47:17.796345 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.796149 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-web-config\") pod \"3ec6e82a-526d-47b6-941a-107d82a48f95\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " Sep 11 18:47:17.796345 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.796168 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-configmap-serving-certs-ca-bundle\") pod \"3ec6e82a-526d-47b6-941a-107d82a48f95\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " Sep 11 18:47:17.796345 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.796187 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-configmap-metrics-client-ca\") pod \"3ec6e82a-526d-47b6-941a-107d82a48f95\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " Sep 11 18:47:17.796345 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.796202 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-kube-rbac-proxy\") pod \"3ec6e82a-526d-47b6-941a-107d82a48f95\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " Sep 11 18:47:17.796621 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.796357 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-thanos-prometheus-http-client-file\") pod \"3ec6e82a-526d-47b6-941a-107d82a48f95\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " Sep 11 18:47:17.796621 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.796390 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"3ec6e82a-526d-47b6-941a-107d82a48f95\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " Sep 11 18:47:17.796621 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.796434 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-grpc-tls\") pod \"3ec6e82a-526d-47b6-941a-107d82a48f95\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " Sep 11 18:47:17.796621 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.796464 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-configmap-kubelet-serving-ca-bundle\") pod \"3ec6e82a-526d-47b6-941a-107d82a48f95\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " Sep 11 18:47:17.796621 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.796493 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"3ec6e82a-526d-47b6-941a-107d82a48f95\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " Sep 11 18:47:17.796621 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.796549 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-k8s-db\") pod \"3ec6e82a-526d-47b6-941a-107d82a48f95\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " Sep 11 18:47:17.796621 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.796575 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4nmp\" (UniqueName: \"kubernetes.io/projected/3ec6e82a-526d-47b6-941a-107d82a48f95-kube-api-access-q4nmp\") pod \"3ec6e82a-526d-47b6-941a-107d82a48f95\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " Sep 11 18:47:17.796621 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.796602 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-config\") pod \"3ec6e82a-526d-47b6-941a-107d82a48f95\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " Sep 11 18:47:17.797010 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.796634 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3ec6e82a-526d-47b6-941a-107d82a48f95-tls-assets\") pod \"3ec6e82a-526d-47b6-941a-107d82a48f95\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " Sep 11 18:47:17.797010 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.796673 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-trusted-ca-bundle\") pod \"3ec6e82a-526d-47b6-941a-107d82a48f95\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " Sep 11 18:47:17.797010 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.796715 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3ec6e82a-526d-47b6-941a-107d82a48f95-config-out\") pod \"3ec6e82a-526d-47b6-941a-107d82a48f95\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " Sep 11 18:47:17.797010 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.796718 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "3ec6e82a-526d-47b6-941a-107d82a48f95" (UID: "3ec6e82a-526d-47b6-941a-107d82a48f95"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 18:47:17.797010 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.796740 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "3ec6e82a-526d-47b6-941a-107d82a48f95" (UID: "3ec6e82a-526d-47b6-941a-107d82a48f95"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 18:47:17.797010 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.796751 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-k8s-rulefiles-1\") pod \"3ec6e82a-526d-47b6-941a-107d82a48f95\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " Sep 11 18:47:17.797010 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.796809 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-metrics-client-certs\") pod \"3ec6e82a-526d-47b6-941a-107d82a48f95\" (UID: \"3ec6e82a-526d-47b6-941a-107d82a48f95\") " Sep 11 18:47:17.797337 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.797051 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "3ec6e82a-526d-47b6-941a-107d82a48f95" (UID: "3ec6e82a-526d-47b6-941a-107d82a48f95"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 18:47:17.797337 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.797063 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "3ec6e82a-526d-47b6-941a-107d82a48f95" (UID: "3ec6e82a-526d-47b6-941a-107d82a48f95"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 18:47:17.797337 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.797197 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:47:17.797337 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.797215 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:47:17.797549 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.797349 2563 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:47:17.797549 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.797367 2563 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-configmap-metrics-client-ca\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:47:17.798140 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.798106 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "3ec6e82a-526d-47b6-941a-107d82a48f95" (UID: "3ec6e82a-526d-47b6-941a-107d82a48f95"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 18:47:17.798255 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.798231 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "3ec6e82a-526d-47b6-941a-107d82a48f95" (UID: "3ec6e82a-526d-47b6-941a-107d82a48f95"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 18:47:17.798956 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.798894 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "3ec6e82a-526d-47b6-941a-107d82a48f95" (UID: "3ec6e82a-526d-47b6-941a-107d82a48f95"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 18:47:17.798956 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.798940 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "3ec6e82a-526d-47b6-941a-107d82a48f95" (UID: "3ec6e82a-526d-47b6-941a-107d82a48f95"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 18:47:17.799466 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.799439 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "3ec6e82a-526d-47b6-941a-107d82a48f95" (UID: "3ec6e82a-526d-47b6-941a-107d82a48f95"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 18:47:17.800036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.800005 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "3ec6e82a-526d-47b6-941a-107d82a48f95" (UID: "3ec6e82a-526d-47b6-941a-107d82a48f95"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 18:47:17.800118 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.800096 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "3ec6e82a-526d-47b6-941a-107d82a48f95" (UID: "3ec6e82a-526d-47b6-941a-107d82a48f95"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 18:47:17.800118 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.800104 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "3ec6e82a-526d-47b6-941a-107d82a48f95" (UID: "3ec6e82a-526d-47b6-941a-107d82a48f95"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 18:47:17.800593 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.800559 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "3ec6e82a-526d-47b6-941a-107d82a48f95" (UID: "3ec6e82a-526d-47b6-941a-107d82a48f95"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 18:47:17.801028 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.801001 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "3ec6e82a-526d-47b6-941a-107d82a48f95" (UID: "3ec6e82a-526d-47b6-941a-107d82a48f95"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 18:47:17.801580 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.801533 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ec6e82a-526d-47b6-941a-107d82a48f95-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "3ec6e82a-526d-47b6-941a-107d82a48f95" (UID: "3ec6e82a-526d-47b6-941a-107d82a48f95"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 18:47:17.801770 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.801746 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "3ec6e82a-526d-47b6-941a-107d82a48f95" (UID: "3ec6e82a-526d-47b6-941a-107d82a48f95"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 18:47:17.802043 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.802017 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ec6e82a-526d-47b6-941a-107d82a48f95-kube-api-access-q4nmp" (OuterVolumeSpecName: "kube-api-access-q4nmp") pod "3ec6e82a-526d-47b6-941a-107d82a48f95" (UID: "3ec6e82a-526d-47b6-941a-107d82a48f95"). InnerVolumeSpecName "kube-api-access-q4nmp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 18:47:17.802125 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.802098 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ec6e82a-526d-47b6-941a-107d82a48f95-config-out" (OuterVolumeSpecName: "config-out") pod "3ec6e82a-526d-47b6-941a-107d82a48f95" (UID: "3ec6e82a-526d-47b6-941a-107d82a48f95"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 18:47:17.802433 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.802407 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-config" (OuterVolumeSpecName: "config") pod "3ec6e82a-526d-47b6-941a-107d82a48f95" (UID: "3ec6e82a-526d-47b6-941a-107d82a48f95"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 18:47:17.809048 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.809026 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-web-config" (OuterVolumeSpecName: "web-config") pod "3ec6e82a-526d-47b6-941a-107d82a48f95" (UID: "3ec6e82a-526d-47b6-941a-107d82a48f95"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 18:47:17.874839 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.874804 2563 generic.go:356] "Generic (PLEG): container finished" podID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerID="30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b" exitCode=0 Sep 11 18:47:17.874839 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.874829 2563 generic.go:356] "Generic (PLEG): container finished" podID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerID="e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a" exitCode=0 Sep 11 18:47:17.874839 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.874835 2563 generic.go:356] "Generic (PLEG): container finished" podID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerID="5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a" exitCode=0 Sep 11 18:47:17.874839 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.874841 2563 generic.go:356] "Generic (PLEG): container finished" podID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerID="dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787" exitCode=0 Sep 11 18:47:17.874839 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.874847 2563 generic.go:356] "Generic (PLEG): container finished" podID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerID="fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c" exitCode=0 Sep 11 18:47:17.875122 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.874855 2563 generic.go:356] "Generic (PLEG): container finished" podID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerID="4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c" exitCode=0 Sep 11 18:47:17.875122 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.874873 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:17.875122 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.874871 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ec6e82a-526d-47b6-941a-107d82a48f95","Type":"ContainerDied","Data":"30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b"} Sep 11 18:47:17.875122 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.875009 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ec6e82a-526d-47b6-941a-107d82a48f95","Type":"ContainerDied","Data":"e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a"} Sep 11 18:47:17.875122 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.875039 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ec6e82a-526d-47b6-941a-107d82a48f95","Type":"ContainerDied","Data":"5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a"} Sep 11 18:47:17.875122 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.875069 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ec6e82a-526d-47b6-941a-107d82a48f95","Type":"ContainerDied","Data":"dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787"} Sep 11 18:47:17.875122 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.875076 2563 scope.go:117] "RemoveContainer" containerID="30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b" Sep 11 18:47:17.875122 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.875107 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ec6e82a-526d-47b6-941a-107d82a48f95","Type":"ContainerDied","Data":"fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c"} Sep 11 18:47:17.875410 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.875145 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ec6e82a-526d-47b6-941a-107d82a48f95","Type":"ContainerDied","Data":"4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c"} Sep 11 18:47:17.875410 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.875187 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ec6e82a-526d-47b6-941a-107d82a48f95","Type":"ContainerDied","Data":"e32450ebda5d4e10bc3003e894ff35578105fdd61c29316d2aa1199f64f6beec"} Sep 11 18:47:17.883532 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.883490 2563 scope.go:117] "RemoveContainer" containerID="e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a" Sep 11 18:47:17.893868 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.893846 2563 scope.go:117] "RemoveContainer" containerID="5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a" Sep 11 18:47:17.898385 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.898366 2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3ec6e82a-526d-47b6-941a-107d82a48f95-tls-assets\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:47:17.898451 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.898387 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-trusted-ca-bundle\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:47:17.898451 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.898397 2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3ec6e82a-526d-47b6-941a-107d82a48f95-config-out\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:47:17.898451 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.898408 2563 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-metrics-client-certs\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:47:17.898451 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.898417 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:47:17.898451 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.898425 2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-prometheus-k8s-tls\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:47:17.898451 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.898433 2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-web-config\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:47:17.898451 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.898441 2563 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-kube-rbac-proxy\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:47:17.898451 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.898450 2563 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-thanos-prometheus-http-client-file\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:47:17.898720 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.898459 2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:47:17.898720 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.898468 2563 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-grpc-tls\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:47:17.898720 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.898477 2563 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ec6e82a-526d-47b6-941a-107d82a48f95-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:47:17.898720 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.898485 2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:47:17.898720 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.898493 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3ec6e82a-526d-47b6-941a-107d82a48f95-prometheus-k8s-db\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:47:17.898720 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.898519 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4nmp\" (UniqueName: \"kubernetes.io/projected/3ec6e82a-526d-47b6-941a-107d82a48f95-kube-api-access-q4nmp\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:47:17.898720 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.898529 2563 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/3ec6e82a-526d-47b6-941a-107d82a48f95-config\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:47:17.900918 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.900890 2563 scope.go:117] "RemoveContainer" containerID="dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787" Sep 11 18:47:17.905372 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.905350 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 11 18:47:17.907612 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.907594 2563 scope.go:117] "RemoveContainer" containerID="fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c" Sep 11 18:47:17.911368 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.911345 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 11 18:47:17.914495 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.914480 2563 scope.go:117] "RemoveContainer" containerID="4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c" Sep 11 18:47:17.924368 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.924346 2563 scope.go:117] "RemoveContainer" containerID="ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2" Sep 11 18:47:17.931012 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.930993 2563 scope.go:117] "RemoveContainer" containerID="30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b" Sep 11 18:47:17.931333 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:47:17.931311 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b\": container with ID starting with 30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b not found: ID does not exist" containerID="30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b" Sep 11 18:47:17.931376 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.931344 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b"} err="failed to get container status \"30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b\": rpc error: code = NotFound desc = could not find container \"30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b\": container with ID starting with 30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b not found: ID does not exist" Sep 11 18:47:17.931412 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.931378 2563 scope.go:117] "RemoveContainer" containerID="e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a" Sep 11 18:47:17.931656 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:47:17.931640 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a\": container with ID starting with e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a not found: ID does not exist" containerID="e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a" Sep 11 18:47:17.931756 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.931660 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a"} err="failed to get container status \"e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a\": rpc error: code = NotFound desc = could not find container \"e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a\": container with ID starting with e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a not found: ID does not exist" Sep 11 18:47:17.931756 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.931674 2563 scope.go:117] "RemoveContainer" containerID="5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a" Sep 11 18:47:17.931905 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:47:17.931888 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a\": container with ID starting with 5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a not found: ID does not exist" containerID="5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a" Sep 11 18:47:17.931949 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.931907 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a"} err="failed to get container status \"5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a\": rpc error: code = NotFound desc = could not find container \"5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a\": container with ID starting with 5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a not found: ID does not exist" Sep 11 18:47:17.931949 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.931918 2563 scope.go:117] "RemoveContainer" containerID="dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787" Sep 11 18:47:17.932114 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:47:17.932095 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787\": container with ID starting with dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787 not found: ID does not exist" containerID="dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787" Sep 11 18:47:17.932148 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.932120 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787"} err="failed to get container status \"dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787\": rpc error: code = NotFound desc = could not find container \"dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787\": container with ID starting with dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787 not found: ID does not exist" Sep 11 18:47:17.932148 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.932137 2563 scope.go:117] "RemoveContainer" containerID="fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c" Sep 11 18:47:17.932352 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:47:17.932337 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c\": container with ID starting with fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c not found: ID does not exist" containerID="fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c" Sep 11 18:47:17.932393 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.932356 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c"} err="failed to get container status \"fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c\": rpc error: code = NotFound desc = could not find container \"fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c\": container with ID starting with fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c not found: ID does not exist" Sep 11 18:47:17.932393 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.932367 2563 scope.go:117] "RemoveContainer" containerID="4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c" Sep 11 18:47:17.932570 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:47:17.932554 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c\": container with ID starting with 4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c not found: ID does not exist" containerID="4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c" Sep 11 18:47:17.932618 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.932573 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c"} err="failed to get container status \"4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c\": rpc error: code = NotFound desc = could not find container \"4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c\": container with ID starting with 4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c not found: ID does not exist" Sep 11 18:47:17.932618 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.932584 2563 scope.go:117] "RemoveContainer" containerID="ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2" Sep 11 18:47:17.932953 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:47:17.932938 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2\": container with ID starting with ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2 not found: ID does not exist" containerID="ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2" Sep 11 18:47:17.933000 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.932956 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2"} err="failed to get container status \"ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2\": rpc error: code = NotFound desc = could not find container \"ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2\": container with ID starting with ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2 not found: ID does not exist" Sep 11 18:47:17.933000 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.932967 2563 scope.go:117] "RemoveContainer" containerID="30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b" Sep 11 18:47:17.933157 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.933141 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b"} err="failed to get container status \"30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b\": rpc error: code = NotFound desc = could not find container \"30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b\": container with ID starting with 30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b not found: ID does not exist" Sep 11 18:47:17.933195 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.933157 2563 scope.go:117] "RemoveContainer" containerID="e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a" Sep 11 18:47:17.933356 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.933341 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a"} err="failed to get container status \"e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a\": rpc error: code = NotFound desc = could not find container \"e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a\": container with ID starting with e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a not found: ID does not exist" Sep 11 18:47:17.933401 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.933356 2563 scope.go:117] "RemoveContainer" containerID="5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a" Sep 11 18:47:17.933548 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.933533 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a"} err="failed to get container status \"5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a\": rpc error: code = NotFound desc = could not find container \"5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a\": container with ID starting with 5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a not found: ID does not exist" Sep 11 18:47:17.933598 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.933548 2563 scope.go:117] "RemoveContainer" containerID="dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787" Sep 11 18:47:17.933769 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.933752 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787"} err="failed to get container status \"dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787\": rpc error: code = NotFound desc = could not find container \"dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787\": container with ID starting with dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787 not found: ID does not exist" Sep 11 18:47:17.933808 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.933769 2563 scope.go:117] "RemoveContainer" containerID="fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c" Sep 11 18:47:17.933958 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.933943 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c"} err="failed to get container status \"fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c\": rpc error: code = NotFound desc = could not find container \"fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c\": container with ID starting with fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c not found: ID does not exist" Sep 11 18:47:17.934004 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.933959 2563 scope.go:117] "RemoveContainer" containerID="4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c" Sep 11 18:47:17.934135 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.934115 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c"} err="failed to get container status \"4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c\": rpc error: code = NotFound desc = could not find container \"4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c\": container with ID starting with 4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c not found: ID does not exist" Sep 11 18:47:17.934175 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.934134 2563 scope.go:117] "RemoveContainer" containerID="ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2" Sep 11 18:47:17.934308 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.934294 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2"} err="failed to get container status \"ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2\": rpc error: code = NotFound desc = could not find container \"ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2\": container with ID starting with ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2 not found: ID does not exist" Sep 11 18:47:17.934347 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.934308 2563 scope.go:117] "RemoveContainer" containerID="30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b" Sep 11 18:47:17.934534 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.934516 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b"} err="failed to get container status \"30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b\": rpc error: code = NotFound desc = could not find container \"30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b\": container with ID starting with 30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b not found: ID does not exist" Sep 11 18:47:17.934579 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.934536 2563 scope.go:117] "RemoveContainer" containerID="e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a" Sep 11 18:47:17.934756 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.934740 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a"} err="failed to get container status \"e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a\": rpc error: code = NotFound desc = could not find container \"e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a\": container with ID starting with e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a not found: ID does not exist" Sep 11 18:47:17.934801 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.934756 2563 scope.go:117] "RemoveContainer" containerID="5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a" Sep 11 18:47:17.934950 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.934931 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a"} err="failed to get container status \"5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a\": rpc error: code = NotFound desc = could not find container \"5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a\": container with ID starting with 5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a not found: ID does not exist" Sep 11 18:47:17.934999 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.934951 2563 scope.go:117] "RemoveContainer" containerID="dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787" Sep 11 18:47:17.935161 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.935144 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787"} err="failed to get container status \"dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787\": rpc error: code = NotFound desc = could not find container \"dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787\": container with ID starting with dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787 not found: ID does not exist" Sep 11 18:47:17.935204 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.935161 2563 scope.go:117] "RemoveContainer" containerID="fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c" Sep 11 18:47:17.935345 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.935329 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c"} err="failed to get container status \"fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c\": rpc error: code = NotFound desc = could not find container \"fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c\": container with ID starting with fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c not found: ID does not exist" Sep 11 18:47:17.935384 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.935346 2563 scope.go:117] "RemoveContainer" containerID="4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c" Sep 11 18:47:17.935550 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.935528 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c"} err="failed to get container status \"4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c\": rpc error: code = NotFound desc = could not find container \"4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c\": container with ID starting with 4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c not found: ID does not exist" Sep 11 18:47:17.935594 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.935551 2563 scope.go:117] "RemoveContainer" containerID="ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2" Sep 11 18:47:17.935779 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.935761 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2"} err="failed to get container status \"ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2\": rpc error: code = NotFound desc = could not find container \"ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2\": container with ID starting with ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2 not found: ID does not exist" Sep 11 18:47:17.935779 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.935777 2563 scope.go:117] "RemoveContainer" containerID="30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b" Sep 11 18:47:17.935968 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.935952 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b"} err="failed to get container status \"30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b\": rpc error: code = NotFound desc = could not find container \"30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b\": container with ID starting with 30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b not found: ID does not exist" Sep 11 18:47:17.936009 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.935968 2563 scope.go:117] "RemoveContainer" containerID="e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a" Sep 11 18:47:17.936172 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.936155 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a"} err="failed to get container status \"e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a\": rpc error: code = NotFound desc = could not find container \"e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a\": container with ID starting with e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a not found: ID does not exist" Sep 11 18:47:17.936172 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.936171 2563 scope.go:117] "RemoveContainer" containerID="5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a" Sep 11 18:47:17.936369 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.936354 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a"} err="failed to get container status \"5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a\": rpc error: code = NotFound desc = could not find container \"5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a\": container with ID starting with 5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a not found: ID does not exist" Sep 11 18:47:17.936411 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.936370 2563 scope.go:117] "RemoveContainer" containerID="dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787" Sep 11 18:47:17.936628 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.936606 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787"} err="failed to get container status \"dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787\": rpc error: code = NotFound desc = could not find container \"dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787\": container with ID starting with dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787 not found: ID does not exist" Sep 11 18:47:17.936707 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.936629 2563 scope.go:117] "RemoveContainer" containerID="fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c" Sep 11 18:47:17.936926 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.936903 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c"} err="failed to get container status \"fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c\": rpc error: code = NotFound desc = could not find container \"fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c\": container with ID starting with fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c not found: ID does not exist" Sep 11 18:47:17.937004 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.936927 2563 scope.go:117] "RemoveContainer" containerID="4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c" Sep 11 18:47:17.937243 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.937221 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c"} err="failed to get container status \"4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c\": rpc error: code = NotFound desc = could not find container \"4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c\": container with ID starting with 4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c not found: ID does not exist" Sep 11 18:47:17.937297 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.937243 2563 scope.go:117] "RemoveContainer" containerID="ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2" Sep 11 18:47:17.937453 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.937436 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2"} err="failed to get container status \"ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2\": rpc error: code = NotFound desc = could not find container \"ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2\": container with ID starting with ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2 not found: ID does not exist" Sep 11 18:47:17.937523 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.937453 2563 scope.go:117] "RemoveContainer" containerID="30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b" Sep 11 18:47:17.937706 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.937687 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b"} err="failed to get container status \"30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b\": rpc error: code = NotFound desc = could not find container \"30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b\": container with ID starting with 30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b not found: ID does not exist" Sep 11 18:47:17.937706 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.937704 2563 scope.go:117] "RemoveContainer" containerID="e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a" Sep 11 18:47:17.937932 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.937916 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a"} err="failed to get container status \"e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a\": rpc error: code = NotFound desc = could not find container \"e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a\": container with ID starting with e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a not found: ID does not exist" Sep 11 18:47:17.937979 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.937932 2563 scope.go:117] "RemoveContainer" containerID="5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a" Sep 11 18:47:17.938121 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.938106 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a"} err="failed to get container status \"5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a\": rpc error: code = NotFound desc = could not find container \"5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a\": container with ID starting with 5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a not found: ID does not exist" Sep 11 18:47:17.938159 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.938120 2563 scope.go:117] "RemoveContainer" containerID="dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787" Sep 11 18:47:17.938316 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.938301 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787"} err="failed to get container status \"dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787\": rpc error: code = NotFound desc = could not find container \"dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787\": container with ID starting with dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787 not found: ID does not exist" Sep 11 18:47:17.938360 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.938316 2563 scope.go:117] "RemoveContainer" containerID="fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c" Sep 11 18:47:17.938519 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.938489 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c"} err="failed to get container status \"fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c\": rpc error: code = NotFound desc = could not find container \"fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c\": container with ID starting with fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c not found: ID does not exist" Sep 11 18:47:17.938519 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.938517 2563 scope.go:117] "RemoveContainer" containerID="4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c" Sep 11 18:47:17.938737 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.938720 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c"} err="failed to get container status \"4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c\": rpc error: code = NotFound desc = could not find container \"4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c\": container with ID starting with 4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c not found: ID does not exist" Sep 11 18:47:17.938786 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.938737 2563 scope.go:117] "RemoveContainer" containerID="ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2" Sep 11 18:47:17.938926 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.938908 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2"} err="failed to get container status \"ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2\": rpc error: code = NotFound desc = could not find container \"ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2\": container with ID starting with ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2 not found: ID does not exist" Sep 11 18:47:17.938965 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.938928 2563 scope.go:117] "RemoveContainer" containerID="30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b" Sep 11 18:47:17.939128 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.939110 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b"} err="failed to get container status \"30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b\": rpc error: code = NotFound desc = could not find container \"30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b\": container with ID starting with 30554e8794ab3f2beae351d22db76fc7f725157dc691c4196b1b2b734f5c493b not found: ID does not exist" Sep 11 18:47:17.939169 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.939129 2563 scope.go:117] "RemoveContainer" containerID="e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a" Sep 11 18:47:17.939314 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.939299 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a"} err="failed to get container status \"e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a\": rpc error: code = NotFound desc = could not find container \"e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a\": container with ID starting with e48c63482b2db3c8292d742099c821482f20380cd85f4016720627484422f15a not found: ID does not exist" Sep 11 18:47:17.939355 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.939314 2563 scope.go:117] "RemoveContainer" containerID="5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a" Sep 11 18:47:17.939558 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.939543 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a"} err="failed to get container status \"5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a\": rpc error: code = NotFound desc = could not find container \"5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a\": container with ID starting with 5328b92d94ee4f7b5b8b1937e88e9384ff678e709dd9546ee3aefd224a81280a not found: ID does not exist" Sep 11 18:47:17.939611 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.939559 2563 scope.go:117] "RemoveContainer" containerID="dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787" Sep 11 18:47:17.939731 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.939717 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787"} err="failed to get container status \"dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787\": rpc error: code = NotFound desc = could not find container \"dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787\": container with ID starting with dfd59315b64bd9c7837e0fdb485fe6070a442486b72446da360b9cb2b2ed6787 not found: ID does not exist" Sep 11 18:47:17.939770 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.939732 2563 scope.go:117] "RemoveContainer" containerID="fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c" Sep 11 18:47:17.939913 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.939898 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c"} err="failed to get container status \"fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c\": rpc error: code = NotFound desc = could not find container \"fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c\": container with ID starting with fd6ec9756df2f897a72b11b3579335c82e69cb608cf11ecc7ef660f1d7b0276c not found: ID does not exist" Sep 11 18:47:17.939959 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.939913 2563 scope.go:117] "RemoveContainer" containerID="4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c" Sep 11 18:47:17.940092 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.940077 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c"} err="failed to get container status \"4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c\": rpc error: code = NotFound desc = could not find container \"4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c\": container with ID starting with 4e4d0b9b4b15f4add33b1a8f59d1963aa4756906c893ce8e0fc06c7aceb9488c not found: ID does not exist" Sep 11 18:47:17.940137 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.940092 2563 scope.go:117] "RemoveContainer" containerID="ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2" Sep 11 18:47:17.940270 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.940255 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2"} err="failed to get container status \"ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2\": rpc error: code = NotFound desc = could not find container \"ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2\": container with ID starting with ef7a9df23374fa9af106f3b3da8958841b5dd420e9ded9ed0ef20d63fbf511c2 not found: ID does not exist" Sep 11 18:47:17.949204 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.949185 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 11 18:47:17.949433 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.949421 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerName="init-config-reloader" Sep 11 18:47:17.949488 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.949434 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerName="init-config-reloader" Sep 11 18:47:17.949488 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.949446 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerName="prometheus" Sep 11 18:47:17.949488 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.949452 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerName="prometheus" Sep 11 18:47:17.949488 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.949459 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerName="thanos-sidecar" Sep 11 18:47:17.949488 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.949464 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerName="thanos-sidecar" Sep 11 18:47:17.949488 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.949469 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerName="kube-rbac-proxy-thanos" Sep 11 18:47:17.949488 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.949474 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerName="kube-rbac-proxy-thanos" Sep 11 18:47:17.949488 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.949483 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerName="kube-rbac-proxy-web" Sep 11 18:47:17.949488 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.949488 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerName="kube-rbac-proxy-web" Sep 11 18:47:17.949768 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.949496 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerName="config-reloader" Sep 11 18:47:17.949768 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.949522 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerName="config-reloader" Sep 11 18:47:17.949768 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.949528 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerName="kube-rbac-proxy" Sep 11 18:47:17.949768 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.949533 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerName="kube-rbac-proxy" Sep 11 18:47:17.949768 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.949573 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerName="thanos-sidecar" Sep 11 18:47:17.949768 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.949580 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerName="kube-rbac-proxy" Sep 11 18:47:17.949768 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.949586 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerName="config-reloader" Sep 11 18:47:17.949768 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.949593 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerName="kube-rbac-proxy-thanos" Sep 11 18:47:17.949768 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.949598 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerName="prometheus" Sep 11 18:47:17.949768 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.949604 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ec6e82a-526d-47b6-941a-107d82a48f95" containerName="kube-rbac-proxy-web" Sep 11 18:47:17.954686 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.954669 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:17.959312 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.959290 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 11 18:47:17.959403 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.959350 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 11 18:47:17.959544 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.959528 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 11 18:47:17.959647 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.959625 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 11 18:47:17.959647 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.959636 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 11 18:47:17.959647 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.959642 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 11 18:47:17.959882 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.959629 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-f84vmbqctpnh0\"" Sep 11 18:47:17.959882 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.959746 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 11 18:47:17.960170 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.960156 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-ddrh7\"" Sep 11 18:47:17.960229 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.960196 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 11 18:47:17.960447 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.960432 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 11 18:47:17.960549 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.960533 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 11 18:47:17.960707 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.960668 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 11 18:47:17.960803 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.960744 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 11 18:47:17.961013 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.960969 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 11 18:47:17.963292 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.963271 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 11 18:47:17.964859 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.964844 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 11 18:47:17.989894 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:17.989856 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 11 18:47:18.099587 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.099479 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/de371545-5c04-40e9-8047-146de0f87134-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.099587 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.099548 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/de371545-5c04-40e9-8047-146de0f87134-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.099764 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.099609 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/de371545-5c04-40e9-8047-146de0f87134-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.099764 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.099646 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/de371545-5c04-40e9-8047-146de0f87134-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.099764 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.099671 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de371545-5c04-40e9-8047-146de0f87134-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.099764 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.099709 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/de371545-5c04-40e9-8047-146de0f87134-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.099764 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.099735 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de371545-5c04-40e9-8047-146de0f87134-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.099909 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.099764 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de371545-5c04-40e9-8047-146de0f87134-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.099909 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.099784 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/de371545-5c04-40e9-8047-146de0f87134-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.099909 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.099824 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/de371545-5c04-40e9-8047-146de0f87134-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.099909 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.099849 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/de371545-5c04-40e9-8047-146de0f87134-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.099909 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.099872 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/de371545-5c04-40e9-8047-146de0f87134-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.099909 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.099899 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/de371545-5c04-40e9-8047-146de0f87134-config\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.100090 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.099973 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2rgm\" (UniqueName: \"kubernetes.io/projected/de371545-5c04-40e9-8047-146de0f87134-kube-api-access-g2rgm\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.100090 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.099992 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/de371545-5c04-40e9-8047-146de0f87134-config-out\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.100090 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.100005 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/de371545-5c04-40e9-8047-146de0f87134-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.100090 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.100024 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de371545-5c04-40e9-8047-146de0f87134-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.100090 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.100080 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/de371545-5c04-40e9-8047-146de0f87134-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.100230 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.100101 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/de371545-5c04-40e9-8047-146de0f87134-web-config\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.100230 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.100124 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/de371545-5c04-40e9-8047-146de0f87134-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.200753 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.200714 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/de371545-5c04-40e9-8047-146de0f87134-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.200822 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.200760 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/de371545-5c04-40e9-8047-146de0f87134-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.200822 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.200786 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/de371545-5c04-40e9-8047-146de0f87134-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.200911 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.200890 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de371545-5c04-40e9-8047-146de0f87134-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.200946 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.200927 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/de371545-5c04-40e9-8047-146de0f87134-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.200995 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.200945 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de371545-5c04-40e9-8047-146de0f87134-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.200995 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.200975 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de371545-5c04-40e9-8047-146de0f87134-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.200995 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.201002 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/de371545-5c04-40e9-8047-146de0f87134-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.200995 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.201034 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/de371545-5c04-40e9-8047-146de0f87134-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.200995 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.201058 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/de371545-5c04-40e9-8047-146de0f87134-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.201339 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.201081 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/de371545-5c04-40e9-8047-146de0f87134-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.201339 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.201102 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/de371545-5c04-40e9-8047-146de0f87134-config\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.201339 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.201139 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2rgm\" (UniqueName: \"kubernetes.io/projected/de371545-5c04-40e9-8047-146de0f87134-kube-api-access-g2rgm\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.201339 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.201173 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/de371545-5c04-40e9-8047-146de0f87134-config-out\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.201339 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.201200 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/de371545-5c04-40e9-8047-146de0f87134-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.201339 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.201227 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de371545-5c04-40e9-8047-146de0f87134-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.201339 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.201270 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/de371545-5c04-40e9-8047-146de0f87134-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.201339 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.201299 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/de371545-5c04-40e9-8047-146de0f87134-web-config\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.201339 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.201333 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/de371545-5c04-40e9-8047-146de0f87134-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.201867 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.201361 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/de371545-5c04-40e9-8047-146de0f87134-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.201867 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.201425 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/de371545-5c04-40e9-8047-146de0f87134-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.201867 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.201773 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de371545-5c04-40e9-8047-146de0f87134-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.201867 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.201787 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de371545-5c04-40e9-8047-146de0f87134-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.202064 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.201928 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/de371545-5c04-40e9-8047-146de0f87134-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.202064 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.201994 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de371545-5c04-40e9-8047-146de0f87134-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.202518 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.202211 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/de371545-5c04-40e9-8047-146de0f87134-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.204063 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.204031 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/de371545-5c04-40e9-8047-146de0f87134-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.204216 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.204196 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/de371545-5c04-40e9-8047-146de0f87134-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.204483 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.204458 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/de371545-5c04-40e9-8047-146de0f87134-web-config\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.204685 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.204662 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/de371545-5c04-40e9-8047-146de0f87134-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.204685 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.204677 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/de371545-5c04-40e9-8047-146de0f87134-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.204814 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.204682 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/de371545-5c04-40e9-8047-146de0f87134-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.204814 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.204680 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de371545-5c04-40e9-8047-146de0f87134-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.205020 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.205001 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/de371545-5c04-40e9-8047-146de0f87134-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.205269 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.205247 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/de371545-5c04-40e9-8047-146de0f87134-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.206320 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.206298 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/de371545-5c04-40e9-8047-146de0f87134-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.206400 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.206354 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/de371545-5c04-40e9-8047-146de0f87134-config-out\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.206651 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.206636 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/de371545-5c04-40e9-8047-146de0f87134-config\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.207234 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.207216 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/de371545-5c04-40e9-8047-146de0f87134-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.210785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.210768 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2rgm\" (UniqueName: \"kubernetes.io/projected/de371545-5c04-40e9-8047-146de0f87134-kube-api-access-g2rgm\") pod \"prometheus-k8s-0\" (UID: \"de371545-5c04-40e9-8047-146de0f87134\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.263753 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.263719 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:18.549034 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.549008 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 11 18:47:18.550926 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:47:18.550894 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde371545_5c04_40e9_8047_146de0f87134.slice/crio-bcf67d718348d413143dc2f2d517a35510eaacc428f8b535dd4bf5de66cfad55 WatchSource:0}: Error finding container bcf67d718348d413143dc2f2d517a35510eaacc428f8b535dd4bf5de66cfad55: Status 404 returned error can't find the container with id bcf67d718348d413143dc2f2d517a35510eaacc428f8b535dd4bf5de66cfad55 Sep 11 18:47:18.880487 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.880395 2563 generic.go:356] "Generic (PLEG): container finished" podID="de371545-5c04-40e9-8047-146de0f87134" containerID="59930d15f86a95911cdf6f5d89851fa362abc3e17bd91948803df32399e5e7c6" exitCode=0 Sep 11 18:47:18.880487 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.880436 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de371545-5c04-40e9-8047-146de0f87134","Type":"ContainerDied","Data":"59930d15f86a95911cdf6f5d89851fa362abc3e17bd91948803df32399e5e7c6"} Sep 11 18:47:18.880487 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:18.880483 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de371545-5c04-40e9-8047-146de0f87134","Type":"ContainerStarted","Data":"bcf67d718348d413143dc2f2d517a35510eaacc428f8b535dd4bf5de66cfad55"} Sep 11 18:47:19.060109 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:19.060073 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ec6e82a-526d-47b6-941a-107d82a48f95" path="/var/lib/kubelet/pods/3ec6e82a-526d-47b6-941a-107d82a48f95/volumes" Sep 11 18:47:19.888023 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:19.887986 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de371545-5c04-40e9-8047-146de0f87134","Type":"ContainerStarted","Data":"a378292afa52772bae4f0b0689b8ea9a2aa31ee6491a8b01342b18f51e15b79c"} Sep 11 18:47:19.888421 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:19.888048 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de371545-5c04-40e9-8047-146de0f87134","Type":"ContainerStarted","Data":"793ef76c3b80c482be0ee12c1dfd615b9820a62951b80bbbc7db8b4e53a752a2"} Sep 11 18:47:19.888421 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:19.888073 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de371545-5c04-40e9-8047-146de0f87134","Type":"ContainerStarted","Data":"4be76a2bfc6e61eff4241cecae841e2303c5d6ea24b9b0890c99bd3d500fb5f6"} Sep 11 18:47:19.888421 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:19.888097 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de371545-5c04-40e9-8047-146de0f87134","Type":"ContainerStarted","Data":"2d364af23d77f5759b626ba26f2eb91730f8250c6496ed2c321f66dad4fa729a"} Sep 11 18:47:19.888421 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:19.888120 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de371545-5c04-40e9-8047-146de0f87134","Type":"ContainerStarted","Data":"3a89a0da0cd42ba965dc3c674505c2baa8a0cd9e32a76e1329b11b870ab14222"} Sep 11 18:47:19.888421 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:19.888143 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de371545-5c04-40e9-8047-146de0f87134","Type":"ContainerStarted","Data":"0447e8c7ecbad7158ef70ae6679fa03dd84ace1fb481db1e30ca3f976ab5b2d4"} Sep 11 18:47:19.925962 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:19.925909 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.905114766 podStartE2EDuration="2.925882205s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.767439ms" isStatefulPod=false podCreationTimestamp="2026-09-11 18:47:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:47:19.924322884 +0000 UTC m=+161.386928903" watchObservedRunningTime="2026-09-11 18:47:19.925882205 +0000 UTC m=+161.388488240" Sep 11 18:47:20.561829 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:20.561795 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:47:20.632084 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:20.632046 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:47:23.264852 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:23.264800 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:47:31.018528 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:31.018468 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:47:31.119729 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:31.119659 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:47:31.219660 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:47:31.219626 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:48:18.264329 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:48:18.264282 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:48:18.279402 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:48:18.279373 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:48:19.111251 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:48:19.111223 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 11 18:48:27.147322 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:48:27.147284 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:48:27.247635 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:48:27.247601 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:48:31.081946 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:48:31.081908 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:48:31.164718 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:48:31.164662 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:48:39.035639 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:48:39.035601 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:48:39.101866 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:48:39.101832 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:48:42.289397 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:48:42.289355 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:48:42.396302 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:48:42.396267 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:48:42.502918 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:48:42.502887 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:48:54.524142 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:48:54.524108 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:48:54.627266 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:48:54.627236 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:48:58.494120 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:48:58.494083 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:49:19.276262 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:49:19.276134 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:49:19.362939 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:49:19.362911 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:49:38.910148 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:49:38.910119 2563 kubelet.go:1648] "Image garbage collection succeeded" Sep 11 18:49:41.663816 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:49:41.663782 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:49:43.048982 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:49:43.048946 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:49:45.491842 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:49:45.491765 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:49:45.566095 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:49:45.566046 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:49:46.945206 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:49:46.945168 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:49:47.088650 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:49:47.088614 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:49:47.561308 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:49:47.561275 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:49:47.682357 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:49:47.682322 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:49:49.916233 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:49:49.916204 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:49:50.117441 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:49:50.117416 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:49:52.212080 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:49:52.212045 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:49:52.281563 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:49:52.281528 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:49:56.532055 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:49:56.532011 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:49:56.657039 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:49:56.657008 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:49:57.531064 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:49:57.531032 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:10.969323 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:10.969295 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:11.054207 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:11.054179 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:16.255814 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:16.255778 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:16.355785 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:16.355755 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:18.795969 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:18.795936 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:21.734060 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:21.734022 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:21.844605 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:21.844578 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:23.066189 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:23.066152 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:23.241919 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:23.241883 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:33.252830 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:33.252754 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:33.356783 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:33.356748 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:33.904639 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:33.904609 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:34.148186 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.148145 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:34.267076 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.267043 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:34.360979 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.360946 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld"] Sep 11 18:50:34.364171 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.364153 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld" Sep 11 18:50:34.370304 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.370225 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 11 18:50:34.370484 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.370468 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ckl8b\"" Sep 11 18:50:34.370595 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.370486 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 11 18:50:34.370681 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.370669 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 11 18:50:34.372553 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.372537 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 11 18:50:34.399121 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.399083 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld"] Sep 11 18:50:34.527887 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.527786 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4f288658-5f78-4b23-a916-6121f1793fb3-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld\" (UID: \"4f288658-5f78-4b23-a916-6121f1793fb3\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld" Sep 11 18:50:34.527887 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.527831 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f288658-5f78-4b23-a916-6121f1793fb3-tmp\") pod \"opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld\" (UID: \"4f288658-5f78-4b23-a916-6121f1793fb3\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld" Sep 11 18:50:34.527887 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.527857 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdbcb\" (UniqueName: \"kubernetes.io/projected/4f288658-5f78-4b23-a916-6121f1793fb3-kube-api-access-xdbcb\") pod \"opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld\" (UID: \"4f288658-5f78-4b23-a916-6121f1793fb3\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld" Sep 11 18:50:34.528119 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.527976 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4f288658-5f78-4b23-a916-6121f1793fb3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld\" (UID: \"4f288658-5f78-4b23-a916-6121f1793fb3\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld" Sep 11 18:50:34.528119 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.528020 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4f288658-5f78-4b23-a916-6121f1793fb3-manifests\") pod \"opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld\" (UID: \"4f288658-5f78-4b23-a916-6121f1793fb3\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld" Sep 11 18:50:34.593080 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.593047 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:34.629275 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.629232 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4f288658-5f78-4b23-a916-6121f1793fb3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld\" (UID: \"4f288658-5f78-4b23-a916-6121f1793fb3\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld" Sep 11 18:50:34.629275 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.629280 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4f288658-5f78-4b23-a916-6121f1793fb3-manifests\") pod \"opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld\" (UID: \"4f288658-5f78-4b23-a916-6121f1793fb3\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld" Sep 11 18:50:34.629542 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.629414 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4f288658-5f78-4b23-a916-6121f1793fb3-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld\" (UID: \"4f288658-5f78-4b23-a916-6121f1793fb3\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld" Sep 11 18:50:34.629542 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.629456 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f288658-5f78-4b23-a916-6121f1793fb3-tmp\") pod \"opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld\" (UID: \"4f288658-5f78-4b23-a916-6121f1793fb3\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld" Sep 11 18:50:34.629542 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.629490 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdbcb\" (UniqueName: \"kubernetes.io/projected/4f288658-5f78-4b23-a916-6121f1793fb3-kube-api-access-xdbcb\") pod \"opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld\" (UID: \"4f288658-5f78-4b23-a916-6121f1793fb3\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld" Sep 11 18:50:34.629704 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.629681 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4f288658-5f78-4b23-a916-6121f1793fb3-manifests\") pod \"opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld\" (UID: \"4f288658-5f78-4b23-a916-6121f1793fb3\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld" Sep 11 18:50:34.631676 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.631641 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f288658-5f78-4b23-a916-6121f1793fb3-tmp\") pod \"opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld\" (UID: \"4f288658-5f78-4b23-a916-6121f1793fb3\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld" Sep 11 18:50:34.631903 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.631886 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4f288658-5f78-4b23-a916-6121f1793fb3-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld\" (UID: \"4f288658-5f78-4b23-a916-6121f1793fb3\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld" Sep 11 18:50:34.631949 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.631917 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4f288658-5f78-4b23-a916-6121f1793fb3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld\" (UID: \"4f288658-5f78-4b23-a916-6121f1793fb3\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld" Sep 11 18:50:34.640020 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.639993 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdbcb\" (UniqueName: \"kubernetes.io/projected/4f288658-5f78-4b23-a916-6121f1793fb3-kube-api-access-xdbcb\") pod \"opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld\" (UID: \"4f288658-5f78-4b23-a916-6121f1793fb3\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld" Sep 11 18:50:34.674051 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.674017 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld" Sep 11 18:50:34.691431 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.691408 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:34.792577 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.792474 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:34.891199 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.891167 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:34.953481 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.953451 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld"] Sep 11 18:50:34.956587 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:50:34.956554 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f288658_5f78_4b23_a916_6121f1793fb3.slice/crio-79b9731f28c1c9a7cf089aaeeb36e315594d66fd0b1ff481769d200e41342378 WatchSource:0}: Error finding container 79b9731f28c1c9a7cf089aaeeb36e315594d66fd0b1ff481769d200e41342378: Status 404 returned error can't find the container with id 79b9731f28c1c9a7cf089aaeeb36e315594d66fd0b1ff481769d200e41342378 Sep 11 18:50:34.958271 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:34.958253 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 11 18:50:35.006996 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:35.006962 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:35.093802 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:35.093724 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:35.558826 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:35.558785 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld" event={"ID":"4f288658-5f78-4b23-a916-6121f1793fb3","Type":"ContainerStarted","Data":"79b9731f28c1c9a7cf089aaeeb36e315594d66fd0b1ff481769d200e41342378"} Sep 11 18:50:38.572780 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:38.572738 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld" event={"ID":"4f288658-5f78-4b23-a916-6121f1793fb3","Type":"ContainerStarted","Data":"7a278a06b4de5b0b91445932a490053d89772b5268c0bae0dcbd9b6132b37f82"} Sep 11 18:50:39.578204 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:39.578167 2563 generic.go:356] "Generic (PLEG): container finished" podID="4f288658-5f78-4b23-a916-6121f1793fb3" containerID="7a278a06b4de5b0b91445932a490053d89772b5268c0bae0dcbd9b6132b37f82" exitCode=0 Sep 11 18:50:39.578599 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:39.578259 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld" event={"ID":"4f288658-5f78-4b23-a916-6121f1793fb3","Type":"ContainerDied","Data":"7a278a06b4de5b0b91445932a490053d89772b5268c0bae0dcbd9b6132b37f82"} Sep 11 18:50:40.583213 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:40.583175 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld" event={"ID":"4f288658-5f78-4b23-a916-6121f1793fb3","Type":"ContainerStarted","Data":"fec960812b40cee398dbdf74fc85674705183366363f4e6df2ff5306e3dbb76c"} Sep 11 18:50:40.583711 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:40.583254 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld" Sep 11 18:50:40.608222 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:40.608163 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld" podStartSLOduration=2.016661496 podStartE2EDuration="6.608145816s" totalImagesPullingTime="3.566387054s" totalInitContainerRuntime="1.025097266s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:50:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:50:40.606785755 +0000 UTC m=+362.069391774" watchObservedRunningTime="2026-09-11 18:50:40.608145816 +0000 UTC m=+362.070751835" Sep 11 18:50:47.348387 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:47.348348 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:47.420556 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:47.420520 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:47.493865 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:47.493825 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:48.075845 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:48.075809 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:48.178571 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:48.178537 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:49.569822 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:49.569787 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:49.671850 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:49.671822 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:51.588475 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:51.588442 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld" Sep 11 18:50:53.504988 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:53.504941 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:53.689124 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:53.689087 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:57.963803 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:57.963772 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:58.063689 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:58.063650 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:50:58.131441 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:50:58.131407 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:51:04.383059 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.379711 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:51:04.406837 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.406798 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-cdff97488-84z2c"] Sep 11 18:51:04.409748 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.409724 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-cdff97488-84z2c" Sep 11 18:51:04.413788 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.413766 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 11 18:51:04.413929 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.413825 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-88n8s\"" Sep 11 18:51:04.413929 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.413766 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 11 18:51:04.415240 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.415218 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 11 18:51:04.415625 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.415318 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 11 18:51:04.424895 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.424870 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-cdff97488-84z2c"] Sep 11 18:51:04.446747 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.446716 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e78e9ec5-2629-4097-a4a7-ee2581075ad9-tls-certs\") pod \"kube-auth-proxy-cdff97488-84z2c\" (UID: \"e78e9ec5-2629-4097-a4a7-ee2581075ad9\") " pod="openshift-ingress/kube-auth-proxy-cdff97488-84z2c" Sep 11 18:51:04.446747 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.446749 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e78e9ec5-2629-4097-a4a7-ee2581075ad9-tmp\") pod \"kube-auth-proxy-cdff97488-84z2c\" (UID: \"e78e9ec5-2629-4097-a4a7-ee2581075ad9\") " pod="openshift-ingress/kube-auth-proxy-cdff97488-84z2c" Sep 11 18:51:04.446952 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.446769 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4f6p\" (UniqueName: \"kubernetes.io/projected/e78e9ec5-2629-4097-a4a7-ee2581075ad9-kube-api-access-t4f6p\") pod \"kube-auth-proxy-cdff97488-84z2c\" (UID: \"e78e9ec5-2629-4097-a4a7-ee2581075ad9\") " pod="openshift-ingress/kube-auth-proxy-cdff97488-84z2c" Sep 11 18:51:04.539668 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.539630 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-chgzp"] Sep 11 18:51:04.542888 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.542871 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-chgzp" Sep 11 18:51:04.545860 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.545833 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Sep 11 18:51:04.545860 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.545859 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-j55l9\"" Sep 11 18:51:04.547128 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.547107 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4f6p\" (UniqueName: \"kubernetes.io/projected/e78e9ec5-2629-4097-a4a7-ee2581075ad9-kube-api-access-t4f6p\") pod \"kube-auth-proxy-cdff97488-84z2c\" (UID: \"e78e9ec5-2629-4097-a4a7-ee2581075ad9\") " pod="openshift-ingress/kube-auth-proxy-cdff97488-84z2c" Sep 11 18:51:04.547235 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.547145 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlnf6\" (UniqueName: \"kubernetes.io/projected/1cca8dee-f089-4c64-9838-e721507b2569-kube-api-access-nlnf6\") pod \"odh-model-controller-7998c67dc4-chgzp\" (UID: \"1cca8dee-f089-4c64-9838-e721507b2569\") " pod="opendatahub/odh-model-controller-7998c67dc4-chgzp" Sep 11 18:51:04.547235 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.547183 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1cca8dee-f089-4c64-9838-e721507b2569-cert\") pod \"odh-model-controller-7998c67dc4-chgzp\" (UID: \"1cca8dee-f089-4c64-9838-e721507b2569\") " pod="opendatahub/odh-model-controller-7998c67dc4-chgzp" Sep 11 18:51:04.547235 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.547223 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e78e9ec5-2629-4097-a4a7-ee2581075ad9-tls-certs\") pod \"kube-auth-proxy-cdff97488-84z2c\" (UID: \"e78e9ec5-2629-4097-a4a7-ee2581075ad9\") " pod="openshift-ingress/kube-auth-proxy-cdff97488-84z2c" Sep 11 18:51:04.547423 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.547238 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e78e9ec5-2629-4097-a4a7-ee2581075ad9-tmp\") pod \"kube-auth-proxy-cdff97488-84z2c\" (UID: \"e78e9ec5-2629-4097-a4a7-ee2581075ad9\") " pod="openshift-ingress/kube-auth-proxy-cdff97488-84z2c" Sep 11 18:51:04.549750 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.549728 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e78e9ec5-2629-4097-a4a7-ee2581075ad9-tmp\") pod \"kube-auth-proxy-cdff97488-84z2c\" (UID: \"e78e9ec5-2629-4097-a4a7-ee2581075ad9\") " pod="openshift-ingress/kube-auth-proxy-cdff97488-84z2c" Sep 11 18:51:04.549972 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.549955 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e78e9ec5-2629-4097-a4a7-ee2581075ad9-tls-certs\") pod \"kube-auth-proxy-cdff97488-84z2c\" (UID: \"e78e9ec5-2629-4097-a4a7-ee2581075ad9\") " pod="openshift-ingress/kube-auth-proxy-cdff97488-84z2c" Sep 11 18:51:04.552410 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.552388 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-chgzp"] Sep 11 18:51:04.558636 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.558606 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4f6p\" (UniqueName: \"kubernetes.io/projected/e78e9ec5-2629-4097-a4a7-ee2581075ad9-kube-api-access-t4f6p\") pod \"kube-auth-proxy-cdff97488-84z2c\" (UID: \"e78e9ec5-2629-4097-a4a7-ee2581075ad9\") " pod="openshift-ingress/kube-auth-proxy-cdff97488-84z2c" Sep 11 18:51:04.571805 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.571777 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:51:04.647827 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.647728 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1cca8dee-f089-4c64-9838-e721507b2569-cert\") pod \"odh-model-controller-7998c67dc4-chgzp\" (UID: \"1cca8dee-f089-4c64-9838-e721507b2569\") " pod="opendatahub/odh-model-controller-7998c67dc4-chgzp" Sep 11 18:51:04.647827 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.647812 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlnf6\" (UniqueName: \"kubernetes.io/projected/1cca8dee-f089-4c64-9838-e721507b2569-kube-api-access-nlnf6\") pod \"odh-model-controller-7998c67dc4-chgzp\" (UID: \"1cca8dee-f089-4c64-9838-e721507b2569\") " pod="opendatahub/odh-model-controller-7998c67dc4-chgzp" Sep 11 18:51:04.648057 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:51:04.647889 2563 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Sep 11 18:51:04.648057 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:51:04.647982 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cca8dee-f089-4c64-9838-e721507b2569-cert podName:1cca8dee-f089-4c64-9838-e721507b2569 nodeName:}" failed. No retries permitted until 2026-09-11 18:51:05.147960433 +0000 UTC m=+386.610566442 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1cca8dee-f089-4c64-9838-e721507b2569-cert") pod "odh-model-controller-7998c67dc4-chgzp" (UID: "1cca8dee-f089-4c64-9838-e721507b2569") : secret "odh-model-controller-webhook-cert" not found Sep 11 18:51:04.661130 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.661102 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlnf6\" (UniqueName: \"kubernetes.io/projected/1cca8dee-f089-4c64-9838-e721507b2569-kube-api-access-nlnf6\") pod \"odh-model-controller-7998c67dc4-chgzp\" (UID: \"1cca8dee-f089-4c64-9838-e721507b2569\") " pod="opendatahub/odh-model-controller-7998c67dc4-chgzp" Sep 11 18:51:04.705071 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.705043 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:51:04.719558 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.719527 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-cdff97488-84z2c" Sep 11 18:51:04.805057 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.805028 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:51:04.993191 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:04.993117 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-cdff97488-84z2c"] Sep 11 18:51:04.996341 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:51:04.996315 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode78e9ec5_2629_4097_a4a7_ee2581075ad9.slice/crio-f7e33fd0fbd42011d7eb4569b7872afdac7965d8a816d611f97898e169324fc8 WatchSource:0}: Error finding container f7e33fd0fbd42011d7eb4569b7872afdac7965d8a816d611f97898e169324fc8: Status 404 returned error can't find the container with id f7e33fd0fbd42011d7eb4569b7872afdac7965d8a816d611f97898e169324fc8 Sep 11 18:51:05.151479 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.151444 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1cca8dee-f089-4c64-9838-e721507b2569-cert\") pod \"odh-model-controller-7998c67dc4-chgzp\" (UID: \"1cca8dee-f089-4c64-9838-e721507b2569\") " pod="opendatahub/odh-model-controller-7998c67dc4-chgzp" Sep 11 18:51:05.153735 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.153717 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1cca8dee-f089-4c64-9838-e721507b2569-cert\") pod \"odh-model-controller-7998c67dc4-chgzp\" (UID: \"1cca8dee-f089-4c64-9838-e721507b2569\") " pod="opendatahub/odh-model-controller-7998c67dc4-chgzp" Sep 11 18:51:05.158035 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.158008 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mpxv2"] Sep 11 18:51:05.158035 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.158008 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:51:05.160909 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.160888 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-chgzp" Sep 11 18:51:05.161956 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.161940 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-mpxv2" Sep 11 18:51:05.165889 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.165870 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-26jvg\"" Sep 11 18:51:05.165889 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.165880 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Sep 11 18:51:05.178978 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.178949 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mpxv2"] Sep 11 18:51:05.252417 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.252331 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgt96\" (UniqueName: \"kubernetes.io/projected/eb78723e-f54e-438c-8328-759456ee91f1-kube-api-access-sgt96\") pod \"model-serving-api-54d89bd5fc-mpxv2\" (UID: \"eb78723e-f54e-438c-8328-759456ee91f1\") " pod="opendatahub/model-serving-api-54d89bd5fc-mpxv2" Sep 11 18:51:05.252417 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.252399 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb78723e-f54e-438c-8328-759456ee91f1-tls-certs\") pod \"model-serving-api-54d89bd5fc-mpxv2\" (UID: \"eb78723e-f54e-438c-8328-759456ee91f1\") " pod="opendatahub/model-serving-api-54d89bd5fc-mpxv2" Sep 11 18:51:05.353688 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.353654 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb78723e-f54e-438c-8328-759456ee91f1-tls-certs\") pod \"model-serving-api-54d89bd5fc-mpxv2\" (UID: \"eb78723e-f54e-438c-8328-759456ee91f1\") " pod="opendatahub/model-serving-api-54d89bd5fc-mpxv2" Sep 11 18:51:05.353863 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.353726 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgt96\" (UniqueName: \"kubernetes.io/projected/eb78723e-f54e-438c-8328-759456ee91f1-kube-api-access-sgt96\") pod \"model-serving-api-54d89bd5fc-mpxv2\" (UID: \"eb78723e-f54e-438c-8328-759456ee91f1\") " pod="opendatahub/model-serving-api-54d89bd5fc-mpxv2" Sep 11 18:51:05.356142 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.356124 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb78723e-f54e-438c-8328-759456ee91f1-tls-certs\") pod \"model-serving-api-54d89bd5fc-mpxv2\" (UID: \"eb78723e-f54e-438c-8328-759456ee91f1\") " pod="opendatahub/model-serving-api-54d89bd5fc-mpxv2" Sep 11 18:51:05.362419 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.362400 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgt96\" (UniqueName: \"kubernetes.io/projected/eb78723e-f54e-438c-8328-759456ee91f1-kube-api-access-sgt96\") pod \"model-serving-api-54d89bd5fc-mpxv2\" (UID: \"eb78723e-f54e-438c-8328-759456ee91f1\") " pod="opendatahub/model-serving-api-54d89bd5fc-mpxv2" Sep 11 18:51:05.398302 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.398269 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:51:05.437034 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.437010 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-chgzp"] Sep 11 18:51:05.438108 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:51:05.438083 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cca8dee_f089_4c64_9838_e721507b2569.slice/crio-1a7dd399d67ffdc96f3eb161357ee14fdb76a2fda3bea44594ad536a4e934a45 WatchSource:0}: Error finding container 1a7dd399d67ffdc96f3eb161357ee14fdb76a2fda3bea44594ad536a4e934a45: Status 404 returned error can't find the container with id 1a7dd399d67ffdc96f3eb161357ee14fdb76a2fda3bea44594ad536a4e934a45 Sep 11 18:51:05.472570 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.472538 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-mpxv2" Sep 11 18:51:05.671814 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.671777 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-chgzp" event={"ID":"1cca8dee-f089-4c64-9838-e721507b2569","Type":"ContainerStarted","Data":"1a7dd399d67ffdc96f3eb161357ee14fdb76a2fda3bea44594ad536a4e934a45"} Sep 11 18:51:05.673307 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.673241 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-cdff97488-84z2c" event={"ID":"e78e9ec5-2629-4097-a4a7-ee2581075ad9","Type":"ContainerStarted","Data":"f7e33fd0fbd42011d7eb4569b7872afdac7965d8a816d611f97898e169324fc8"} Sep 11 18:51:05.747697 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.747666 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mpxv2"] Sep 11 18:51:05.751454 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:51:05.751422 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb78723e_f54e_438c_8328_759456ee91f1.slice/crio-36c5a3dfd10f669e007fb9420a5521a27af62adb7731952dfdaf1a8c96118d8e WatchSource:0}: Error finding container 36c5a3dfd10f669e007fb9420a5521a27af62adb7731952dfdaf1a8c96118d8e: Status 404 returned error can't find the container with id 36c5a3dfd10f669e007fb9420a5521a27af62adb7731952dfdaf1a8c96118d8e Sep 11 18:51:05.941673 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.941595 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj"] Sep 11 18:51:05.945246 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.945225 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" Sep 11 18:51:05.948602 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.948365 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 11 18:51:05.948602 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.948423 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 11 18:51:05.948602 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.948365 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 11 18:51:05.948839 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.948694 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 11 18:51:05.948952 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.948931 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-x57gc\"" Sep 11 18:51:05.951960 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.951937 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:51:05.958644 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.958537 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/86268662-6b82-4fa3-ac42-edafa7bf3ea9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-6r4gj\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" Sep 11 18:51:05.958644 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.958594 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/86268662-6b82-4fa3-ac42-edafa7bf3ea9-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-6r4gj\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" Sep 11 18:51:05.958812 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.958717 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86268662-6b82-4fa3-ac42-edafa7bf3ea9-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-6r4gj\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" Sep 11 18:51:05.958812 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.958753 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/86268662-6b82-4fa3-ac42-edafa7bf3ea9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-6r4gj\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" Sep 11 18:51:05.958812 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.958788 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/86268662-6b82-4fa3-ac42-edafa7bf3ea9-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-6r4gj\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" Sep 11 18:51:05.958812 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.958806 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/86268662-6b82-4fa3-ac42-edafa7bf3ea9-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-6r4gj\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" Sep 11 18:51:05.958987 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.958821 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mpc8\" (UniqueName: \"kubernetes.io/projected/86268662-6b82-4fa3-ac42-edafa7bf3ea9-kube-api-access-6mpc8\") pod \"istiod-openshift-gateway-7b7995d5cd-6r4gj\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" Sep 11 18:51:05.961152 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:05.961130 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj"] Sep 11 18:51:06.059983 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:06.059948 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86268662-6b82-4fa3-ac42-edafa7bf3ea9-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-6r4gj\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" Sep 11 18:51:06.060168 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:06.060011 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/86268662-6b82-4fa3-ac42-edafa7bf3ea9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-6r4gj\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" Sep 11 18:51:06.060168 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:06.060059 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/86268662-6b82-4fa3-ac42-edafa7bf3ea9-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-6r4gj\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" Sep 11 18:51:06.060168 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:06.060114 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/86268662-6b82-4fa3-ac42-edafa7bf3ea9-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-6r4gj\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" Sep 11 18:51:06.060168 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:06.060137 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mpc8\" (UniqueName: \"kubernetes.io/projected/86268662-6b82-4fa3-ac42-edafa7bf3ea9-kube-api-access-6mpc8\") pod \"istiod-openshift-gateway-7b7995d5cd-6r4gj\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" Sep 11 18:51:06.060485 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:06.060177 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/86268662-6b82-4fa3-ac42-edafa7bf3ea9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-6r4gj\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" Sep 11 18:51:06.060609 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:06.060563 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/86268662-6b82-4fa3-ac42-edafa7bf3ea9-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-6r4gj\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" Sep 11 18:51:06.062080 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:06.061383 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/86268662-6b82-4fa3-ac42-edafa7bf3ea9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-6r4gj\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" Sep 11 18:51:06.063416 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:06.063329 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/86268662-6b82-4fa3-ac42-edafa7bf3ea9-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-6r4gj\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" Sep 11 18:51:06.064406 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:06.063946 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/86268662-6b82-4fa3-ac42-edafa7bf3ea9-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-6r4gj\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" Sep 11 18:51:06.064406 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:06.064109 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/86268662-6b82-4fa3-ac42-edafa7bf3ea9-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-6r4gj\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" Sep 11 18:51:06.065180 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:06.065158 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/86268662-6b82-4fa3-ac42-edafa7bf3ea9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-6r4gj\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" Sep 11 18:51:06.082396 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:06.082367 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86268662-6b82-4fa3-ac42-edafa7bf3ea9-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-6r4gj\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" Sep 11 18:51:06.085899 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:06.085846 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:51:06.086783 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:06.086761 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mpc8\" (UniqueName: \"kubernetes.io/projected/86268662-6b82-4fa3-ac42-edafa7bf3ea9-kube-api-access-6mpc8\") pod \"istiod-openshift-gateway-7b7995d5cd-6r4gj\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" Sep 11 18:51:06.257866 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:06.257524 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" Sep 11 18:51:06.569132 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:06.569094 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj"] Sep 11 18:51:06.575740 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:51:06.575690 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86268662_6b82_4fa3_ac42_edafa7bf3ea9.slice/crio-0a2e24e3f43819f01f5a2c5151f0c50750dc980c8cd0df30636ba54d14a625be WatchSource:0}: Error finding container 0a2e24e3f43819f01f5a2c5151f0c50750dc980c8cd0df30636ba54d14a625be: Status 404 returned error can't find the container with id 0a2e24e3f43819f01f5a2c5151f0c50750dc980c8cd0df30636ba54d14a625be Sep 11 18:51:06.683345 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:06.683251 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-mpxv2" event={"ID":"eb78723e-f54e-438c-8328-759456ee91f1","Type":"ContainerStarted","Data":"36c5a3dfd10f669e007fb9420a5521a27af62adb7731952dfdaf1a8c96118d8e"} Sep 11 18:51:06.685982 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:06.685905 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" event={"ID":"86268662-6b82-4fa3-ac42-edafa7bf3ea9","Type":"ContainerStarted","Data":"0a2e24e3f43819f01f5a2c5151f0c50750dc980c8cd0df30636ba54d14a625be"} Sep 11 18:51:07.166608 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:07.166558 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:51:08.231906 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:08.231841 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:51:11.149897 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:11.149857 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 18:51:11.150220 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:11.149925 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 18:51:11.716742 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:11.716699 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-chgzp" event={"ID":"1cca8dee-f089-4c64-9838-e721507b2569","Type":"ContainerStarted","Data":"fb2439f677b32c1c66c2e7ebe6b79eaa199c25ac7497c702ec9915a9ef966a51"} Sep 11 18:51:11.716942 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:11.716769 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-chgzp" Sep 11 18:51:11.718312 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:11.718283 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-mpxv2" event={"ID":"eb78723e-f54e-438c-8328-759456ee91f1","Type":"ContainerStarted","Data":"38e6ce8e0f9aaf3ac49c3d9a07a7e88c4e5d2de49fec29afa18d731465eff58c"} Sep 11 18:51:11.718414 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:11.718379 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-mpxv2" Sep 11 18:51:11.719885 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:11.719864 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" event={"ID":"86268662-6b82-4fa3-ac42-edafa7bf3ea9","Type":"ContainerStarted","Data":"53bfb7b74f8de145b2105ab412cfe412fcd8778c5e2b0faca9247c7de3555c11"} Sep 11 18:51:11.719987 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:11.719938 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" Sep 11 18:51:11.721338 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:11.721320 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-cdff97488-84z2c" event={"ID":"e78e9ec5-2629-4097-a4a7-ee2581075ad9","Type":"ContainerStarted","Data":"66fe561fc33655c4e603146a76e2ecfd506908cc1d6c3bc85e37cc2ee9910c9e"} Sep 11 18:51:11.766177 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:11.766120 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-chgzp" podStartSLOduration=2.106591602 podStartE2EDuration="7.766098803s" totalImagesPullingTime="5.659507201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:51:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:51:11.764311886 +0000 UTC m=+393.226917904" watchObservedRunningTime="2026-09-11 18:51:11.766098803 +0000 UTC m=+393.228704823" Sep 11 18:51:11.845054 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:11.844997 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" podStartSLOduration=2.275787534 podStartE2EDuration="6.844979449s" totalImagesPullingTime="4.569191915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:51:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:51:11.843535467 +0000 UTC m=+393.306141476" watchObservedRunningTime="2026-09-11 18:51:11.844979449 +0000 UTC m=+393.307585458" Sep 11 18:51:11.876278 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:11.876222 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-cdff97488-84z2c" podStartSLOduration=1.785380016 podStartE2EDuration="7.876205628s" totalImagesPullingTime="6.090825612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:51:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:51:11.874859763 +0000 UTC m=+393.337465783" watchObservedRunningTime="2026-09-11 18:51:11.876205628 +0000 UTC m=+393.338811645" Sep 11 18:51:11.900203 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:11.900133 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-mpxv2" podStartSLOduration=1.5654076190000001 podStartE2EDuration="6.900111942s" totalImagesPullingTime="5.334704323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:51:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:51:11.899910651 +0000 UTC m=+393.362516669" watchObservedRunningTime="2026-09-11 18:51:11.900111942 +0000 UTC m=+393.362717944" Sep 11 18:51:12.433931 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.433883 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:51:12.527475 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.527441 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:51:12.712909 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.712827 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk"] Sep 11 18:51:12.716578 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.716552 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:12.721215 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.721187 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-92vz5\"" Sep 11 18:51:12.728083 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.727790 2563 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-6r4gj container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 11 18:51:12.728226 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.728133 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" podUID="86268662-6b82-4fa3-ac42-edafa7bf3ea9" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 18:51:12.729162 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.729137 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk"] Sep 11 18:51:12.752402 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.752373 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:51:12.820882 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.820847 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e094fe2-8167-4db6-aa36-25f565b07177-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk\" (UID: \"6e094fe2-8167-4db6-aa36-25f565b07177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:12.821068 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.820919 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e094fe2-8167-4db6-aa36-25f565b07177-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk\" (UID: \"6e094fe2-8167-4db6-aa36-25f565b07177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:12.821068 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.820937 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e094fe2-8167-4db6-aa36-25f565b07177-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk\" (UID: \"6e094fe2-8167-4db6-aa36-25f565b07177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:12.821068 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.821040 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e094fe2-8167-4db6-aa36-25f565b07177-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk\" (UID: \"6e094fe2-8167-4db6-aa36-25f565b07177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:12.821250 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.821196 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e094fe2-8167-4db6-aa36-25f565b07177-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk\" (UID: \"6e094fe2-8167-4db6-aa36-25f565b07177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:12.821250 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.821240 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfkbl\" (UniqueName: \"kubernetes.io/projected/6e094fe2-8167-4db6-aa36-25f565b07177-kube-api-access-tfkbl\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk\" (UID: \"6e094fe2-8167-4db6-aa36-25f565b07177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:12.821378 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.821339 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e094fe2-8167-4db6-aa36-25f565b07177-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk\" (UID: \"6e094fe2-8167-4db6-aa36-25f565b07177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:12.821487 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.821468 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e094fe2-8167-4db6-aa36-25f565b07177-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk\" (UID: \"6e094fe2-8167-4db6-aa36-25f565b07177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:12.821565 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.821547 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e094fe2-8167-4db6-aa36-25f565b07177-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk\" (UID: \"6e094fe2-8167-4db6-aa36-25f565b07177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:12.855104 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.855077 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:51:12.923000 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.922963 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e094fe2-8167-4db6-aa36-25f565b07177-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk\" (UID: \"6e094fe2-8167-4db6-aa36-25f565b07177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:12.923201 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.923007 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e094fe2-8167-4db6-aa36-25f565b07177-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk\" (UID: \"6e094fe2-8167-4db6-aa36-25f565b07177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:12.923201 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.923030 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e094fe2-8167-4db6-aa36-25f565b07177-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk\" (UID: \"6e094fe2-8167-4db6-aa36-25f565b07177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:12.923201 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.923062 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e094fe2-8167-4db6-aa36-25f565b07177-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk\" (UID: \"6e094fe2-8167-4db6-aa36-25f565b07177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:12.923201 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.923085 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e094fe2-8167-4db6-aa36-25f565b07177-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk\" (UID: \"6e094fe2-8167-4db6-aa36-25f565b07177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:12.923201 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.923123 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e094fe2-8167-4db6-aa36-25f565b07177-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk\" (UID: \"6e094fe2-8167-4db6-aa36-25f565b07177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:12.923201 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.923172 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e094fe2-8167-4db6-aa36-25f565b07177-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk\" (UID: \"6e094fe2-8167-4db6-aa36-25f565b07177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:12.923617 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.923205 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfkbl\" (UniqueName: \"kubernetes.io/projected/6e094fe2-8167-4db6-aa36-25f565b07177-kube-api-access-tfkbl\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk\" (UID: \"6e094fe2-8167-4db6-aa36-25f565b07177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:12.923617 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.923236 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e094fe2-8167-4db6-aa36-25f565b07177-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk\" (UID: \"6e094fe2-8167-4db6-aa36-25f565b07177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:12.923617 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.923408 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e094fe2-8167-4db6-aa36-25f565b07177-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk\" (UID: \"6e094fe2-8167-4db6-aa36-25f565b07177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:12.923617 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.923578 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e094fe2-8167-4db6-aa36-25f565b07177-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk\" (UID: \"6e094fe2-8167-4db6-aa36-25f565b07177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:12.923784 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.923620 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e094fe2-8167-4db6-aa36-25f565b07177-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk\" (UID: \"6e094fe2-8167-4db6-aa36-25f565b07177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:12.923892 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.923866 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e094fe2-8167-4db6-aa36-25f565b07177-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk\" (UID: \"6e094fe2-8167-4db6-aa36-25f565b07177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:12.924026 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.924003 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e094fe2-8167-4db6-aa36-25f565b07177-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk\" (UID: \"6e094fe2-8167-4db6-aa36-25f565b07177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:12.925569 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.925546 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e094fe2-8167-4db6-aa36-25f565b07177-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk\" (UID: \"6e094fe2-8167-4db6-aa36-25f565b07177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:12.925765 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.925745 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e094fe2-8167-4db6-aa36-25f565b07177-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk\" (UID: \"6e094fe2-8167-4db6-aa36-25f565b07177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:12.936985 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.936956 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfkbl\" (UniqueName: \"kubernetes.io/projected/6e094fe2-8167-4db6-aa36-25f565b07177-kube-api-access-tfkbl\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk\" (UID: \"6e094fe2-8167-4db6-aa36-25f565b07177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:12.938787 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:12.938761 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e094fe2-8167-4db6-aa36-25f565b07177-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk\" (UID: \"6e094fe2-8167-4db6-aa36-25f565b07177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:13.028615 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:13.028580 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:13.134370 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:13.134334 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:51:13.309650 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:13.309560 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk"] Sep 11 18:51:13.312210 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:51:13.312181 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e094fe2_8167_4db6_aa36_25f565b07177.slice/crio-ca03746def6e10c30cf3d3b47ba12c51b067abcc3f789ef0d178dde2c7540e7c WatchSource:0}: Error finding container ca03746def6e10c30cf3d3b47ba12c51b067abcc3f789ef0d178dde2c7540e7c: Status 404 returned error can't find the container with id ca03746def6e10c30cf3d3b47ba12c51b067abcc3f789ef0d178dde2c7540e7c Sep 11 18:51:13.730616 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:13.730529 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" event={"ID":"6e094fe2-8167-4db6-aa36-25f565b07177","Type":"ContainerStarted","Data":"ca03746def6e10c30cf3d3b47ba12c51b067abcc3f789ef0d178dde2c7540e7c"} Sep 11 18:51:15.334494 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:15.334337 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:51:15.396855 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:15.396817 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qq4sv"] Sep 11 18:51:15.403755 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:15.403730 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qq4sv" Sep 11 18:51:15.406930 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:15.406905 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Sep 11 18:51:15.408373 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:15.408349 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-fllfd\"" Sep 11 18:51:15.422859 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:15.422833 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qq4sv"] Sep 11 18:51:15.441912 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:15.441883 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-c7vwr"] Sep 11 18:51:15.445610 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:15.445580 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-c7vwr" Sep 11 18:51:15.445790 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:15.445767 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f5392b9a-3f94-4e11-a551-6944660149a6-cert\") pod \"kserve-controller-manager-d6f4b65b8-qq4sv\" (UID: \"f5392b9a-3f94-4e11-a551-6944660149a6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qq4sv" Sep 11 18:51:15.445874 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:15.445854 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49k7d\" (UniqueName: \"kubernetes.io/projected/f5392b9a-3f94-4e11-a551-6944660149a6-kube-api-access-49k7d\") pod \"kserve-controller-manager-d6f4b65b8-qq4sv\" (UID: \"f5392b9a-3f94-4e11-a551-6944660149a6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qq4sv" Sep 11 18:51:15.449699 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:15.449676 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Sep 11 18:51:15.449699 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:15.449695 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-6w9zx\"" Sep 11 18:51:15.464605 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:15.464579 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-c7vwr"] Sep 11 18:51:15.473071 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:15.473043 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:51:15.547194 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:15.547152 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxbp4\" (UniqueName: \"kubernetes.io/projected/c47a4a2a-320c-467a-8704-6110b9c097b5-kube-api-access-fxbp4\") pod \"llmisvc-controller-manager-777c6bd7d5-c7vwr\" (UID: \"c47a4a2a-320c-467a-8704-6110b9c097b5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-c7vwr" Sep 11 18:51:15.547389 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:15.547228 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f5392b9a-3f94-4e11-a551-6944660149a6-cert\") pod \"kserve-controller-manager-d6f4b65b8-qq4sv\" (UID: \"f5392b9a-3f94-4e11-a551-6944660149a6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qq4sv" Sep 11 18:51:15.547389 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:15.547301 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49k7d\" (UniqueName: \"kubernetes.io/projected/f5392b9a-3f94-4e11-a551-6944660149a6-kube-api-access-49k7d\") pod \"kserve-controller-manager-d6f4b65b8-qq4sv\" (UID: \"f5392b9a-3f94-4e11-a551-6944660149a6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qq4sv" Sep 11 18:51:15.547389 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:15.547326 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c47a4a2a-320c-467a-8704-6110b9c097b5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-c7vwr\" (UID: \"c47a4a2a-320c-467a-8704-6110b9c097b5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-c7vwr" Sep 11 18:51:15.547389 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:51:15.547337 2563 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Sep 11 18:51:15.547688 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:51:15.547416 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5392b9a-3f94-4e11-a551-6944660149a6-cert podName:f5392b9a-3f94-4e11-a551-6944660149a6 nodeName:}" failed. No retries permitted until 2026-09-11 18:51:16.047394448 +0000 UTC m=+397.510000450 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f5392b9a-3f94-4e11-a551-6944660149a6-cert") pod "kserve-controller-manager-d6f4b65b8-qq4sv" (UID: "f5392b9a-3f94-4e11-a551-6944660149a6") : secret "kserve-webhook-server-cert" not found Sep 11 18:51:15.560397 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:15.560340 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49k7d\" (UniqueName: \"kubernetes.io/projected/f5392b9a-3f94-4e11-a551-6944660149a6-kube-api-access-49k7d\") pod \"kserve-controller-manager-d6f4b65b8-qq4sv\" (UID: \"f5392b9a-3f94-4e11-a551-6944660149a6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qq4sv" Sep 11 18:51:15.648359 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:15.648259 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c47a4a2a-320c-467a-8704-6110b9c097b5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-c7vwr\" (UID: \"c47a4a2a-320c-467a-8704-6110b9c097b5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-c7vwr" Sep 11 18:51:15.648359 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:15.648338 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxbp4\" (UniqueName: \"kubernetes.io/projected/c47a4a2a-320c-467a-8704-6110b9c097b5-kube-api-access-fxbp4\") pod \"llmisvc-controller-manager-777c6bd7d5-c7vwr\" (UID: \"c47a4a2a-320c-467a-8704-6110b9c097b5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-c7vwr" Sep 11 18:51:15.648643 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:51:15.648440 2563 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Sep 11 18:51:15.648643 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:51:15.648541 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c47a4a2a-320c-467a-8704-6110b9c097b5-cert podName:c47a4a2a-320c-467a-8704-6110b9c097b5 nodeName:}" failed. No retries permitted until 2026-09-11 18:51:16.148519198 +0000 UTC m=+397.611125214 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c47a4a2a-320c-467a-8704-6110b9c097b5-cert") pod "llmisvc-controller-manager-777c6bd7d5-c7vwr" (UID: "c47a4a2a-320c-467a-8704-6110b9c097b5") : secret "llmisvc-webhook-server-cert" not found Sep 11 18:51:15.681063 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:15.681030 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxbp4\" (UniqueName: \"kubernetes.io/projected/c47a4a2a-320c-467a-8704-6110b9c097b5-kube-api-access-fxbp4\") pod \"llmisvc-controller-manager-777c6bd7d5-c7vwr\" (UID: \"c47a4a2a-320c-467a-8704-6110b9c097b5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-c7vwr" Sep 11 18:51:15.722581 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:15.722535 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:51:15.727131 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:15.727103 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" Sep 11 18:51:15.906522 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:15.906429 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:51:16.003350 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:16.003292 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 18:51:16.003480 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:16.003388 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 18:51:16.003480 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:16.003418 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 18:51:16.052106 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:16.052081 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f5392b9a-3f94-4e11-a551-6944660149a6-cert\") pod \"kserve-controller-manager-d6f4b65b8-qq4sv\" (UID: \"f5392b9a-3f94-4e11-a551-6944660149a6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qq4sv" Sep 11 18:51:16.062075 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:16.062017 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f5392b9a-3f94-4e11-a551-6944660149a6-cert\") pod \"kserve-controller-manager-d6f4b65b8-qq4sv\" (UID: \"f5392b9a-3f94-4e11-a551-6944660149a6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qq4sv" Sep 11 18:51:16.153461 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:16.153429 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c47a4a2a-320c-467a-8704-6110b9c097b5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-c7vwr\" (UID: \"c47a4a2a-320c-467a-8704-6110b9c097b5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-c7vwr" Sep 11 18:51:16.156239 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:16.156208 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c47a4a2a-320c-467a-8704-6110b9c097b5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-c7vwr\" (UID: \"c47a4a2a-320c-467a-8704-6110b9c097b5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-c7vwr" Sep 11 18:51:16.195991 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:16.195896 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:51:16.317285 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:16.317250 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qq4sv" Sep 11 18:51:16.357856 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:16.357811 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-c7vwr" Sep 11 18:51:16.601623 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:16.601597 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qq4sv"] Sep 11 18:51:16.602992 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:51:16.602967 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5392b9a_3f94_4e11_a551_6944660149a6.slice/crio-4e4384de12028740162db98fd7518217a6c46f7f5bdf852bbe338b8cc932e768 WatchSource:0}: Error finding container 4e4384de12028740162db98fd7518217a6c46f7f5bdf852bbe338b8cc932e768: Status 404 returned error can't find the container with id 4e4384de12028740162db98fd7518217a6c46f7f5bdf852bbe338b8cc932e768 Sep 11 18:51:16.644071 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:16.644013 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-c7vwr"] Sep 11 18:51:16.650211 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:51:16.650185 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc47a4a2a_320c_467a_8704_6110b9c097b5.slice/crio-692568658c87df0b9f1c4b9c9869fcb520e096b7022fc1df8ec59f88cf4bb692 WatchSource:0}: Error finding container 692568658c87df0b9f1c4b9c9869fcb520e096b7022fc1df8ec59f88cf4bb692: Status 404 returned error can't find the container with id 692568658c87df0b9f1c4b9c9869fcb520e096b7022fc1df8ec59f88cf4bb692 Sep 11 18:51:16.744167 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:16.744122 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-c7vwr" event={"ID":"c47a4a2a-320c-467a-8704-6110b9c097b5","Type":"ContainerStarted","Data":"692568658c87df0b9f1c4b9c9869fcb520e096b7022fc1df8ec59f88cf4bb692"} Sep 11 18:51:16.745825 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:16.745791 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" event={"ID":"6e094fe2-8167-4db6-aa36-25f565b07177","Type":"ContainerStarted","Data":"3a81f649bf6057b884c5db31ec75439bcf51c4a3ee40f47a9a98263dc58bf753"} Sep 11 18:51:16.746979 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:16.746949 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qq4sv" event={"ID":"f5392b9a-3f94-4e11-a551-6944660149a6","Type":"ContainerStarted","Data":"4e4384de12028740162db98fd7518217a6c46f7f5bdf852bbe338b8cc932e768"} Sep 11 18:51:16.776912 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:16.776867 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" podStartSLOduration=2.088123572 podStartE2EDuration="4.776852875s" totalImagesPullingTime="2.688729303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:51:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:51:16.774818659 +0000 UTC m=+398.237424677" watchObservedRunningTime="2026-09-11 18:51:16.776852875 +0000 UTC m=+398.239458893" Sep 11 18:51:16.998574 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:16.998532 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj"] Sep 11 18:51:16.998757 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:16.998735 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" podUID="86268662-6b82-4fa3-ac42-edafa7bf3ea9" containerName="discovery" containerID="cri-o://53bfb7b74f8de145b2105ab412cfe412fcd8778c5e2b0faca9247c7de3555c11" gracePeriod=30 Sep 11 18:51:17.029121 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.029092 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:17.033596 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.033568 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:17.253394 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.253368 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" Sep 11 18:51:17.364706 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.364083 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/86268662-6b82-4fa3-ac42-edafa7bf3ea9-istio-csr-ca-configmap\") pod \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " Sep 11 18:51:17.364706 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.364147 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/86268662-6b82-4fa3-ac42-edafa7bf3ea9-local-certs\") pod \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " Sep 11 18:51:17.364706 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.364177 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86268662-6b82-4fa3-ac42-edafa7bf3ea9-istio-token\") pod \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " Sep 11 18:51:17.364706 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.364209 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/86268662-6b82-4fa3-ac42-edafa7bf3ea9-cacerts\") pod \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " Sep 11 18:51:17.364706 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.364275 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/86268662-6b82-4fa3-ac42-edafa7bf3ea9-istio-csr-dns-cert\") pod \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " Sep 11 18:51:17.364706 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.364304 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/86268662-6b82-4fa3-ac42-edafa7bf3ea9-istio-kubeconfig\") pod \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " Sep 11 18:51:17.364706 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.364333 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mpc8\" (UniqueName: \"kubernetes.io/projected/86268662-6b82-4fa3-ac42-edafa7bf3ea9-kube-api-access-6mpc8\") pod \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\" (UID: \"86268662-6b82-4fa3-ac42-edafa7bf3ea9\") " Sep 11 18:51:17.364706 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.364576 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86268662-6b82-4fa3-ac42-edafa7bf3ea9-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "86268662-6b82-4fa3-ac42-edafa7bf3ea9" (UID: "86268662-6b82-4fa3-ac42-edafa7bf3ea9"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 18:51:17.364706 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.364639 2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/86268662-6b82-4fa3-ac42-edafa7bf3ea9-istio-csr-ca-configmap\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:51:17.367453 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.367420 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86268662-6b82-4fa3-ac42-edafa7bf3ea9-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "86268662-6b82-4fa3-ac42-edafa7bf3ea9" (UID: "86268662-6b82-4fa3-ac42-edafa7bf3ea9"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 18:51:17.367752 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.367701 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86268662-6b82-4fa3-ac42-edafa7bf3ea9-cacerts" (OuterVolumeSpecName: "cacerts") pod "86268662-6b82-4fa3-ac42-edafa7bf3ea9" (UID: "86268662-6b82-4fa3-ac42-edafa7bf3ea9"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 18:51:17.367942 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.367907 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86268662-6b82-4fa3-ac42-edafa7bf3ea9-local-certs" (OuterVolumeSpecName: "local-certs") pod "86268662-6b82-4fa3-ac42-edafa7bf3ea9" (UID: "86268662-6b82-4fa3-ac42-edafa7bf3ea9"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 18:51:17.369384 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.369335 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86268662-6b82-4fa3-ac42-edafa7bf3ea9-istio-token" (OuterVolumeSpecName: "istio-token") pod "86268662-6b82-4fa3-ac42-edafa7bf3ea9" (UID: "86268662-6b82-4fa3-ac42-edafa7bf3ea9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 18:51:17.370787 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.370580 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86268662-6b82-4fa3-ac42-edafa7bf3ea9-kube-api-access-6mpc8" (OuterVolumeSpecName: "kube-api-access-6mpc8") pod "86268662-6b82-4fa3-ac42-edafa7bf3ea9" (UID: "86268662-6b82-4fa3-ac42-edafa7bf3ea9"). InnerVolumeSpecName "kube-api-access-6mpc8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 18:51:17.370787 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.370749 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86268662-6b82-4fa3-ac42-edafa7bf3ea9-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "86268662-6b82-4fa3-ac42-edafa7bf3ea9" (UID: "86268662-6b82-4fa3-ac42-edafa7bf3ea9"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 18:51:17.465751 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.465635 2563 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/86268662-6b82-4fa3-ac42-edafa7bf3ea9-local-certs\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:51:17.465751 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.465672 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86268662-6b82-4fa3-ac42-edafa7bf3ea9-istio-token\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:51:17.465751 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.465685 2563 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/86268662-6b82-4fa3-ac42-edafa7bf3ea9-cacerts\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:51:17.465751 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.465698 2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/86268662-6b82-4fa3-ac42-edafa7bf3ea9-istio-csr-dns-cert\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:51:17.465751 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.465712 2563 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/86268662-6b82-4fa3-ac42-edafa7bf3ea9-istio-kubeconfig\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:51:17.465751 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.465725 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mpc8\" (UniqueName: \"kubernetes.io/projected/86268662-6b82-4fa3-ac42-edafa7bf3ea9-kube-api-access-6mpc8\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:51:17.757305 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.757264 2563 generic.go:356] "Generic (PLEG): container finished" podID="86268662-6b82-4fa3-ac42-edafa7bf3ea9" containerID="53bfb7b74f8de145b2105ab412cfe412fcd8778c5e2b0faca9247c7de3555c11" exitCode=0 Sep 11 18:51:17.757565 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.757487 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" event={"ID":"86268662-6b82-4fa3-ac42-edafa7bf3ea9","Type":"ContainerDied","Data":"53bfb7b74f8de145b2105ab412cfe412fcd8778c5e2b0faca9247c7de3555c11"} Sep 11 18:51:17.757713 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.757587 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" event={"ID":"86268662-6b82-4fa3-ac42-edafa7bf3ea9","Type":"ContainerDied","Data":"0a2e24e3f43819f01f5a2c5151f0c50750dc980c8cd0df30636ba54d14a625be"} Sep 11 18:51:17.757713 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.757596 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj" Sep 11 18:51:17.757713 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.757620 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:17.757713 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.757654 2563 scope.go:117] "RemoveContainer" containerID="53bfb7b74f8de145b2105ab412cfe412fcd8778c5e2b0faca9247c7de3555c11" Sep 11 18:51:17.759244 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.759222 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk" Sep 11 18:51:17.807680 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.807614 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj"] Sep 11 18:51:17.814144 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.814082 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6r4gj"] Sep 11 18:51:17.885853 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.885824 2563 scope.go:117] "RemoveContainer" containerID="53bfb7b74f8de145b2105ab412cfe412fcd8778c5e2b0faca9247c7de3555c11" Sep 11 18:51:17.886279 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:51:17.886251 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53bfb7b74f8de145b2105ab412cfe412fcd8778c5e2b0faca9247c7de3555c11\": container with ID starting with 53bfb7b74f8de145b2105ab412cfe412fcd8778c5e2b0faca9247c7de3555c11 not found: ID does not exist" containerID="53bfb7b74f8de145b2105ab412cfe412fcd8778c5e2b0faca9247c7de3555c11" Sep 11 18:51:17.886359 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:17.886293 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53bfb7b74f8de145b2105ab412cfe412fcd8778c5e2b0faca9247c7de3555c11"} err="failed to get container status \"53bfb7b74f8de145b2105ab412cfe412fcd8778c5e2b0faca9247c7de3555c11\": rpc error: code = NotFound desc = could not find container \"53bfb7b74f8de145b2105ab412cfe412fcd8778c5e2b0faca9247c7de3555c11\": container with ID starting with 53bfb7b74f8de145b2105ab412cfe412fcd8778c5e2b0faca9247c7de3555c11 not found: ID does not exist" Sep 11 18:51:19.062553 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:19.062470 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86268662-6b82-4fa3-ac42-edafa7bf3ea9" path="/var/lib/kubelet/pods/86268662-6b82-4fa3-ac42-edafa7bf3ea9/volumes" Sep 11 18:51:19.769016 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:19.768983 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-c7vwr" event={"ID":"c47a4a2a-320c-467a-8704-6110b9c097b5","Type":"ContainerStarted","Data":"3c9e05272d3b4f8027504e1f54f1d01a484e52f21760d896f0826773b21a85f4"} Sep 11 18:51:19.769216 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:19.769193 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-c7vwr" Sep 11 18:51:19.770645 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:19.770620 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qq4sv" event={"ID":"f5392b9a-3f94-4e11-a551-6944660149a6","Type":"ContainerStarted","Data":"549cb31a770a5bdaede3a3a4d2fde47597d258b1b6f1acbfaf592c0bc87ee51d"} Sep 11 18:51:19.803195 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:19.803139 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-c7vwr" podStartSLOduration=2.030737162 podStartE2EDuration="4.803124645s" totalImagesPullingTime="2.772387483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:51:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:51:19.801721987 +0000 UTC m=+401.264328031" watchObservedRunningTime="2026-09-11 18:51:19.803124645 +0000 UTC m=+401.265730661" Sep 11 18:51:19.832621 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:19.832571 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qq4sv" podStartSLOduration=2.056557847 podStartE2EDuration="4.832560242s" totalImagesPullingTime="2.776002395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:51:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:51:19.831251146 +0000 UTC m=+401.293857163" watchObservedRunningTime="2026-09-11 18:51:19.832560242 +0000 UTC m=+401.295166259" Sep 11 18:51:20.775256 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:20.775223 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qq4sv" Sep 11 18:51:22.728656 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:22.728627 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-chgzp" Sep 11 18:51:22.732015 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:22.731994 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-mpxv2" Sep 11 18:51:50.777145 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:50.777112 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-c7vwr" Sep 11 18:51:51.785212 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:51:51.785181 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qq4sv" Sep 11 18:52:05.574116 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:05.574028 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:05.674756 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:05.674721 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:10.319077 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:10.319042 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:10.419066 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:10.419033 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:12.154912 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:12.154872 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:18.451675 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:18.451639 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:19.273428 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:19.273395 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:20.089679 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:20.089645 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:21.971032 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:21.971004 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:23.150652 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:23.150615 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-46zxz"] Sep 11 18:52:23.151143 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:23.151069 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86268662-6b82-4fa3-ac42-edafa7bf3ea9" containerName="discovery" Sep 11 18:52:23.151143 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:23.151088 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="86268662-6b82-4fa3-ac42-edafa7bf3ea9" containerName="discovery" Sep 11 18:52:23.151278 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:23.151194 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="86268662-6b82-4fa3-ac42-edafa7bf3ea9" containerName="discovery" Sep 11 18:52:23.153081 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:23.153060 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-46zxz" Sep 11 18:52:23.156839 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:23.156813 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 11 18:52:23.156839 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:23.156835 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-x7wrv\"" Sep 11 18:52:23.157002 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:23.156928 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 11 18:52:23.173584 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:23.173554 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-46zxz"] Sep 11 18:52:23.231011 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:23.230974 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/48bdb353-9b76-4bb3-bc91-f965ff55e6eb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-46zxz\" (UID: \"48bdb353-9b76-4bb3-bc91-f965ff55e6eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-46zxz" Sep 11 18:52:23.231179 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:23.231019 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbgn4\" (UniqueName: \"kubernetes.io/projected/48bdb353-9b76-4bb3-bc91-f965ff55e6eb-kube-api-access-bbgn4\") pod \"kuadrant-operator-controller-manager-76f8f686bf-46zxz\" (UID: \"48bdb353-9b76-4bb3-bc91-f965ff55e6eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-46zxz" Sep 11 18:52:23.260444 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:23.260408 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:23.332320 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:23.332282 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/48bdb353-9b76-4bb3-bc91-f965ff55e6eb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-46zxz\" (UID: \"48bdb353-9b76-4bb3-bc91-f965ff55e6eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-46zxz" Sep 11 18:52:23.332320 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:23.332324 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbgn4\" (UniqueName: \"kubernetes.io/projected/48bdb353-9b76-4bb3-bc91-f965ff55e6eb-kube-api-access-bbgn4\") pod \"kuadrant-operator-controller-manager-76f8f686bf-46zxz\" (UID: \"48bdb353-9b76-4bb3-bc91-f965ff55e6eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-46zxz" Sep 11 18:52:23.332753 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:23.332728 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/48bdb353-9b76-4bb3-bc91-f965ff55e6eb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-46zxz\" (UID: \"48bdb353-9b76-4bb3-bc91-f965ff55e6eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-46zxz" Sep 11 18:52:23.344813 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:23.344783 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbgn4\" (UniqueName: \"kubernetes.io/projected/48bdb353-9b76-4bb3-bc91-f965ff55e6eb-kube-api-access-bbgn4\") pod \"kuadrant-operator-controller-manager-76f8f686bf-46zxz\" (UID: \"48bdb353-9b76-4bb3-bc91-f965ff55e6eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-46zxz" Sep 11 18:52:23.462673 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:23.462587 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-46zxz" Sep 11 18:52:23.510698 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:23.510652 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:23.660447 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:23.660408 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:23.744191 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:23.744166 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-46zxz"] Sep 11 18:52:23.746984 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:52:23.746959 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48bdb353_9b76_4bb3_bc91_f965ff55e6eb.slice/crio-53269656b316ac4dc805520bf07140275ec7a083e9f483233bbce76936625985 WatchSource:0}: Error finding container 53269656b316ac4dc805520bf07140275ec7a083e9f483233bbce76936625985: Status 404 returned error can't find the container with id 53269656b316ac4dc805520bf07140275ec7a083e9f483233bbce76936625985 Sep 11 18:52:23.752757 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:23.752733 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:24.037800 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:24.037757 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-46zxz" event={"ID":"48bdb353-9b76-4bb3-bc91-f965ff55e6eb","Type":"ContainerStarted","Data":"53269656b316ac4dc805520bf07140275ec7a083e9f483233bbce76936625985"} Sep 11 18:52:25.617648 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:25.617587 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-mvv7c"] Sep 11 18:52:25.620252 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:25.620225 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-mvv7c" Sep 11 18:52:25.623628 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:25.623601 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-s9fgr\"" Sep 11 18:52:25.634413 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:25.634385 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-mvv7c"] Sep 11 18:52:25.755049 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:25.755006 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spg2v\" (UniqueName: \"kubernetes.io/projected/55149df4-4fa7-4e56-9913-a4ef3838643d-kube-api-access-spg2v\") pod \"authorino-operator-76b478bd78-mvv7c\" (UID: \"55149df4-4fa7-4e56-9913-a4ef3838643d\") " pod="kuadrant-system/authorino-operator-76b478bd78-mvv7c" Sep 11 18:52:25.855819 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:25.855785 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spg2v\" (UniqueName: \"kubernetes.io/projected/55149df4-4fa7-4e56-9913-a4ef3838643d-kube-api-access-spg2v\") pod \"authorino-operator-76b478bd78-mvv7c\" (UID: \"55149df4-4fa7-4e56-9913-a4ef3838643d\") " pod="kuadrant-system/authorino-operator-76b478bd78-mvv7c" Sep 11 18:52:25.866989 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:25.866954 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spg2v\" (UniqueName: \"kubernetes.io/projected/55149df4-4fa7-4e56-9913-a4ef3838643d-kube-api-access-spg2v\") pod \"authorino-operator-76b478bd78-mvv7c\" (UID: \"55149df4-4fa7-4e56-9913-a4ef3838643d\") " pod="kuadrant-system/authorino-operator-76b478bd78-mvv7c" Sep 11 18:52:25.934825 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:25.934716 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-mvv7c" Sep 11 18:52:26.253859 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:26.253781 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:26.702183 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:26.702152 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-mvv7c"] Sep 11 18:52:26.704195 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:52:26.704171 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55149df4_4fa7_4e56_9913_a4ef3838643d.slice/crio-70fa90f3f0660122e4e718f685c9e1178f9ac88f9d052ee48c9bba7539cf2c12 WatchSource:0}: Error finding container 70fa90f3f0660122e4e718f685c9e1178f9ac88f9d052ee48c9bba7539cf2c12: Status 404 returned error can't find the container with id 70fa90f3f0660122e4e718f685c9e1178f9ac88f9d052ee48c9bba7539cf2c12 Sep 11 18:52:27.061152 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:27.061103 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-mvv7c" event={"ID":"55149df4-4fa7-4e56-9913-a4ef3838643d","Type":"ContainerStarted","Data":"70fa90f3f0660122e4e718f685c9e1178f9ac88f9d052ee48c9bba7539cf2c12"} Sep 11 18:52:27.655477 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:27.655437 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-g984d"] Sep 11 18:52:27.657927 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:27.657907 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g984d" Sep 11 18:52:27.660899 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:27.660875 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-wclxn\"" Sep 11 18:52:27.670993 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:27.670967 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-g984d"] Sep 11 18:52:27.771966 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:27.771928 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqqj5\" (UniqueName: \"kubernetes.io/projected/2375fb95-12cb-4d7b-96aa-f6195aa9d9df-kube-api-access-tqqj5\") pod \"limitador-operator-controller-manager-74495c69f8-g984d\" (UID: \"2375fb95-12cb-4d7b-96aa-f6195aa9d9df\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g984d" Sep 11 18:52:27.872795 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:27.872749 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqqj5\" (UniqueName: \"kubernetes.io/projected/2375fb95-12cb-4d7b-96aa-f6195aa9d9df-kube-api-access-tqqj5\") pod \"limitador-operator-controller-manager-74495c69f8-g984d\" (UID: \"2375fb95-12cb-4d7b-96aa-f6195aa9d9df\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g984d" Sep 11 18:52:27.885463 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:27.885425 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqqj5\" (UniqueName: \"kubernetes.io/projected/2375fb95-12cb-4d7b-96aa-f6195aa9d9df-kube-api-access-tqqj5\") pod \"limitador-operator-controller-manager-74495c69f8-g984d\" (UID: \"2375fb95-12cb-4d7b-96aa-f6195aa9d9df\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g984d" Sep 11 18:52:27.969935 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:27.969847 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g984d" Sep 11 18:52:28.259402 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:28.259371 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-g984d"] Sep 11 18:52:28.263111 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:52:28.263067 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2375fb95_12cb_4d7b_96aa_f6195aa9d9df.slice/crio-2dab1a52a1ca3d0d2433f21f5fbb8d3e6bb9c9c6874a481fe71e81a3dd2e14bf WatchSource:0}: Error finding container 2dab1a52a1ca3d0d2433f21f5fbb8d3e6bb9c9c6874a481fe71e81a3dd2e14bf: Status 404 returned error can't find the container with id 2dab1a52a1ca3d0d2433f21f5fbb8d3e6bb9c9c6874a481fe71e81a3dd2e14bf Sep 11 18:52:28.319265 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:28.319233 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:28.529150 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:28.529117 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:29.065180 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:29.065142 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g984d" event={"ID":"2375fb95-12cb-4d7b-96aa-f6195aa9d9df","Type":"ContainerStarted","Data":"2dab1a52a1ca3d0d2433f21f5fbb8d3e6bb9c9c6874a481fe71e81a3dd2e14bf"} Sep 11 18:52:29.067001 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:29.066970 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-mvv7c" event={"ID":"55149df4-4fa7-4e56-9913-a4ef3838643d","Type":"ContainerStarted","Data":"b61170ee72d45572055399760cc55a7ca49d9f80f7c7433ab2c47e1fca04530f"} Sep 11 18:52:29.067151 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:29.067079 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-mvv7c" Sep 11 18:52:29.069058 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:29.069035 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-46zxz" event={"ID":"48bdb353-9b76-4bb3-bc91-f965ff55e6eb","Type":"ContainerStarted","Data":"8d87742e3afa93f887e5e0e39c53c61ce3c84ab2fd258ba702aac5d198dbe930"} Sep 11 18:52:29.069189 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:29.069177 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-46zxz" Sep 11 18:52:29.103320 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:29.103258 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-mvv7c" podStartSLOduration=2.115710711 podStartE2EDuration="4.103238638s" totalImagesPullingTime="1.987527927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:52:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:52:29.101020074 +0000 UTC m=+470.563626094" watchObservedRunningTime="2026-09-11 18:52:29.103238638 +0000 UTC m=+470.565844659" Sep 11 18:52:29.131683 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:29.131626 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-46zxz" podStartSLOduration=1.844316085 podStartE2EDuration="6.131607923s" totalImagesPullingTime="4.287291838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:52:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:52:29.129566584 +0000 UTC m=+470.592172605" watchObservedRunningTime="2026-09-11 18:52:29.131607923 +0000 UTC m=+470.594213941" Sep 11 18:52:29.930469 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:29.930431 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-625tz"] Sep 11 18:52:29.932582 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:29.932563 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-625tz" Sep 11 18:52:29.935457 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:29.935380 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Sep 11 18:52:29.935457 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:29.935388 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-n42v4\"" Sep 11 18:52:29.947671 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:29.947643 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-625tz"] Sep 11 18:52:30.075316 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:30.075283 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g984d" event={"ID":"2375fb95-12cb-4d7b-96aa-f6195aa9d9df","Type":"ContainerStarted","Data":"2207d448f4902ad49c300edcf02baf6f382d1b6a258715f1b0a441393d075397"} Sep 11 18:52:30.075745 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:30.075366 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g984d" Sep 11 18:52:30.093634 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:30.093580 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g984d" podStartSLOduration=1.674045999 podStartE2EDuration="3.093567016s" totalImagesPullingTime="1.419521017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:52:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:52:30.093164501 +0000 UTC m=+471.555770519" watchObservedRunningTime="2026-09-11 18:52:30.093567016 +0000 UTC m=+471.556173036" Sep 11 18:52:30.094478 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:30.094449 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2j9s\" (UniqueName: \"kubernetes.io/projected/254a47f5-19ea-4fd5-84d7-29427b749b9d-kube-api-access-f2j9s\") pod \"dns-operator-controller-manager-7b84cd7c9c-625tz\" (UID: \"254a47f5-19ea-4fd5-84d7-29427b749b9d\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-625tz" Sep 11 18:52:30.195849 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:30.195749 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2j9s\" (UniqueName: \"kubernetes.io/projected/254a47f5-19ea-4fd5-84d7-29427b749b9d-kube-api-access-f2j9s\") pod \"dns-operator-controller-manager-7b84cd7c9c-625tz\" (UID: \"254a47f5-19ea-4fd5-84d7-29427b749b9d\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-625tz" Sep 11 18:52:30.205590 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:30.205557 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2j9s\" (UniqueName: \"kubernetes.io/projected/254a47f5-19ea-4fd5-84d7-29427b749b9d-kube-api-access-f2j9s\") pod \"dns-operator-controller-manager-7b84cd7c9c-625tz\" (UID: \"254a47f5-19ea-4fd5-84d7-29427b749b9d\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-625tz" Sep 11 18:52:30.243118 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:30.243083 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-625tz" Sep 11 18:52:30.526950 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:30.526922 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-625tz"] Sep 11 18:52:30.528782 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:52:30.528757 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod254a47f5_19ea_4fd5_84d7_29427b749b9d.slice/crio-916b58d531bbd4010d32ff5f07389c0b9b47ce551bbcff75cdae023da209c393 WatchSource:0}: Error finding container 916b58d531bbd4010d32ff5f07389c0b9b47ce551bbcff75cdae023da209c393: Status 404 returned error can't find the container with id 916b58d531bbd4010d32ff5f07389c0b9b47ce551bbcff75cdae023da209c393 Sep 11 18:52:31.001548 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:31.001439 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:31.081722 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:31.081685 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-625tz" event={"ID":"254a47f5-19ea-4fd5-84d7-29427b749b9d","Type":"ContainerStarted","Data":"916b58d531bbd4010d32ff5f07389c0b9b47ce551bbcff75cdae023da209c393"} Sep 11 18:52:31.328534 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:31.328483 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:31.625336 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:31.625247 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:32.954476 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:32.954444 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:33.056834 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:33.056801 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:33.093376 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:33.093342 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-625tz" event={"ID":"254a47f5-19ea-4fd5-84d7-29427b749b9d","Type":"ContainerStarted","Data":"d912e7090ca8e772e6abeddc1b9cdc46b84d708936f0d4aa45b6118f2b98186a"} Sep 11 18:52:33.093569 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:33.093427 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-625tz" Sep 11 18:52:33.115853 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:33.115801 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-625tz" podStartSLOduration=2.464296966 podStartE2EDuration="4.115785753s" totalImagesPullingTime="1.651488787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:52:33.114030832 +0000 UTC m=+474.576636851" watchObservedRunningTime="2026-09-11 18:52:33.115785753 +0000 UTC m=+474.578391770" Sep 11 18:52:33.153678 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:33.153647 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:33.275719 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:33.275691 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:33.389804 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:33.389771 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:40.077377 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:40.077341 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-mvv7c" Sep 11 18:52:40.077794 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:40.077429 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-46zxz" Sep 11 18:52:41.083908 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:41.083878 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g984d" Sep 11 18:52:42.461001 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.460970 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-46zxz"] Sep 11 18:52:42.461452 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.461139 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-46zxz" podUID="48bdb353-9b76-4bb3-bc91-f965ff55e6eb" containerName="manager" containerID="cri-o://8d87742e3afa93f887e5e0e39c53c61ce3c84ab2fd258ba702aac5d198dbe930" gracePeriod=2 Sep 11 18:52:42.470325 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.470300 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-46zxz"] Sep 11 18:52:42.499061 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.499028 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-g984d"] Sep 11 18:52:42.499235 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.499215 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g984d" podUID="2375fb95-12cb-4d7b-96aa-f6195aa9d9df" containerName="manager" containerID="cri-o://2207d448f4902ad49c300edcf02baf6f382d1b6a258715f1b0a441393d075397" gracePeriod=2 Sep 11 18:52:42.525994 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.525901 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-g984d"] Sep 11 18:52:42.528274 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.528241 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-n6c6r"] Sep 11 18:52:42.528730 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.528713 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2375fb95-12cb-4d7b-96aa-f6195aa9d9df" containerName="manager" Sep 11 18:52:42.528793 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.528733 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2375fb95-12cb-4d7b-96aa-f6195aa9d9df" containerName="manager" Sep 11 18:52:42.528793 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.528762 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48bdb353-9b76-4bb3-bc91-f965ff55e6eb" containerName="manager" Sep 11 18:52:42.528793 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.528771 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="48bdb353-9b76-4bb3-bc91-f965ff55e6eb" containerName="manager" Sep 11 18:52:42.528897 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.528838 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="48bdb353-9b76-4bb3-bc91-f965ff55e6eb" containerName="manager" Sep 11 18:52:42.528897 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.528849 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2375fb95-12cb-4d7b-96aa-f6195aa9d9df" containerName="manager" Sep 11 18:52:42.530333 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:52:42.530221 2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"2375fb95-12cb-4d7b-96aa-f6195aa9d9df\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-74495c69f8-g984d\": pods \"limitador-operator-controller-manager-74495c69f8-g984d\" not found" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g984d" Sep 11 18:52:42.531794 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.531779 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-n6c6r" Sep 11 18:52:42.532637 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:52:42.532615 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-46zxz\" is forbidden: User \"system:node:ip-10-0-134-192.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-192.ec2.internal' and this object" podUID="48bdb353-9b76-4bb3-bc91-f965ff55e6eb" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-46zxz" Sep 11 18:52:42.537801 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:52:42.537773 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-g984d\" is forbidden: User \"system:node:ip-10-0-134-192.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-192.ec2.internal' and this object" podUID="2375fb95-12cb-4d7b-96aa-f6195aa9d9df" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g984d" Sep 11 18:52:42.540158 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:52:42.540136 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-46zxz\" is forbidden: User \"system:node:ip-10-0-134-192.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-192.ec2.internal' and this object" podUID="48bdb353-9b76-4bb3-bc91-f965ff55e6eb" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-46zxz" Sep 11 18:52:42.544846 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.544814 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-kdgpx"] Sep 11 18:52:42.551720 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.551692 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kdgpx" Sep 11 18:52:42.553926 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.553901 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-n6c6r"] Sep 11 18:52:42.566115 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.566088 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-kdgpx"] Sep 11 18:52:42.580898 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:52:42.580868 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-g984d\" is forbidden: User \"system:node:ip-10-0-134-192.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-192.ec2.internal' and this object" podUID="2375fb95-12cb-4d7b-96aa-f6195aa9d9df" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g984d" Sep 11 18:52:42.591813 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:52:42.591781 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-46zxz\" is forbidden: User \"system:node:ip-10-0-134-192.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-192.ec2.internal' and this object" podUID="48bdb353-9b76-4bb3-bc91-f965ff55e6eb" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-46zxz" Sep 11 18:52:42.701091 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.701058 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkcpp\" (UniqueName: \"kubernetes.io/projected/79f2446b-9ff0-44cc-8aff-24ee7f116311-kube-api-access-kkcpp\") pod \"limitador-operator-controller-manager-74495c69f8-kdgpx\" (UID: \"79f2446b-9ff0-44cc-8aff-24ee7f116311\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kdgpx" Sep 11 18:52:42.701239 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.701097 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk9r2\" (UniqueName: \"kubernetes.io/projected/7e471a05-1c18-40ce-b9f1-54d4c4770a0a-kube-api-access-fk9r2\") pod \"kuadrant-operator-controller-manager-76f8f686bf-n6c6r\" (UID: \"7e471a05-1c18-40ce-b9f1-54d4c4770a0a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-n6c6r" Sep 11 18:52:42.701239 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.701157 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e471a05-1c18-40ce-b9f1-54d4c4770a0a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-n6c6r\" (UID: \"7e471a05-1c18-40ce-b9f1-54d4c4770a0a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-n6c6r" Sep 11 18:52:42.734336 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.734310 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-46zxz" Sep 11 18:52:42.736819 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:52:42.736792 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-g984d\" is forbidden: User \"system:node:ip-10-0-134-192.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-192.ec2.internal' and this object" podUID="2375fb95-12cb-4d7b-96aa-f6195aa9d9df" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g984d" Sep 11 18:52:42.737191 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.737173 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g984d" Sep 11 18:52:42.739082 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:52:42.739059 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-46zxz\" is forbidden: User \"system:node:ip-10-0-134-192.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-192.ec2.internal' and this object" podUID="48bdb353-9b76-4bb3-bc91-f965ff55e6eb" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-46zxz" Sep 11 18:52:42.741478 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:52:42.741458 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-46zxz\" is forbidden: User \"system:node:ip-10-0-134-192.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-192.ec2.internal' and this object" podUID="48bdb353-9b76-4bb3-bc91-f965ff55e6eb" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-46zxz" Sep 11 18:52:42.743810 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:52:42.743789 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-g984d\" is forbidden: User \"system:node:ip-10-0-134-192.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-192.ec2.internal' and this object" podUID="2375fb95-12cb-4d7b-96aa-f6195aa9d9df" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g984d" Sep 11 18:52:42.801992 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.801961 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e471a05-1c18-40ce-b9f1-54d4c4770a0a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-n6c6r\" (UID: \"7e471a05-1c18-40ce-b9f1-54d4c4770a0a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-n6c6r" Sep 11 18:52:42.802139 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.802025 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkcpp\" (UniqueName: \"kubernetes.io/projected/79f2446b-9ff0-44cc-8aff-24ee7f116311-kube-api-access-kkcpp\") pod \"limitador-operator-controller-manager-74495c69f8-kdgpx\" (UID: \"79f2446b-9ff0-44cc-8aff-24ee7f116311\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kdgpx" Sep 11 18:52:42.802139 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.802066 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk9r2\" (UniqueName: \"kubernetes.io/projected/7e471a05-1c18-40ce-b9f1-54d4c4770a0a-kube-api-access-fk9r2\") pod \"kuadrant-operator-controller-manager-76f8f686bf-n6c6r\" (UID: \"7e471a05-1c18-40ce-b9f1-54d4c4770a0a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-n6c6r" Sep 11 18:52:42.802395 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.802361 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e471a05-1c18-40ce-b9f1-54d4c4770a0a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-n6c6r\" (UID: \"7e471a05-1c18-40ce-b9f1-54d4c4770a0a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-n6c6r" Sep 11 18:52:42.822975 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.822949 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkcpp\" (UniqueName: \"kubernetes.io/projected/79f2446b-9ff0-44cc-8aff-24ee7f116311-kube-api-access-kkcpp\") pod \"limitador-operator-controller-manager-74495c69f8-kdgpx\" (UID: \"79f2446b-9ff0-44cc-8aff-24ee7f116311\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kdgpx" Sep 11 18:52:42.826199 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.826173 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk9r2\" (UniqueName: \"kubernetes.io/projected/7e471a05-1c18-40ce-b9f1-54d4c4770a0a-kube-api-access-fk9r2\") pod \"kuadrant-operator-controller-manager-76f8f686bf-n6c6r\" (UID: \"7e471a05-1c18-40ce-b9f1-54d4c4770a0a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-n6c6r" Sep 11 18:52:42.887978 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.887945 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-n6c6r" Sep 11 18:52:42.893963 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.893943 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kdgpx" Sep 11 18:52:42.902957 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.902936 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbgn4\" (UniqueName: \"kubernetes.io/projected/48bdb353-9b76-4bb3-bc91-f965ff55e6eb-kube-api-access-bbgn4\") pod \"48bdb353-9b76-4bb3-bc91-f965ff55e6eb\" (UID: \"48bdb353-9b76-4bb3-bc91-f965ff55e6eb\") " Sep 11 18:52:42.903047 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.902988 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/48bdb353-9b76-4bb3-bc91-f965ff55e6eb-extensions-socket-volume\") pod \"48bdb353-9b76-4bb3-bc91-f965ff55e6eb\" (UID: \"48bdb353-9b76-4bb3-bc91-f965ff55e6eb\") " Sep 11 18:52:42.903047 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.903021 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqqj5\" (UniqueName: \"kubernetes.io/projected/2375fb95-12cb-4d7b-96aa-f6195aa9d9df-kube-api-access-tqqj5\") pod \"2375fb95-12cb-4d7b-96aa-f6195aa9d9df\" (UID: \"2375fb95-12cb-4d7b-96aa-f6195aa9d9df\") " Sep 11 18:52:42.903419 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.903391 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48bdb353-9b76-4bb3-bc91-f965ff55e6eb-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "48bdb353-9b76-4bb3-bc91-f965ff55e6eb" (UID: "48bdb353-9b76-4bb3-bc91-f965ff55e6eb"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 18:52:42.905387 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.905352 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2375fb95-12cb-4d7b-96aa-f6195aa9d9df-kube-api-access-tqqj5" (OuterVolumeSpecName: "kube-api-access-tqqj5") pod "2375fb95-12cb-4d7b-96aa-f6195aa9d9df" (UID: "2375fb95-12cb-4d7b-96aa-f6195aa9d9df"). InnerVolumeSpecName "kube-api-access-tqqj5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 18:52:42.905471 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:42.905444 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48bdb353-9b76-4bb3-bc91-f965ff55e6eb-kube-api-access-bbgn4" (OuterVolumeSpecName: "kube-api-access-bbgn4") pod "48bdb353-9b76-4bb3-bc91-f965ff55e6eb" (UID: "48bdb353-9b76-4bb3-bc91-f965ff55e6eb"). InnerVolumeSpecName "kube-api-access-bbgn4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 18:52:43.003829 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:43.003795 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbgn4\" (UniqueName: \"kubernetes.io/projected/48bdb353-9b76-4bb3-bc91-f965ff55e6eb-kube-api-access-bbgn4\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:52:43.003829 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:43.003827 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/48bdb353-9b76-4bb3-bc91-f965ff55e6eb-extensions-socket-volume\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:52:43.004030 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:43.003839 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqqj5\" (UniqueName: \"kubernetes.io/projected/2375fb95-12cb-4d7b-96aa-f6195aa9d9df-kube-api-access-tqqj5\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:52:43.060716 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:43.060683 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2375fb95-12cb-4d7b-96aa-f6195aa9d9df" path="/var/lib/kubelet/pods/2375fb95-12cb-4d7b-96aa-f6195aa9d9df/volumes" Sep 11 18:52:43.061057 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:43.061044 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48bdb353-9b76-4bb3-bc91-f965ff55e6eb" path="/var/lib/kubelet/pods/48bdb353-9b76-4bb3-bc91-f965ff55e6eb/volumes" Sep 11 18:52:43.139285 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:43.139251 2563 generic.go:356] "Generic (PLEG): container finished" podID="2375fb95-12cb-4d7b-96aa-f6195aa9d9df" containerID="2207d448f4902ad49c300edcf02baf6f382d1b6a258715f1b0a441393d075397" exitCode=0 Sep 11 18:52:43.139464 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:43.139299 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g984d" Sep 11 18:52:43.139464 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:43.139354 2563 scope.go:117] "RemoveContainer" containerID="2207d448f4902ad49c300edcf02baf6f382d1b6a258715f1b0a441393d075397" Sep 11 18:52:43.141085 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:43.140997 2563 generic.go:356] "Generic (PLEG): container finished" podID="48bdb353-9b76-4bb3-bc91-f965ff55e6eb" containerID="8d87742e3afa93f887e5e0e39c53c61ce3c84ab2fd258ba702aac5d198dbe930" exitCode=0 Sep 11 18:52:43.141085 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:43.141049 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-46zxz" Sep 11 18:52:43.149853 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:43.149831 2563 scope.go:117] "RemoveContainer" containerID="2207d448f4902ad49c300edcf02baf6f382d1b6a258715f1b0a441393d075397" Sep 11 18:52:43.150867 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:52:43.150832 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2207d448f4902ad49c300edcf02baf6f382d1b6a258715f1b0a441393d075397\": container with ID starting with 2207d448f4902ad49c300edcf02baf6f382d1b6a258715f1b0a441393d075397 not found: ID does not exist" containerID="2207d448f4902ad49c300edcf02baf6f382d1b6a258715f1b0a441393d075397" Sep 11 18:52:43.150976 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:43.150871 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2207d448f4902ad49c300edcf02baf6f382d1b6a258715f1b0a441393d075397"} err="failed to get container status \"2207d448f4902ad49c300edcf02baf6f382d1b6a258715f1b0a441393d075397\": rpc error: code = NotFound desc = could not find container \"2207d448f4902ad49c300edcf02baf6f382d1b6a258715f1b0a441393d075397\": container with ID starting with 2207d448f4902ad49c300edcf02baf6f382d1b6a258715f1b0a441393d075397 not found: ID does not exist" Sep 11 18:52:43.150976 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:43.150895 2563 scope.go:117] "RemoveContainer" containerID="8d87742e3afa93f887e5e0e39c53c61ce3c84ab2fd258ba702aac5d198dbe930" Sep 11 18:52:43.158882 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:43.158860 2563 scope.go:117] "RemoveContainer" containerID="8d87742e3afa93f887e5e0e39c53c61ce3c84ab2fd258ba702aac5d198dbe930" Sep 11 18:52:43.159162 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:52:43.159137 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d87742e3afa93f887e5e0e39c53c61ce3c84ab2fd258ba702aac5d198dbe930\": container with ID starting with 8d87742e3afa93f887e5e0e39c53c61ce3c84ab2fd258ba702aac5d198dbe930 not found: ID does not exist" containerID="8d87742e3afa93f887e5e0e39c53c61ce3c84ab2fd258ba702aac5d198dbe930" Sep 11 18:52:43.159249 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:43.159165 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d87742e3afa93f887e5e0e39c53c61ce3c84ab2fd258ba702aac5d198dbe930"} err="failed to get container status \"8d87742e3afa93f887e5e0e39c53c61ce3c84ab2fd258ba702aac5d198dbe930\": rpc error: code = NotFound desc = could not find container \"8d87742e3afa93f887e5e0e39c53c61ce3c84ab2fd258ba702aac5d198dbe930\": container with ID starting with 8d87742e3afa93f887e5e0e39c53c61ce3c84ab2fd258ba702aac5d198dbe930 not found: ID does not exist" Sep 11 18:52:43.185560 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:43.185464 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-n6c6r"] Sep 11 18:52:43.188345 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:52:43.188309 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e471a05_1c18_40ce_b9f1_54d4c4770a0a.slice/crio-241e3c7b52f763b889e172c03d707f9ec2abbea0a48927c45116a9bb054b97bf WatchSource:0}: Error finding container 241e3c7b52f763b889e172c03d707f9ec2abbea0a48927c45116a9bb054b97bf: Status 404 returned error can't find the container with id 241e3c7b52f763b889e172c03d707f9ec2abbea0a48927c45116a9bb054b97bf Sep 11 18:52:43.217121 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:43.217097 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-kdgpx"] Sep 11 18:52:43.220151 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:52:43.220125 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79f2446b_9ff0_44cc_8aff_24ee7f116311.slice/crio-4a36d8979cb9bf21570d6d99b94d6a936f54c3d4a6b6cff2646e00bcf7146902 WatchSource:0}: Error finding container 4a36d8979cb9bf21570d6d99b94d6a936f54c3d4a6b6cff2646e00bcf7146902: Status 404 returned error can't find the container with id 4a36d8979cb9bf21570d6d99b94d6a936f54c3d4a6b6cff2646e00bcf7146902 Sep 11 18:52:43.447488 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:43.447459 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:43.581232 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:43.581191 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:43.741901 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:43.741814 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:44.100393 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:44.100366 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-625tz" Sep 11 18:52:44.149536 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:44.149461 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kdgpx" event={"ID":"79f2446b-9ff0-44cc-8aff-24ee7f116311","Type":"ContainerStarted","Data":"6c5bcccbe74e6e28767a93ec6496d8144df7028721cfc66e08fc1007b756e010"} Sep 11 18:52:44.149702 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:44.149552 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kdgpx" Sep 11 18:52:44.149702 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:44.149577 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kdgpx" event={"ID":"79f2446b-9ff0-44cc-8aff-24ee7f116311","Type":"ContainerStarted","Data":"4a36d8979cb9bf21570d6d99b94d6a936f54c3d4a6b6cff2646e00bcf7146902"} Sep 11 18:52:44.153188 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:44.153161 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-n6c6r" event={"ID":"7e471a05-1c18-40ce-b9f1-54d4c4770a0a","Type":"ContainerStarted","Data":"74c3e231057c47056d1ad510ec2ae484f274ea9137024ea2bcbfd3b973727059"} Sep 11 18:52:44.153297 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:44.153204 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-n6c6r" event={"ID":"7e471a05-1c18-40ce-b9f1-54d4c4770a0a","Type":"ContainerStarted","Data":"241e3c7b52f763b889e172c03d707f9ec2abbea0a48927c45116a9bb054b97bf"} Sep 11 18:52:44.153297 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:44.153236 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-n6c6r" Sep 11 18:52:44.200479 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:44.200418 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kdgpx" podStartSLOduration=2.200401515 podStartE2EDuration="2.200401515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:52:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:52:44.199361482 +0000 UTC m=+485.661967513" watchObservedRunningTime="2026-09-11 18:52:44.200401515 +0000 UTC m=+485.663007533" Sep 11 18:52:44.233029 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:44.232976 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-n6c6r" podStartSLOduration=2.232960331 podStartE2EDuration="2.232960331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:52:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:52:44.229497626 +0000 UTC m=+485.692103645" watchObservedRunningTime="2026-09-11 18:52:44.232960331 +0000 UTC m=+485.695566350" Sep 11 18:52:46.338314 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:46.338280 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:46.437041 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:46.437007 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:48.850803 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:48.850770 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:49.042006 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:49.041945 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:55.160387 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:55.160350 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kdgpx" Sep 11 18:52:55.160944 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:55.160437 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-n6c6r" Sep 11 18:52:55.399162 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:55.399128 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:55.499928 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:55.499844 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:52:55.605424 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:52:55.605389 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:00.428366 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:00.428332 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-n6c6r"] Sep 11 18:53:00.428819 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:00.428583 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-n6c6r" podUID="7e471a05-1c18-40ce-b9f1-54d4c4770a0a" containerName="manager" containerID="cri-o://74c3e231057c47056d1ad510ec2ae484f274ea9137024ea2bcbfd3b973727059" gracePeriod=10 Sep 11 18:53:00.669354 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:00.669327 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-n6c6r" Sep 11 18:53:00.751784 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:00.751695 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fk9r2\" (UniqueName: \"kubernetes.io/projected/7e471a05-1c18-40ce-b9f1-54d4c4770a0a-kube-api-access-fk9r2\") pod \"7e471a05-1c18-40ce-b9f1-54d4c4770a0a\" (UID: \"7e471a05-1c18-40ce-b9f1-54d4c4770a0a\") " Sep 11 18:53:00.751784 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:00.751773 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e471a05-1c18-40ce-b9f1-54d4c4770a0a-extensions-socket-volume\") pod \"7e471a05-1c18-40ce-b9f1-54d4c4770a0a\" (UID: \"7e471a05-1c18-40ce-b9f1-54d4c4770a0a\") " Sep 11 18:53:00.752215 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:00.752182 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e471a05-1c18-40ce-b9f1-54d4c4770a0a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7e471a05-1c18-40ce-b9f1-54d4c4770a0a" (UID: "7e471a05-1c18-40ce-b9f1-54d4c4770a0a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 18:53:00.754008 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:00.753977 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e471a05-1c18-40ce-b9f1-54d4c4770a0a-kube-api-access-fk9r2" (OuterVolumeSpecName: "kube-api-access-fk9r2") pod "7e471a05-1c18-40ce-b9f1-54d4c4770a0a" (UID: "7e471a05-1c18-40ce-b9f1-54d4c4770a0a"). InnerVolumeSpecName "kube-api-access-fk9r2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 18:53:00.852860 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:00.852821 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fk9r2\" (UniqueName: \"kubernetes.io/projected/7e471a05-1c18-40ce-b9f1-54d4c4770a0a-kube-api-access-fk9r2\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:53:00.852860 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:00.852855 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e471a05-1c18-40ce-b9f1-54d4c4770a0a-extensions-socket-volume\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:53:01.230215 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:01.230184 2563 generic.go:356] "Generic (PLEG): container finished" podID="7e471a05-1c18-40ce-b9f1-54d4c4770a0a" containerID="74c3e231057c47056d1ad510ec2ae484f274ea9137024ea2bcbfd3b973727059" exitCode=0 Sep 11 18:53:01.230394 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:01.230235 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-n6c6r" Sep 11 18:53:01.230394 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:01.230255 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-n6c6r" event={"ID":"7e471a05-1c18-40ce-b9f1-54d4c4770a0a","Type":"ContainerDied","Data":"74c3e231057c47056d1ad510ec2ae484f274ea9137024ea2bcbfd3b973727059"} Sep 11 18:53:01.230394 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:01.230296 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-n6c6r" event={"ID":"7e471a05-1c18-40ce-b9f1-54d4c4770a0a","Type":"ContainerDied","Data":"241e3c7b52f763b889e172c03d707f9ec2abbea0a48927c45116a9bb054b97bf"} Sep 11 18:53:01.230394 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:01.230318 2563 scope.go:117] "RemoveContainer" containerID="74c3e231057c47056d1ad510ec2ae484f274ea9137024ea2bcbfd3b973727059" Sep 11 18:53:01.238545 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:01.238526 2563 scope.go:117] "RemoveContainer" containerID="74c3e231057c47056d1ad510ec2ae484f274ea9137024ea2bcbfd3b973727059" Sep 11 18:53:01.238858 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:53:01.238837 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74c3e231057c47056d1ad510ec2ae484f274ea9137024ea2bcbfd3b973727059\": container with ID starting with 74c3e231057c47056d1ad510ec2ae484f274ea9137024ea2bcbfd3b973727059 not found: ID does not exist" containerID="74c3e231057c47056d1ad510ec2ae484f274ea9137024ea2bcbfd3b973727059" Sep 11 18:53:01.238944 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:01.238868 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74c3e231057c47056d1ad510ec2ae484f274ea9137024ea2bcbfd3b973727059"} err="failed to get container status \"74c3e231057c47056d1ad510ec2ae484f274ea9137024ea2bcbfd3b973727059\": rpc error: code = NotFound desc = could not find container \"74c3e231057c47056d1ad510ec2ae484f274ea9137024ea2bcbfd3b973727059\": container with ID starting with 74c3e231057c47056d1ad510ec2ae484f274ea9137024ea2bcbfd3b973727059 not found: ID does not exist" Sep 11 18:53:01.262888 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:01.262855 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-n6c6r"] Sep 11 18:53:01.269650 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:01.269627 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-n6c6r"] Sep 11 18:53:03.060576 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:03.060543 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e471a05-1c18-40ce-b9f1-54d4c4770a0a" path="/var/lib/kubelet/pods/7e471a05-1c18-40ce-b9f1-54d4c4770a0a/volumes" Sep 11 18:53:04.236065 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:04.236033 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:04.335294 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:04.335259 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:07.297009 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:07.296975 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:07.421179 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:07.421150 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:09.702236 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:09.702206 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:09.769246 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:09.769212 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:16.693314 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.693274 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:16.802632 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.802601 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:16.830657 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.830619 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x"] Sep 11 18:53:16.830956 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.830944 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e471a05-1c18-40ce-b9f1-54d4c4770a0a" containerName="manager" Sep 11 18:53:16.831002 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.830957 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e471a05-1c18-40ce-b9f1-54d4c4770a0a" containerName="manager" Sep 11 18:53:16.831036 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.831013 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e471a05-1c18-40ce-b9f1-54d4c4770a0a" containerName="manager" Sep 11 18:53:16.832951 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.832936 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:16.835952 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.835935 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-d2vbr\"" Sep 11 18:53:16.851019 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.850993 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x"] Sep 11 18:53:16.889059 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.889025 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fa085de-dbaf-4ead-9545-3e80fdd52307-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-klg7x\" (UID: \"0fa085de-dbaf-4ead-9545-3e80fdd52307\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:16.889238 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.889070 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fa085de-dbaf-4ead-9545-3e80fdd52307-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-klg7x\" (UID: \"0fa085de-dbaf-4ead-9545-3e80fdd52307\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:16.889238 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.889100 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fa085de-dbaf-4ead-9545-3e80fdd52307-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-klg7x\" (UID: \"0fa085de-dbaf-4ead-9545-3e80fdd52307\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:16.889238 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.889171 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp7bg\" (UniqueName: \"kubernetes.io/projected/0fa085de-dbaf-4ead-9545-3e80fdd52307-kube-api-access-gp7bg\") pod \"maas-default-gateway-openshift-default-78bb78ff78-klg7x\" (UID: \"0fa085de-dbaf-4ead-9545-3e80fdd52307\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:16.889238 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.889210 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fa085de-dbaf-4ead-9545-3e80fdd52307-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-klg7x\" (UID: \"0fa085de-dbaf-4ead-9545-3e80fdd52307\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:16.889238 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.889230 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fa085de-dbaf-4ead-9545-3e80fdd52307-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-klg7x\" (UID: \"0fa085de-dbaf-4ead-9545-3e80fdd52307\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:16.889406 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.889244 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fa085de-dbaf-4ead-9545-3e80fdd52307-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-klg7x\" (UID: \"0fa085de-dbaf-4ead-9545-3e80fdd52307\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:16.889406 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.889280 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fa085de-dbaf-4ead-9545-3e80fdd52307-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-klg7x\" (UID: \"0fa085de-dbaf-4ead-9545-3e80fdd52307\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:16.889406 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.889301 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fa085de-dbaf-4ead-9545-3e80fdd52307-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-klg7x\" (UID: \"0fa085de-dbaf-4ead-9545-3e80fdd52307\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:16.912568 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.912523 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:16.990731 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.990630 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp7bg\" (UniqueName: \"kubernetes.io/projected/0fa085de-dbaf-4ead-9545-3e80fdd52307-kube-api-access-gp7bg\") pod \"maas-default-gateway-openshift-default-78bb78ff78-klg7x\" (UID: \"0fa085de-dbaf-4ead-9545-3e80fdd52307\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:16.990731 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.990682 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fa085de-dbaf-4ead-9545-3e80fdd52307-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-klg7x\" (UID: \"0fa085de-dbaf-4ead-9545-3e80fdd52307\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:16.990731 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.990702 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fa085de-dbaf-4ead-9545-3e80fdd52307-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-klg7x\" (UID: \"0fa085de-dbaf-4ead-9545-3e80fdd52307\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:16.990731 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.990720 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fa085de-dbaf-4ead-9545-3e80fdd52307-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-klg7x\" (UID: \"0fa085de-dbaf-4ead-9545-3e80fdd52307\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:16.991086 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.990746 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fa085de-dbaf-4ead-9545-3e80fdd52307-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-klg7x\" (UID: \"0fa085de-dbaf-4ead-9545-3e80fdd52307\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:16.991086 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.990764 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fa085de-dbaf-4ead-9545-3e80fdd52307-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-klg7x\" (UID: \"0fa085de-dbaf-4ead-9545-3e80fdd52307\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:16.991086 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.990787 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fa085de-dbaf-4ead-9545-3e80fdd52307-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-klg7x\" (UID: \"0fa085de-dbaf-4ead-9545-3e80fdd52307\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:16.991086 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.990826 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fa085de-dbaf-4ead-9545-3e80fdd52307-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-klg7x\" (UID: \"0fa085de-dbaf-4ead-9545-3e80fdd52307\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:16.991086 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.990876 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fa085de-dbaf-4ead-9545-3e80fdd52307-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-klg7x\" (UID: \"0fa085de-dbaf-4ead-9545-3e80fdd52307\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:16.991337 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.991219 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fa085de-dbaf-4ead-9545-3e80fdd52307-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-klg7x\" (UID: \"0fa085de-dbaf-4ead-9545-3e80fdd52307\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:16.991337 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.991245 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fa085de-dbaf-4ead-9545-3e80fdd52307-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-klg7x\" (UID: \"0fa085de-dbaf-4ead-9545-3e80fdd52307\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:16.991447 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.991332 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fa085de-dbaf-4ead-9545-3e80fdd52307-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-klg7x\" (UID: \"0fa085de-dbaf-4ead-9545-3e80fdd52307\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:16.991447 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.991376 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fa085de-dbaf-4ead-9545-3e80fdd52307-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-klg7x\" (UID: \"0fa085de-dbaf-4ead-9545-3e80fdd52307\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:16.991726 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.991700 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fa085de-dbaf-4ead-9545-3e80fdd52307-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-klg7x\" (UID: \"0fa085de-dbaf-4ead-9545-3e80fdd52307\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:16.993250 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.993223 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fa085de-dbaf-4ead-9545-3e80fdd52307-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-klg7x\" (UID: \"0fa085de-dbaf-4ead-9545-3e80fdd52307\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:16.993357 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:16.993338 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fa085de-dbaf-4ead-9545-3e80fdd52307-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-klg7x\" (UID: \"0fa085de-dbaf-4ead-9545-3e80fdd52307\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:17.002852 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:17.002827 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp7bg\" (UniqueName: \"kubernetes.io/projected/0fa085de-dbaf-4ead-9545-3e80fdd52307-kube-api-access-gp7bg\") pod \"maas-default-gateway-openshift-default-78bb78ff78-klg7x\" (UID: \"0fa085de-dbaf-4ead-9545-3e80fdd52307\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:17.004316 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:17.004287 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fa085de-dbaf-4ead-9545-3e80fdd52307-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-klg7x\" (UID: \"0fa085de-dbaf-4ead-9545-3e80fdd52307\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:17.028544 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:17.028489 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:17.123801 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:17.123765 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:17.142768 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:17.142730 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:17.261546 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:17.261450 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:17.361872 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:17.361841 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:17.421345 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:17.421320 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x"] Sep 11 18:53:17.423672 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:53:17.423642 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fa085de_dbaf_4ead_9545_3e80fdd52307.slice/crio-b4ebeb2637ff08bc53b56e3a9e4223669b1dd5f82c3918bc85d1fe3880894969 WatchSource:0}: Error finding container b4ebeb2637ff08bc53b56e3a9e4223669b1dd5f82c3918bc85d1fe3880894969: Status 404 returned error can't find the container with id b4ebeb2637ff08bc53b56e3a9e4223669b1dd5f82c3918bc85d1fe3880894969 Sep 11 18:53:17.425669 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:17.425637 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 18:53:17.425750 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:17.425711 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 18:53:17.425750 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:17.425738 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 11 18:53:18.309662 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:18.309625 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" event={"ID":"0fa085de-dbaf-4ead-9545-3e80fdd52307","Type":"ContainerStarted","Data":"c67c7f6ed5e6379541226f6599c75e3359e0b1b69258b4832ed3ca75000eed8a"} Sep 11 18:53:18.310057 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:18.309678 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" event={"ID":"0fa085de-dbaf-4ead-9545-3e80fdd52307","Type":"ContainerStarted","Data":"b4ebeb2637ff08bc53b56e3a9e4223669b1dd5f82c3918bc85d1fe3880894969"} Sep 11 18:53:18.332807 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:18.332755 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" podStartSLOduration=2.332739429 podStartE2EDuration="2.332739429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:53:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:53:18.330849197 +0000 UTC m=+519.793455216" watchObservedRunningTime="2026-09-11 18:53:18.332739429 +0000 UTC m=+519.795345447" Sep 11 18:53:19.143836 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:19.143806 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:19.149271 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:19.149244 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:19.314417 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:19.314377 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:19.315407 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:19.315384 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-klg7x" Sep 11 18:53:19.478130 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:19.478019 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:19.506187 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:19.506157 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-vt74q"] Sep 11 18:53:19.508387 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:19.508364 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-vt74q" Sep 11 18:53:19.512161 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:19.512138 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-q4t76\"" Sep 11 18:53:19.512716 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:19.512696 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Sep 11 18:53:19.525916 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:19.525889 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-vt74q"] Sep 11 18:53:19.587519 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:19.587481 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-vt74q"] Sep 11 18:53:19.604768 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:19.604735 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:19.615913 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:19.615884 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2bf99578-15c4-4869-8476-73daeb4d2a71-config-file\") pod \"limitador-limitador-8588749977-vt74q\" (UID: \"2bf99578-15c4-4869-8476-73daeb4d2a71\") " pod="kuadrant-system/limitador-limitador-8588749977-vt74q" Sep 11 18:53:19.616020 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:19.615979 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4p4t\" (UniqueName: \"kubernetes.io/projected/2bf99578-15c4-4869-8476-73daeb4d2a71-kube-api-access-g4p4t\") pod \"limitador-limitador-8588749977-vt74q\" (UID: \"2bf99578-15c4-4869-8476-73daeb4d2a71\") " pod="kuadrant-system/limitador-limitador-8588749977-vt74q" Sep 11 18:53:19.686384 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:19.686344 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:19.716605 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:19.716569 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4p4t\" (UniqueName: \"kubernetes.io/projected/2bf99578-15c4-4869-8476-73daeb4d2a71-kube-api-access-g4p4t\") pod \"limitador-limitador-8588749977-vt74q\" (UID: \"2bf99578-15c4-4869-8476-73daeb4d2a71\") " pod="kuadrant-system/limitador-limitador-8588749977-vt74q" Sep 11 18:53:19.716789 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:19.716742 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2bf99578-15c4-4869-8476-73daeb4d2a71-config-file\") pod \"limitador-limitador-8588749977-vt74q\" (UID: \"2bf99578-15c4-4869-8476-73daeb4d2a71\") " pod="kuadrant-system/limitador-limitador-8588749977-vt74q" Sep 11 18:53:19.717347 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:19.717329 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2bf99578-15c4-4869-8476-73daeb4d2a71-config-file\") pod \"limitador-limitador-8588749977-vt74q\" (UID: \"2bf99578-15c4-4869-8476-73daeb4d2a71\") " pod="kuadrant-system/limitador-limitador-8588749977-vt74q" Sep 11 18:53:19.725466 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:19.725439 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4p4t\" (UniqueName: \"kubernetes.io/projected/2bf99578-15c4-4869-8476-73daeb4d2a71-kube-api-access-g4p4t\") pod \"limitador-limitador-8588749977-vt74q\" (UID: \"2bf99578-15c4-4869-8476-73daeb4d2a71\") " pod="kuadrant-system/limitador-limitador-8588749977-vt74q" Sep 11 18:53:19.819786 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:19.819753 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-vt74q" Sep 11 18:53:20.096483 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:20.096458 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-vt74q"] Sep 11 18:53:20.097470 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:53:20.097441 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bf99578_15c4_4869_8476_73daeb4d2a71.slice/crio-df34c2d859668bedb776e550bcfd56259687593485927c9ab78e36e10226c15b WatchSource:0}: Error finding container df34c2d859668bedb776e550bcfd56259687593485927c9ab78e36e10226c15b: Status 404 returned error can't find the container with id df34c2d859668bedb776e550bcfd56259687593485927c9ab78e36e10226c15b Sep 11 18:53:20.319842 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:20.319805 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-vt74q" event={"ID":"2bf99578-15c4-4869-8476-73daeb4d2a71","Type":"ContainerStarted","Data":"df34c2d859668bedb776e550bcfd56259687593485927c9ab78e36e10226c15b"} Sep 11 18:53:20.413109 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:20.413033 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:20.512077 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:20.512044 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:20.585861 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:20.585825 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-ws4fq"] Sep 11 18:53:20.588593 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:20.588567 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-ws4fq" Sep 11 18:53:20.591565 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:20.591543 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-kvm6t\"" Sep 11 18:53:20.601572 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:20.601547 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-ws4fq"] Sep 11 18:53:20.726654 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:20.726578 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wrck\" (UniqueName: \"kubernetes.io/projected/da2e4010-abdd-4dd4-a60b-38c30195f4e5-kube-api-access-8wrck\") pod \"authorino-59c974bd48-ws4fq\" (UID: \"da2e4010-abdd-4dd4-a60b-38c30195f4e5\") " pod="kuadrant-system/authorino-59c974bd48-ws4fq" Sep 11 18:53:20.827045 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:20.827014 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wrck\" (UniqueName: \"kubernetes.io/projected/da2e4010-abdd-4dd4-a60b-38c30195f4e5-kube-api-access-8wrck\") pod \"authorino-59c974bd48-ws4fq\" (UID: \"da2e4010-abdd-4dd4-a60b-38c30195f4e5\") " pod="kuadrant-system/authorino-59c974bd48-ws4fq" Sep 11 18:53:20.837084 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:20.837053 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wrck\" (UniqueName: \"kubernetes.io/projected/da2e4010-abdd-4dd4-a60b-38c30195f4e5-kube-api-access-8wrck\") pod \"authorino-59c974bd48-ws4fq\" (UID: \"da2e4010-abdd-4dd4-a60b-38c30195f4e5\") " pod="kuadrant-system/authorino-59c974bd48-ws4fq" Sep 11 18:53:20.897940 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:20.897901 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-ws4fq" Sep 11 18:53:21.167326 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:21.167299 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-ws4fq"] Sep 11 18:53:21.168936 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:53:21.168907 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda2e4010_abdd_4dd4_a60b_38c30195f4e5.slice/crio-9d85561eb609bb89b7bd3337e96a57ba4aed2f54470426a187bfa9d08b226c9a WatchSource:0}: Error finding container 9d85561eb609bb89b7bd3337e96a57ba4aed2f54470426a187bfa9d08b226c9a: Status 404 returned error can't find the container with id 9d85561eb609bb89b7bd3337e96a57ba4aed2f54470426a187bfa9d08b226c9a Sep 11 18:53:21.331647 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:21.331528 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-ws4fq" event={"ID":"da2e4010-abdd-4dd4-a60b-38c30195f4e5","Type":"ContainerStarted","Data":"9d85561eb609bb89b7bd3337e96a57ba4aed2f54470426a187bfa9d08b226c9a"} Sep 11 18:53:22.338680 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:22.338641 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-vt74q" event={"ID":"2bf99578-15c4-4869-8476-73daeb4d2a71","Type":"ContainerStarted","Data":"b0b42234416ceab0a47af3d302f200cb756b4fe041b1426ae8518f21d7462cab"} Sep 11 18:53:22.339161 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:22.338739 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-vt74q" Sep 11 18:53:22.380363 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:22.380299 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-vt74q" podStartSLOduration=1.862421045 podStartE2EDuration="3.38027801s" totalImagesPullingTime="1.517856965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:53:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:53:22.379663541 +0000 UTC m=+523.842269572" watchObservedRunningTime="2026-09-11 18:53:22.38027801 +0000 UTC m=+523.842884028" Sep 11 18:53:23.344451 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:23.344413 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-ws4fq" event={"ID":"da2e4010-abdd-4dd4-a60b-38c30195f4e5","Type":"ContainerStarted","Data":"710005be28da959eb5f8176ef45e93130c376c369512c23f53389e645054b868"} Sep 11 18:53:23.370343 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:23.370288 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-ws4fq" podStartSLOduration=1.872468654 podStartE2EDuration="3.370272282s" totalImagesPullingTime="1.497803628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:53:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:53:23.368730685 +0000 UTC m=+524.831336703" watchObservedRunningTime="2026-09-11 18:53:23.370272282 +0000 UTC m=+524.832878299" Sep 11 18:53:23.766226 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:23.766193 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:23.866619 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:23.866578 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:25.684108 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:25.684074 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:25.783142 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:25.783110 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:28.684624 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:28.684595 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:28.837191 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:28.837158 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:30.795025 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:30.794993 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:30.892988 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:30.892955 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:31.110750 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:31.110616 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:31.175528 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:31.175472 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:33.345313 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:33.345278 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-vt74q" Sep 11 18:53:34.971923 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:34.971884 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-vt74q"] Sep 11 18:53:34.972318 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:34.972083 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-vt74q" podUID="2bf99578-15c4-4869-8476-73daeb4d2a71" containerName="limitador" containerID="cri-o://b0b42234416ceab0a47af3d302f200cb756b4fe041b1426ae8518f21d7462cab" gracePeriod=30 Sep 11 18:53:35.403355 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:35.403317 2563 generic.go:356] "Generic (PLEG): container finished" podID="2bf99578-15c4-4869-8476-73daeb4d2a71" containerID="b0b42234416ceab0a47af3d302f200cb756b4fe041b1426ae8518f21d7462cab" exitCode=0 Sep 11 18:53:35.403524 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:35.403390 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-vt74q" event={"ID":"2bf99578-15c4-4869-8476-73daeb4d2a71","Type":"ContainerDied","Data":"b0b42234416ceab0a47af3d302f200cb756b4fe041b1426ae8518f21d7462cab"} Sep 11 18:53:35.508622 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:35.508593 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-vt74q" Sep 11 18:53:35.655293 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:35.655200 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4p4t\" (UniqueName: \"kubernetes.io/projected/2bf99578-15c4-4869-8476-73daeb4d2a71-kube-api-access-g4p4t\") pod \"2bf99578-15c4-4869-8476-73daeb4d2a71\" (UID: \"2bf99578-15c4-4869-8476-73daeb4d2a71\") " Sep 11 18:53:35.655293 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:35.655243 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2bf99578-15c4-4869-8476-73daeb4d2a71-config-file\") pod \"2bf99578-15c4-4869-8476-73daeb4d2a71\" (UID: \"2bf99578-15c4-4869-8476-73daeb4d2a71\") " Sep 11 18:53:35.655721 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:35.655690 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bf99578-15c4-4869-8476-73daeb4d2a71-config-file" (OuterVolumeSpecName: "config-file") pod "2bf99578-15c4-4869-8476-73daeb4d2a71" (UID: "2bf99578-15c4-4869-8476-73daeb4d2a71"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 18:53:35.657380 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:35.657358 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bf99578-15c4-4869-8476-73daeb4d2a71-kube-api-access-g4p4t" (OuterVolumeSpecName: "kube-api-access-g4p4t") pod "2bf99578-15c4-4869-8476-73daeb4d2a71" (UID: "2bf99578-15c4-4869-8476-73daeb4d2a71"). InnerVolumeSpecName "kube-api-access-g4p4t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 18:53:35.755934 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:35.755890 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4p4t\" (UniqueName: \"kubernetes.io/projected/2bf99578-15c4-4869-8476-73daeb4d2a71-kube-api-access-g4p4t\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:53:35.755934 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:35.755924 2563 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2bf99578-15c4-4869-8476-73daeb4d2a71-config-file\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:53:36.409422 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:36.409390 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-vt74q" event={"ID":"2bf99578-15c4-4869-8476-73daeb4d2a71","Type":"ContainerDied","Data":"df34c2d859668bedb776e550bcfd56259687593485927c9ab78e36e10226c15b"} Sep 11 18:53:36.409849 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:36.409396 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-vt74q" Sep 11 18:53:36.409849 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:36.409476 2563 scope.go:117] "RemoveContainer" containerID="b0b42234416ceab0a47af3d302f200cb756b4fe041b1426ae8518f21d7462cab" Sep 11 18:53:36.433182 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:36.433155 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-vt74q"] Sep 11 18:53:36.439766 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:36.439739 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-vt74q"] Sep 11 18:53:37.060894 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:37.060860 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bf99578-15c4-4869-8476-73daeb4d2a71" path="/var/lib/kubelet/pods/2bf99578-15c4-4869-8476-73daeb4d2a71/volumes" Sep 11 18:53:40.614073 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:40.614016 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:40.711947 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:40.711913 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:41.036132 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:41.036100 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6mnj6"] Sep 11 18:53:41.036459 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:41.036446 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bf99578-15c4-4869-8476-73daeb4d2a71" containerName="limitador" Sep 11 18:53:41.036523 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:41.036460 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bf99578-15c4-4869-8476-73daeb4d2a71" containerName="limitador" Sep 11 18:53:41.036561 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:41.036532 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bf99578-15c4-4869-8476-73daeb4d2a71" containerName="limitador" Sep 11 18:53:41.040846 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:41.040827 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6mnj6" Sep 11 18:53:41.045084 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:41.045060 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Sep 11 18:53:41.045391 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:41.045372 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-9skz8\"" Sep 11 18:53:41.046072 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:41.046053 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 11 18:53:41.054365 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:41.054342 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6mnj6"] Sep 11 18:53:41.055285 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:41.055260 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 11 18:53:41.102768 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:41.102734 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjpxt\" (UniqueName: \"kubernetes.io/projected/423aa3c2-859e-4ef9-aeaf-e8e543ddb088-kube-api-access-cjpxt\") pod \"postgres-69fb4594cf-6mnj6\" (UID: \"423aa3c2-859e-4ef9-aeaf-e8e543ddb088\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6mnj6" Sep 11 18:53:41.102947 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:41.102795 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/423aa3c2-859e-4ef9-aeaf-e8e543ddb088-data\") pod \"postgres-69fb4594cf-6mnj6\" (UID: \"423aa3c2-859e-4ef9-aeaf-e8e543ddb088\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6mnj6" Sep 11 18:53:41.113412 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:41.113368 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:41.203568 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:41.203530 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjpxt\" (UniqueName: \"kubernetes.io/projected/423aa3c2-859e-4ef9-aeaf-e8e543ddb088-kube-api-access-cjpxt\") pod \"postgres-69fb4594cf-6mnj6\" (UID: \"423aa3c2-859e-4ef9-aeaf-e8e543ddb088\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6mnj6" Sep 11 18:53:41.203789 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:41.203767 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/423aa3c2-859e-4ef9-aeaf-e8e543ddb088-data\") pod \"postgres-69fb4594cf-6mnj6\" (UID: \"423aa3c2-859e-4ef9-aeaf-e8e543ddb088\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6mnj6" Sep 11 18:53:41.204174 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:41.204154 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/423aa3c2-859e-4ef9-aeaf-e8e543ddb088-data\") pod \"postgres-69fb4594cf-6mnj6\" (UID: \"423aa3c2-859e-4ef9-aeaf-e8e543ddb088\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6mnj6" Sep 11 18:53:41.217729 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:41.217705 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjpxt\" (UniqueName: \"kubernetes.io/projected/423aa3c2-859e-4ef9-aeaf-e8e543ddb088-kube-api-access-cjpxt\") pod \"postgres-69fb4594cf-6mnj6\" (UID: \"423aa3c2-859e-4ef9-aeaf-e8e543ddb088\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6mnj6" Sep 11 18:53:41.351436 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:41.351343 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6mnj6" Sep 11 18:53:41.631423 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:41.631397 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6mnj6"] Sep 11 18:53:41.633433 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:53:41.633400 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod423aa3c2_859e_4ef9_aeaf_e8e543ddb088.slice/crio-465c2efb0a515c11c4f501ddf2f697ff259e8f93cccb99f7ca3c586d90c20efb WatchSource:0}: Error finding container 465c2efb0a515c11c4f501ddf2f697ff259e8f93cccb99f7ca3c586d90c20efb: Status 404 returned error can't find the container with id 465c2efb0a515c11c4f501ddf2f697ff259e8f93cccb99f7ca3c586d90c20efb Sep 11 18:53:42.442814 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:42.442708 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6mnj6" event={"ID":"423aa3c2-859e-4ef9-aeaf-e8e543ddb088","Type":"ContainerStarted","Data":"465c2efb0a515c11c4f501ddf2f697ff259e8f93cccb99f7ca3c586d90c20efb"} Sep 11 18:53:46.465128 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:46.465078 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6mnj6" event={"ID":"423aa3c2-859e-4ef9-aeaf-e8e543ddb088","Type":"ContainerStarted","Data":"a79638a1dc5e2bbdcb76ed21bf9a19370421db937e743a0316a8930b4570b5da"} Sep 11 18:53:46.465599 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:46.465144 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6mnj6" Sep 11 18:53:46.487804 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:46.487754 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6mnj6" podStartSLOduration=0.788533924 podStartE2EDuration="5.487740128s" totalImagesPullingTime="4.699206204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:53:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:53:46.486177427 +0000 UTC m=+547.948783445" watchObservedRunningTime="2026-09-11 18:53:46.487740128 +0000 UTC m=+547.950346145" Sep 11 18:53:52.500296 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:52.500267 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6mnj6" Sep 11 18:53:52.829031 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:52.828990 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:52.936900 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:52.936868 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:53.199779 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:53.199700 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-ssd97"] Sep 11 18:53:53.203134 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:53.203116 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-ssd97" Sep 11 18:53:53.206206 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:53.206181 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Sep 11 18:53:53.213309 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:53.213286 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-ssd97"] Sep 11 18:53:53.307391 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:53.307357 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ebe2abdc-c289-457a-bfbe-93a1779cbbed-tls-cert\") pod \"authorino-6d4dfc649f-ssd97\" (UID: \"ebe2abdc-c289-457a-bfbe-93a1779cbbed\") " pod="kuadrant-system/authorino-6d4dfc649f-ssd97" Sep 11 18:53:53.307574 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:53.307445 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xzwt\" (UniqueName: \"kubernetes.io/projected/ebe2abdc-c289-457a-bfbe-93a1779cbbed-kube-api-access-9xzwt\") pod \"authorino-6d4dfc649f-ssd97\" (UID: \"ebe2abdc-c289-457a-bfbe-93a1779cbbed\") " pod="kuadrant-system/authorino-6d4dfc649f-ssd97" Sep 11 18:53:53.333970 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:53.333927 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-ssd97"] Sep 11 18:53:53.334260 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:53:53.334234 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-9xzwt tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-ssd97" podUID="ebe2abdc-c289-457a-bfbe-93a1779cbbed" Sep 11 18:53:53.408045 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:53.408010 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xzwt\" (UniqueName: \"kubernetes.io/projected/ebe2abdc-c289-457a-bfbe-93a1779cbbed-kube-api-access-9xzwt\") pod \"authorino-6d4dfc649f-ssd97\" (UID: \"ebe2abdc-c289-457a-bfbe-93a1779cbbed\") " pod="kuadrant-system/authorino-6d4dfc649f-ssd97" Sep 11 18:53:53.408246 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:53.408062 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ebe2abdc-c289-457a-bfbe-93a1779cbbed-tls-cert\") pod \"authorino-6d4dfc649f-ssd97\" (UID: \"ebe2abdc-c289-457a-bfbe-93a1779cbbed\") " pod="kuadrant-system/authorino-6d4dfc649f-ssd97" Sep 11 18:53:53.410476 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:53.410455 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ebe2abdc-c289-457a-bfbe-93a1779cbbed-tls-cert\") pod \"authorino-6d4dfc649f-ssd97\" (UID: \"ebe2abdc-c289-457a-bfbe-93a1779cbbed\") " pod="kuadrant-system/authorino-6d4dfc649f-ssd97" Sep 11 18:53:53.428835 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:53.428805 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xzwt\" (UniqueName: \"kubernetes.io/projected/ebe2abdc-c289-457a-bfbe-93a1779cbbed-kube-api-access-9xzwt\") pod \"authorino-6d4dfc649f-ssd97\" (UID: \"ebe2abdc-c289-457a-bfbe-93a1779cbbed\") " pod="kuadrant-system/authorino-6d4dfc649f-ssd97" Sep 11 18:53:53.498742 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:53.498659 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-ssd97" Sep 11 18:53:53.506255 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:53.506225 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-ssd97" Sep 11 18:53:53.610334 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:53.610301 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ebe2abdc-c289-457a-bfbe-93a1779cbbed-tls-cert\") pod \"ebe2abdc-c289-457a-bfbe-93a1779cbbed\" (UID: \"ebe2abdc-c289-457a-bfbe-93a1779cbbed\") " Sep 11 18:53:53.610476 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:53.610396 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xzwt\" (UniqueName: \"kubernetes.io/projected/ebe2abdc-c289-457a-bfbe-93a1779cbbed-kube-api-access-9xzwt\") pod \"ebe2abdc-c289-457a-bfbe-93a1779cbbed\" (UID: \"ebe2abdc-c289-457a-bfbe-93a1779cbbed\") " Sep 11 18:53:53.612488 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:53.612456 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebe2abdc-c289-457a-bfbe-93a1779cbbed-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "ebe2abdc-c289-457a-bfbe-93a1779cbbed" (UID: "ebe2abdc-c289-457a-bfbe-93a1779cbbed"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 18:53:53.612639 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:53.612536 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebe2abdc-c289-457a-bfbe-93a1779cbbed-kube-api-access-9xzwt" (OuterVolumeSpecName: "kube-api-access-9xzwt") pod "ebe2abdc-c289-457a-bfbe-93a1779cbbed" (UID: "ebe2abdc-c289-457a-bfbe-93a1779cbbed"). InnerVolumeSpecName "kube-api-access-9xzwt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 18:53:53.711966 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:53.711930 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xzwt\" (UniqueName: \"kubernetes.io/projected/ebe2abdc-c289-457a-bfbe-93a1779cbbed-kube-api-access-9xzwt\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:53:53.711966 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:53.711959 2563 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ebe2abdc-c289-457a-bfbe-93a1779cbbed-tls-cert\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:53:53.772400 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:53.772365 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:53:54.503706 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:54.503673 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-ssd97" Sep 11 18:53:54.549176 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:54.549144 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-ssd97"] Sep 11 18:53:54.553286 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:54.553258 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-ssd97"] Sep 11 18:53:55.060535 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:55.060485 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebe2abdc-c289-457a-bfbe-93a1779cbbed" path="/var/lib/kubelet/pods/ebe2abdc-c289-457a-bfbe-93a1779cbbed/volumes" Sep 11 18:53:55.123175 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:55.123135 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-ws4fq"] Sep 11 18:53:55.123357 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:55.123336 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-ws4fq" podUID="da2e4010-abdd-4dd4-a60b-38c30195f4e5" containerName="authorino" containerID="cri-o://710005be28da959eb5f8176ef45e93130c376c369512c23f53389e645054b868" gracePeriod=30 Sep 11 18:53:55.377663 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:55.377633 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-ws4fq" Sep 11 18:53:55.430266 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:55.430231 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wrck\" (UniqueName: \"kubernetes.io/projected/da2e4010-abdd-4dd4-a60b-38c30195f4e5-kube-api-access-8wrck\") pod \"da2e4010-abdd-4dd4-a60b-38c30195f4e5\" (UID: \"da2e4010-abdd-4dd4-a60b-38c30195f4e5\") " Sep 11 18:53:55.432299 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:55.432271 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da2e4010-abdd-4dd4-a60b-38c30195f4e5-kube-api-access-8wrck" (OuterVolumeSpecName: "kube-api-access-8wrck") pod "da2e4010-abdd-4dd4-a60b-38c30195f4e5" (UID: "da2e4010-abdd-4dd4-a60b-38c30195f4e5"). InnerVolumeSpecName "kube-api-access-8wrck". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 18:53:55.509290 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:55.509253 2563 generic.go:356] "Generic (PLEG): container finished" podID="da2e4010-abdd-4dd4-a60b-38c30195f4e5" containerID="710005be28da959eb5f8176ef45e93130c376c369512c23f53389e645054b868" exitCode=0 Sep 11 18:53:55.509452 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:55.509295 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-ws4fq" Sep 11 18:53:55.509452 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:55.509337 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-ws4fq" event={"ID":"da2e4010-abdd-4dd4-a60b-38c30195f4e5","Type":"ContainerDied","Data":"710005be28da959eb5f8176ef45e93130c376c369512c23f53389e645054b868"} Sep 11 18:53:55.509452 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:55.509382 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-ws4fq" event={"ID":"da2e4010-abdd-4dd4-a60b-38c30195f4e5","Type":"ContainerDied","Data":"9d85561eb609bb89b7bd3337e96a57ba4aed2f54470426a187bfa9d08b226c9a"} Sep 11 18:53:55.509452 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:55.509402 2563 scope.go:117] "RemoveContainer" containerID="710005be28da959eb5f8176ef45e93130c376c369512c23f53389e645054b868" Sep 11 18:53:55.517972 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:55.517911 2563 scope.go:117] "RemoveContainer" containerID="710005be28da959eb5f8176ef45e93130c376c369512c23f53389e645054b868" Sep 11 18:53:55.518220 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:53:55.518198 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"710005be28da959eb5f8176ef45e93130c376c369512c23f53389e645054b868\": container with ID starting with 710005be28da959eb5f8176ef45e93130c376c369512c23f53389e645054b868 not found: ID does not exist" containerID="710005be28da959eb5f8176ef45e93130c376c369512c23f53389e645054b868" Sep 11 18:53:55.518285 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:55.518232 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"710005be28da959eb5f8176ef45e93130c376c369512c23f53389e645054b868"} err="failed to get container status \"710005be28da959eb5f8176ef45e93130c376c369512c23f53389e645054b868\": rpc error: code = NotFound desc = could not find container \"710005be28da959eb5f8176ef45e93130c376c369512c23f53389e645054b868\": container with ID starting with 710005be28da959eb5f8176ef45e93130c376c369512c23f53389e645054b868 not found: ID does not exist" Sep 11 18:53:55.531280 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:55.531256 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wrck\" (UniqueName: \"kubernetes.io/projected/da2e4010-abdd-4dd4-a60b-38c30195f4e5-kube-api-access-8wrck\") on node \"ip-10-0-134-192.ec2.internal\" DevicePath \"\"" Sep 11 18:53:55.534336 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:55.534305 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-ws4fq"] Sep 11 18:53:55.540519 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:55.540483 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-ws4fq"] Sep 11 18:53:57.060183 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:53:57.060147 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da2e4010-abdd-4dd4-a60b-38c30195f4e5" path="/var/lib/kubelet/pods/da2e4010-abdd-4dd4-a60b-38c30195f4e5/volumes" Sep 11 18:54:00.061019 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.060986 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:54:00.092620 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.092582 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-94fd9d49d-cx8jr"] Sep 11 18:54:00.093116 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.093098 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da2e4010-abdd-4dd4-a60b-38c30195f4e5" containerName="authorino" Sep 11 18:54:00.093170 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.093120 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="da2e4010-abdd-4dd4-a60b-38c30195f4e5" containerName="authorino" Sep 11 18:54:00.093241 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.093225 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="da2e4010-abdd-4dd4-a60b-38c30195f4e5" containerName="authorino" Sep 11 18:54:00.098177 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.098152 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" Sep 11 18:54:00.101650 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.101620 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 11 18:54:00.101786 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.101704 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-q29sp\"" Sep 11 18:54:00.101881 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.101863 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 11 18:54:00.102000 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.101979 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 11 18:54:00.111663 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.111625 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-94fd9d49d-cx8jr"] Sep 11 18:54:00.165315 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.165274 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmhqb\" (UniqueName: \"kubernetes.io/projected/8f1ea416-51ed-404b-a143-84230be0d622-kube-api-access-tmhqb\") pod \"maas-controller-94fd9d49d-cx8jr\" (UID: \"8f1ea416-51ed-404b-a143-84230be0d622\") " pod="opendatahub/maas-controller-94fd9d49d-cx8jr" Sep 11 18:54:00.165539 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.165364 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f1ea416-51ed-404b-a143-84230be0d622-metrics-certs\") pod \"maas-controller-94fd9d49d-cx8jr\" (UID: \"8f1ea416-51ed-404b-a143-84230be0d622\") " pod="opendatahub/maas-controller-94fd9d49d-cx8jr" Sep 11 18:54:00.165539 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.165394 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f1ea416-51ed-404b-a143-84230be0d622-webhook-cert\") pod \"maas-controller-94fd9d49d-cx8jr\" (UID: \"8f1ea416-51ed-404b-a143-84230be0d622\") " pod="opendatahub/maas-controller-94fd9d49d-cx8jr" Sep 11 18:54:00.166540 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.166519 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:54:00.266790 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.266758 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmhqb\" (UniqueName: \"kubernetes.io/projected/8f1ea416-51ed-404b-a143-84230be0d622-kube-api-access-tmhqb\") pod \"maas-controller-94fd9d49d-cx8jr\" (UID: \"8f1ea416-51ed-404b-a143-84230be0d622\") " pod="opendatahub/maas-controller-94fd9d49d-cx8jr" Sep 11 18:54:00.266992 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.266829 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f1ea416-51ed-404b-a143-84230be0d622-metrics-certs\") pod \"maas-controller-94fd9d49d-cx8jr\" (UID: \"8f1ea416-51ed-404b-a143-84230be0d622\") " pod="opendatahub/maas-controller-94fd9d49d-cx8jr" Sep 11 18:54:00.266992 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.266856 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f1ea416-51ed-404b-a143-84230be0d622-webhook-cert\") pod \"maas-controller-94fd9d49d-cx8jr\" (UID: \"8f1ea416-51ed-404b-a143-84230be0d622\") " pod="opendatahub/maas-controller-94fd9d49d-cx8jr" Sep 11 18:54:00.266992 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:54:00.266969 2563 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Sep 11 18:54:00.267161 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:54:00.267037 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f1ea416-51ed-404b-a143-84230be0d622-webhook-cert podName:8f1ea416-51ed-404b-a143-84230be0d622 nodeName:}" failed. No retries permitted until 2026-09-11 18:54:00.767017818 +0000 UTC m=+562.229623832 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/8f1ea416-51ed-404b-a143-84230be0d622-webhook-cert") pod "maas-controller-94fd9d49d-cx8jr" (UID: "8f1ea416-51ed-404b-a143-84230be0d622") : secret "maas-controller-webhook-cert" not found Sep 11 18:54:00.269296 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.269267 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f1ea416-51ed-404b-a143-84230be0d622-metrics-certs\") pod \"maas-controller-94fd9d49d-cx8jr\" (UID: \"8f1ea416-51ed-404b-a143-84230be0d622\") " pod="opendatahub/maas-controller-94fd9d49d-cx8jr" Sep 11 18:54:00.273845 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.273822 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:54:00.281521 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.281475 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmhqb\" (UniqueName: \"kubernetes.io/projected/8f1ea416-51ed-404b-a143-84230be0d622-kube-api-access-tmhqb\") pod \"maas-controller-94fd9d49d-cx8jr\" (UID: \"8f1ea416-51ed-404b-a143-84230be0d622\") " pod="opendatahub/maas-controller-94fd9d49d-cx8jr" Sep 11 18:54:00.290631 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.290607 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c69d6b5d-fknnq"] Sep 11 18:54:00.294063 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.294043 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" Sep 11 18:54:00.319493 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.319424 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c69d6b5d-fknnq"] Sep 11 18:54:00.368683 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.368645 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7-webhook-cert\") pod \"maas-controller-6c69d6b5d-fknnq\" (UID: \"5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7\") " pod="opendatahub/maas-controller-6c69d6b5d-fknnq" Sep 11 18:54:00.369046 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.368703 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7-metrics-certs\") pod \"maas-controller-6c69d6b5d-fknnq\" (UID: \"5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7\") " pod="opendatahub/maas-controller-6c69d6b5d-fknnq" Sep 11 18:54:00.369046 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.368803 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kwvg\" (UniqueName: \"kubernetes.io/projected/5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7-kube-api-access-8kwvg\") pod \"maas-controller-6c69d6b5d-fknnq\" (UID: \"5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7\") " pod="opendatahub/maas-controller-6c69d6b5d-fknnq" Sep 11 18:54:00.412307 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.412262 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:54:00.469629 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.469599 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7-webhook-cert\") pod \"maas-controller-6c69d6b5d-fknnq\" (UID: \"5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7\") " pod="opendatahub/maas-controller-6c69d6b5d-fknnq" Sep 11 18:54:00.469820 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.469650 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7-metrics-certs\") pod \"maas-controller-6c69d6b5d-fknnq\" (UID: \"5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7\") " pod="opendatahub/maas-controller-6c69d6b5d-fknnq" Sep 11 18:54:00.469820 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.469687 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kwvg\" (UniqueName: \"kubernetes.io/projected/5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7-kube-api-access-8kwvg\") pod \"maas-controller-6c69d6b5d-fknnq\" (UID: \"5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7\") " pod="opendatahub/maas-controller-6c69d6b5d-fknnq" Sep 11 18:54:00.469820 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:54:00.469766 2563 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Sep 11 18:54:00.469996 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:54:00.469843 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7-webhook-cert podName:5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7 nodeName:}" failed. No retries permitted until 2026-09-11 18:54:00.969822365 +0000 UTC m=+562.432428376 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7-webhook-cert") pod "maas-controller-6c69d6b5d-fknnq" (UID: "5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7") : secret "maas-controller-webhook-cert" not found Sep 11 18:54:00.472275 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.472246 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7-metrics-certs\") pod \"maas-controller-6c69d6b5d-fknnq\" (UID: \"5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7\") " pod="opendatahub/maas-controller-6c69d6b5d-fknnq" Sep 11 18:54:00.502240 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.502199 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kwvg\" (UniqueName: \"kubernetes.io/projected/5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7-kube-api-access-8kwvg\") pod \"maas-controller-6c69d6b5d-fknnq\" (UID: \"5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7\") " pod="opendatahub/maas-controller-6c69d6b5d-fknnq" Sep 11 18:54:00.571333 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.571238 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:54:00.773094 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.773057 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f1ea416-51ed-404b-a143-84230be0d622-webhook-cert\") pod \"maas-controller-94fd9d49d-cx8jr\" (UID: \"8f1ea416-51ed-404b-a143-84230be0d622\") " pod="opendatahub/maas-controller-94fd9d49d-cx8jr" Sep 11 18:54:00.775593 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.775565 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f1ea416-51ed-404b-a143-84230be0d622-webhook-cert\") pod \"maas-controller-94fd9d49d-cx8jr\" (UID: \"8f1ea416-51ed-404b-a143-84230be0d622\") " pod="opendatahub/maas-controller-94fd9d49d-cx8jr" Sep 11 18:54:00.975289 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.975189 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7-webhook-cert\") pod \"maas-controller-6c69d6b5d-fknnq\" (UID: \"5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7\") " pod="opendatahub/maas-controller-6c69d6b5d-fknnq" Sep 11 18:54:00.977559 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:00.977526 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7-webhook-cert\") pod \"maas-controller-6c69d6b5d-fknnq\" (UID: \"5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7\") " pod="opendatahub/maas-controller-6c69d6b5d-fknnq" Sep 11 18:54:01.012989 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:01.012954 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" Sep 11 18:54:01.204298 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:01.204248 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" Sep 11 18:54:01.303805 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:01.303774 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-94fd9d49d-cx8jr"] Sep 11 18:54:01.305250 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:54:01.305211 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f1ea416_51ed_404b_a143_84230be0d622.slice/crio-442f9b15d508cdfb3b52a6639a6158b3d3d214b9e3a2ba291747278f605154c1 WatchSource:0}: Error finding container 442f9b15d508cdfb3b52a6639a6158b3d3d214b9e3a2ba291747278f605154c1: Status 404 returned error can't find the container with id 442f9b15d508cdfb3b52a6639a6158b3d3d214b9e3a2ba291747278f605154c1 Sep 11 18:54:01.483793 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:01.483766 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c69d6b5d-fknnq"] Sep 11 18:54:01.485646 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:54:01.485615 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f69ed98_c1c8_449f_a7b1_bc7fce88d6e7.slice/crio-bb3bd583f9af3ab147bbc59401470ad96f6830be12f257894c2bcae24782f69b WatchSource:0}: Error finding container bb3bd583f9af3ab147bbc59401470ad96f6830be12f257894c2bcae24782f69b: Status 404 returned error can't find the container with id bb3bd583f9af3ab147bbc59401470ad96f6830be12f257894c2bcae24782f69b Sep 11 18:54:01.538201 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:01.538166 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" event={"ID":"8f1ea416-51ed-404b-a143-84230be0d622","Type":"ContainerStarted","Data":"442f9b15d508cdfb3b52a6639a6158b3d3d214b9e3a2ba291747278f605154c1"} Sep 11 18:54:01.539290 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:01.539260 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" event={"ID":"5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7","Type":"ContainerStarted","Data":"bb3bd583f9af3ab147bbc59401470ad96f6830be12f257894c2bcae24782f69b"} Sep 11 18:54:03.551983 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:03.551944 2563 generic.go:356] "Generic (PLEG): container finished" podID="8f1ea416-51ed-404b-a143-84230be0d622" containerID="7374c4b26ab281436eaeca2e431773276660b24d7a91cddca245ee63655693d3" exitCode=2 Sep 11 18:54:03.552401 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:03.552028 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" event={"ID":"8f1ea416-51ed-404b-a143-84230be0d622","Type":"ContainerDied","Data":"7374c4b26ab281436eaeca2e431773276660b24d7a91cddca245ee63655693d3"} Sep 11 18:54:03.552401 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:03.552258 2563 scope.go:117] "RemoveContainer" containerID="7374c4b26ab281436eaeca2e431773276660b24d7a91cddca245ee63655693d3" Sep 11 18:54:03.553620 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:03.553601 2563 generic.go:356] "Generic (PLEG): container finished" podID="5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7" containerID="1a2b58815e954905e6d669a77f89ee169c75eab891cafab78347ddbb397ebc1b" exitCode=2 Sep 11 18:54:03.553733 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:03.553656 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" event={"ID":"5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7","Type":"ContainerDied","Data":"1a2b58815e954905e6d669a77f89ee169c75eab891cafab78347ddbb397ebc1b"} Sep 11 18:54:03.553886 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:03.553869 2563 scope.go:117] "RemoveContainer" containerID="1a2b58815e954905e6d669a77f89ee169c75eab891cafab78347ddbb397ebc1b" Sep 11 18:54:04.559901 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:04.559870 2563 generic.go:356] "Generic (PLEG): container finished" podID="5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7" containerID="c7dd089d3bf1832fecc1582863025b78eadb99a1beea845a7850161f97f577b1" exitCode=2 Sep 11 18:54:04.560351 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:04.559968 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" event={"ID":"5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7","Type":"ContainerDied","Data":"c7dd089d3bf1832fecc1582863025b78eadb99a1beea845a7850161f97f577b1"} Sep 11 18:54:04.560351 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:04.560043 2563 scope.go:117] "RemoveContainer" containerID="1a2b58815e954905e6d669a77f89ee169c75eab891cafab78347ddbb397ebc1b" Sep 11 18:54:04.560351 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:04.560194 2563 scope.go:117] "RemoveContainer" containerID="c7dd089d3bf1832fecc1582863025b78eadb99a1beea845a7850161f97f577b1" Sep 11 18:54:04.560523 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:54:04.560399 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=maas-controller-6c69d6b5d-fknnq_opendatahub(5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7)\"" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" podUID="5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7" Sep 11 18:54:04.561888 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:04.561866 2563 generic.go:356] "Generic (PLEG): container finished" podID="8f1ea416-51ed-404b-a143-84230be0d622" containerID="68e5d0eccefa7e32e06a27125b05690fe151b7d852ceacdf86d0a14ab53afc34" exitCode=2 Sep 11 18:54:04.561993 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:04.561893 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" event={"ID":"8f1ea416-51ed-404b-a143-84230be0d622","Type":"ContainerDied","Data":"68e5d0eccefa7e32e06a27125b05690fe151b7d852ceacdf86d0a14ab53afc34"} Sep 11 18:54:04.562131 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:04.562118 2563 scope.go:117] "RemoveContainer" containerID="68e5d0eccefa7e32e06a27125b05690fe151b7d852ceacdf86d0a14ab53afc34" Sep 11 18:54:04.562289 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:54:04.562273 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=maas-controller-94fd9d49d-cx8jr_opendatahub(8f1ea416-51ed-404b-a143-84230be0d622)\"" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" podUID="8f1ea416-51ed-404b-a143-84230be0d622" Sep 11 18:54:04.571185 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:04.571158 2563 scope.go:117] "RemoveContainer" containerID="7374c4b26ab281436eaeca2e431773276660b24d7a91cddca245ee63655693d3" Sep 11 18:54:05.568096 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:05.568064 2563 scope.go:117] "RemoveContainer" containerID="c7dd089d3bf1832fecc1582863025b78eadb99a1beea845a7850161f97f577b1" Sep 11 18:54:05.568556 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:54:05.568265 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=maas-controller-6c69d6b5d-fknnq_opendatahub(5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7)\"" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" podUID="5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7" Sep 11 18:54:05.569597 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:05.569581 2563 scope.go:117] "RemoveContainer" containerID="68e5d0eccefa7e32e06a27125b05690fe151b7d852ceacdf86d0a14ab53afc34" Sep 11 18:54:05.569736 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:54:05.569720 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=maas-controller-94fd9d49d-cx8jr_opendatahub(8f1ea416-51ed-404b-a143-84230be0d622)\"" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" podUID="8f1ea416-51ed-404b-a143-84230be0d622" Sep 11 18:54:06.415917 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:06.415854 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:54:11.013357 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:11.013322 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" Sep 11 18:54:11.013751 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:11.013734 2563 scope.go:117] "RemoveContainer" containerID="68e5d0eccefa7e32e06a27125b05690fe151b7d852ceacdf86d0a14ab53afc34" Sep 11 18:54:11.013923 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:54:11.013902 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=maas-controller-94fd9d49d-cx8jr_opendatahub(8f1ea416-51ed-404b-a143-84230be0d622)\"" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" podUID="8f1ea416-51ed-404b-a143-84230be0d622" Sep 11 18:54:11.204437 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:11.204405 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" Sep 11 18:54:11.204805 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:11.204790 2563 scope.go:117] "RemoveContainer" containerID="c7dd089d3bf1832fecc1582863025b78eadb99a1beea845a7850161f97f577b1" Sep 11 18:54:11.204989 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:54:11.204971 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=maas-controller-6c69d6b5d-fknnq_opendatahub(5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7)\"" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" podUID="5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7" Sep 11 18:54:18.035575 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:18.035539 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:54:21.013126 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:21.013082 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" Sep 11 18:54:21.013644 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:21.013483 2563 scope.go:117] "RemoveContainer" containerID="68e5d0eccefa7e32e06a27125b05690fe151b7d852ceacdf86d0a14ab53afc34" Sep 11 18:54:21.205052 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:21.205005 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" Sep 11 18:54:21.205450 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:21.205435 2563 scope.go:117] "RemoveContainer" containerID="c7dd089d3bf1832fecc1582863025b78eadb99a1beea845a7850161f97f577b1" Sep 11 18:54:21.644308 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:21.644270 2563 generic.go:356] "Generic (PLEG): container finished" podID="5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7" containerID="77ae5cd61c9356db5c2451a149654ad9308e9cad86231d76e153f637346c6a69" exitCode=2 Sep 11 18:54:21.644489 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:21.644339 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" event={"ID":"5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7","Type":"ContainerDied","Data":"77ae5cd61c9356db5c2451a149654ad9308e9cad86231d76e153f637346c6a69"} Sep 11 18:54:21.644489 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:21.644406 2563 scope.go:117] "RemoveContainer" containerID="c7dd089d3bf1832fecc1582863025b78eadb99a1beea845a7850161f97f577b1" Sep 11 18:54:21.644811 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:21.644784 2563 scope.go:117] "RemoveContainer" containerID="77ae5cd61c9356db5c2451a149654ad9308e9cad86231d76e153f637346c6a69" Sep 11 18:54:21.645056 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:54:21.645033 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 20s restarting failed container=manager pod=maas-controller-6c69d6b5d-fknnq_opendatahub(5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7)\"" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" podUID="5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7" Sep 11 18:54:21.646333 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:21.646312 2563 generic.go:356] "Generic (PLEG): container finished" podID="8f1ea416-51ed-404b-a143-84230be0d622" containerID="ba6a19f8bb56214fd20a207626f250e2bab448a4622e340087b6d05481f6739f" exitCode=2 Sep 11 18:54:21.646413 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:21.646382 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" event={"ID":"8f1ea416-51ed-404b-a143-84230be0d622","Type":"ContainerDied","Data":"ba6a19f8bb56214fd20a207626f250e2bab448a4622e340087b6d05481f6739f"} Sep 11 18:54:21.646684 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:21.646670 2563 scope.go:117] "RemoveContainer" containerID="ba6a19f8bb56214fd20a207626f250e2bab448a4622e340087b6d05481f6739f" Sep 11 18:54:21.646829 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:54:21.646813 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 20s restarting failed container=manager pod=maas-controller-94fd9d49d-cx8jr_opendatahub(8f1ea416-51ed-404b-a143-84230be0d622)\"" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" podUID="8f1ea416-51ed-404b-a143-84230be0d622" Sep 11 18:54:21.655163 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:21.655147 2563 scope.go:117] "RemoveContainer" containerID="68e5d0eccefa7e32e06a27125b05690fe151b7d852ceacdf86d0a14ab53afc34" Sep 11 18:54:28.682558 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:28.682519 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:54:31.013625 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:31.013584 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" Sep 11 18:54:31.014042 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:31.013990 2563 scope.go:117] "RemoveContainer" containerID="ba6a19f8bb56214fd20a207626f250e2bab448a4622e340087b6d05481f6739f" Sep 11 18:54:31.014183 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:54:31.014161 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 20s restarting failed container=manager pod=maas-controller-94fd9d49d-cx8jr_opendatahub(8f1ea416-51ed-404b-a143-84230be0d622)\"" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" podUID="8f1ea416-51ed-404b-a143-84230be0d622" Sep 11 18:54:31.205009 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:31.204960 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" Sep 11 18:54:31.205368 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:31.205354 2563 scope.go:117] "RemoveContainer" containerID="77ae5cd61c9356db5c2451a149654ad9308e9cad86231d76e153f637346c6a69" Sep 11 18:54:31.205613 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:54:31.205593 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 20s restarting failed container=manager pod=maas-controller-6c69d6b5d-fknnq_opendatahub(5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7)\"" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" podUID="5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7" Sep 11 18:54:35.420002 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:35.419971 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:54:35.520608 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:35.520574 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:54:39.165596 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:39.165556 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:54:41.013118 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:41.013072 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" Sep 11 18:54:41.013652 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:41.013631 2563 scope.go:117] "RemoveContainer" containerID="ba6a19f8bb56214fd20a207626f250e2bab448a4622e340087b6d05481f6739f" Sep 11 18:54:41.013875 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:54:41.013851 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 20s restarting failed container=manager pod=maas-controller-94fd9d49d-cx8jr_opendatahub(8f1ea416-51ed-404b-a143-84230be0d622)\"" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" podUID="8f1ea416-51ed-404b-a143-84230be0d622" Sep 11 18:54:41.204476 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:41.204438 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" Sep 11 18:54:41.204949 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:41.204929 2563 scope.go:117] "RemoveContainer" containerID="77ae5cd61c9356db5c2451a149654ad9308e9cad86231d76e153f637346c6a69" Sep 11 18:54:41.205165 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:54:41.205142 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 20s restarting failed container=manager pod=maas-controller-6c69d6b5d-fknnq_opendatahub(5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7)\"" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" podUID="5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7" Sep 11 18:54:55.055696 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:55.055654 2563 scope.go:117] "RemoveContainer" containerID="ba6a19f8bb56214fd20a207626f250e2bab448a4622e340087b6d05481f6739f" Sep 11 18:54:55.799497 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:55.799464 2563 generic.go:356] "Generic (PLEG): container finished" podID="8f1ea416-51ed-404b-a143-84230be0d622" containerID="b3cef19954ce51f11dd20fc120f1dcfa4bc763bc8e7b11aae9bbe63060b84757" exitCode=2 Sep 11 18:54:55.799676 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:55.799543 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" event={"ID":"8f1ea416-51ed-404b-a143-84230be0d622","Type":"ContainerDied","Data":"b3cef19954ce51f11dd20fc120f1dcfa4bc763bc8e7b11aae9bbe63060b84757"} Sep 11 18:54:55.799676 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:55.799602 2563 scope.go:117] "RemoveContainer" containerID="ba6a19f8bb56214fd20a207626f250e2bab448a4622e340087b6d05481f6739f" Sep 11 18:54:55.799944 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:55.799927 2563 scope.go:117] "RemoveContainer" containerID="b3cef19954ce51f11dd20fc120f1dcfa4bc763bc8e7b11aae9bbe63060b84757" Sep 11 18:54:55.800122 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:54:55.800103 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=maas-controller-94fd9d49d-cx8jr_opendatahub(8f1ea416-51ed-404b-a143-84230be0d622)\"" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" podUID="8f1ea416-51ed-404b-a143-84230be0d622" Sep 11 18:54:57.055247 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:57.055208 2563 scope.go:117] "RemoveContainer" containerID="77ae5cd61c9356db5c2451a149654ad9308e9cad86231d76e153f637346c6a69" Sep 11 18:54:57.812433 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:57.812400 2563 generic.go:356] "Generic (PLEG): container finished" podID="5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7" containerID="2f542b4aa3fad39eadf490222f9107532b12e338e96c173d8f6329220f7fb5c7" exitCode=2 Sep 11 18:54:57.812615 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:57.812444 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" event={"ID":"5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7","Type":"ContainerDied","Data":"2f542b4aa3fad39eadf490222f9107532b12e338e96c173d8f6329220f7fb5c7"} Sep 11 18:54:57.812615 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:57.812531 2563 scope.go:117] "RemoveContainer" containerID="77ae5cd61c9356db5c2451a149654ad9308e9cad86231d76e153f637346c6a69" Sep 11 18:54:57.812934 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:54:57.812914 2563 scope.go:117] "RemoveContainer" containerID="2f542b4aa3fad39eadf490222f9107532b12e338e96c173d8f6329220f7fb5c7" Sep 11 18:54:57.813149 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:54:57.813130 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=maas-controller-6c69d6b5d-fknnq_opendatahub(5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7)\"" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" podUID="5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7" Sep 11 18:55:00.078937 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:00.078904 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:55:01.013119 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:01.013081 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" Sep 11 18:55:01.013295 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:01.013138 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" Sep 11 18:55:01.013452 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:01.013439 2563 scope.go:117] "RemoveContainer" containerID="b3cef19954ce51f11dd20fc120f1dcfa4bc763bc8e7b11aae9bbe63060b84757" Sep 11 18:55:01.013642 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:55:01.013624 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=maas-controller-94fd9d49d-cx8jr_opendatahub(8f1ea416-51ed-404b-a143-84230be0d622)\"" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" podUID="8f1ea416-51ed-404b-a143-84230be0d622" Sep 11 18:55:01.204570 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:01.204468 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" Sep 11 18:55:01.204570 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:01.204554 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" Sep 11 18:55:01.204998 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:01.204873 2563 scope.go:117] "RemoveContainer" containerID="2f542b4aa3fad39eadf490222f9107532b12e338e96c173d8f6329220f7fb5c7" Sep 11 18:55:01.205106 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:55:01.205082 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=maas-controller-6c69d6b5d-fknnq_opendatahub(5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7)\"" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" podUID="5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7" Sep 11 18:55:12.054920 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:12.054886 2563 scope.go:117] "RemoveContainer" containerID="b3cef19954ce51f11dd20fc120f1dcfa4bc763bc8e7b11aae9bbe63060b84757" Sep 11 18:55:12.055321 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:55:12.055067 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=maas-controller-94fd9d49d-cx8jr_opendatahub(8f1ea416-51ed-404b-a143-84230be0d622)\"" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" podUID="8f1ea416-51ed-404b-a143-84230be0d622" Sep 11 18:55:13.055322 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:13.055292 2563 scope.go:117] "RemoveContainer" containerID="2f542b4aa3fad39eadf490222f9107532b12e338e96c173d8f6329220f7fb5c7" Sep 11 18:55:13.055727 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:55:13.055457 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=maas-controller-6c69d6b5d-fknnq_opendatahub(5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7)\"" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" podUID="5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7" Sep 11 18:55:25.055285 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:25.055252 2563 scope.go:117] "RemoveContainer" containerID="b3cef19954ce51f11dd20fc120f1dcfa4bc763bc8e7b11aae9bbe63060b84757" Sep 11 18:55:25.055713 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:55:25.055426 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=maas-controller-94fd9d49d-cx8jr_opendatahub(8f1ea416-51ed-404b-a143-84230be0d622)\"" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" podUID="8f1ea416-51ed-404b-a143-84230be0d622" Sep 11 18:55:26.977264 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:26.977234 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:55:27.055625 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:27.055596 2563 scope.go:117] "RemoveContainer" containerID="2f542b4aa3fad39eadf490222f9107532b12e338e96c173d8f6329220f7fb5c7" Sep 11 18:55:27.055773 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:55:27.055756 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=maas-controller-6c69d6b5d-fknnq_opendatahub(5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7)\"" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" podUID="5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7" Sep 11 18:55:27.062634 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:27.062600 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:55:37.055613 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:37.055578 2563 scope.go:117] "RemoveContainer" containerID="b3cef19954ce51f11dd20fc120f1dcfa4bc763bc8e7b11aae9bbe63060b84757" Sep 11 18:55:37.056015 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:37.055880 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 11 18:55:37.992625 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:37.992590 2563 generic.go:356] "Generic (PLEG): container finished" podID="8f1ea416-51ed-404b-a143-84230be0d622" containerID="b435a6dc3bcc3b15db2bca18177f49afce6747b1f3b7e83f7c3a5b4601499a59" exitCode=2 Sep 11 18:55:37.992828 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:37.992671 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" event={"ID":"8f1ea416-51ed-404b-a143-84230be0d622","Type":"ContainerDied","Data":"b435a6dc3bcc3b15db2bca18177f49afce6747b1f3b7e83f7c3a5b4601499a59"} Sep 11 18:55:37.992828 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:37.992744 2563 scope.go:117] "RemoveContainer" containerID="b3cef19954ce51f11dd20fc120f1dcfa4bc763bc8e7b11aae9bbe63060b84757" Sep 11 18:55:37.993090 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:37.993066 2563 scope.go:117] "RemoveContainer" containerID="b435a6dc3bcc3b15db2bca18177f49afce6747b1f3b7e83f7c3a5b4601499a59" Sep 11 18:55:37.993270 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:55:37.993252 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=maas-controller-94fd9d49d-cx8jr_opendatahub(8f1ea416-51ed-404b-a143-84230be0d622)\"" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" podUID="8f1ea416-51ed-404b-a143-84230be0d622" Sep 11 18:55:40.055730 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:40.055692 2563 scope.go:117] "RemoveContainer" containerID="2f542b4aa3fad39eadf490222f9107532b12e338e96c173d8f6329220f7fb5c7" Sep 11 18:55:41.010920 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:41.010885 2563 generic.go:356] "Generic (PLEG): container finished" podID="5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7" containerID="b67de2afe1eba2d5c9b8c8898ffc81f2575d0a64b71546f6fa922ad73da4f69e" exitCode=2 Sep 11 18:55:41.011121 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:41.010963 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" event={"ID":"5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7","Type":"ContainerDied","Data":"b67de2afe1eba2d5c9b8c8898ffc81f2575d0a64b71546f6fa922ad73da4f69e"} Sep 11 18:55:41.011121 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:41.011035 2563 scope.go:117] "RemoveContainer" containerID="2f542b4aa3fad39eadf490222f9107532b12e338e96c173d8f6329220f7fb5c7" Sep 11 18:55:41.011439 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:41.011424 2563 scope.go:117] "RemoveContainer" containerID="b67de2afe1eba2d5c9b8c8898ffc81f2575d0a64b71546f6fa922ad73da4f69e" Sep 11 18:55:41.011673 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:55:41.011655 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=maas-controller-6c69d6b5d-fknnq_opendatahub(5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7)\"" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" podUID="5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7" Sep 11 18:55:41.013081 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:41.013064 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" Sep 11 18:55:41.013183 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:41.013108 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" Sep 11 18:55:41.013428 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:41.013412 2563 scope.go:117] "RemoveContainer" containerID="b435a6dc3bcc3b15db2bca18177f49afce6747b1f3b7e83f7c3a5b4601499a59" Sep 11 18:55:41.013654 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:55:41.013631 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=maas-controller-94fd9d49d-cx8jr_opendatahub(8f1ea416-51ed-404b-a143-84230be0d622)\"" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" podUID="8f1ea416-51ed-404b-a143-84230be0d622" Sep 11 18:55:41.205174 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:41.205133 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" Sep 11 18:55:41.205555 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:41.205201 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" Sep 11 18:55:42.017400 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:42.017375 2563 scope.go:117] "RemoveContainer" containerID="b67de2afe1eba2d5c9b8c8898ffc81f2575d0a64b71546f6fa922ad73da4f69e" Sep 11 18:55:42.017599 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:55:42.017579 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=maas-controller-6c69d6b5d-fknnq_opendatahub(5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7)\"" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" podUID="5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7" Sep 11 18:55:56.055334 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:56.055302 2563 scope.go:117] "RemoveContainer" containerID="b435a6dc3bcc3b15db2bca18177f49afce6747b1f3b7e83f7c3a5b4601499a59" Sep 11 18:55:56.055741 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:55:56.055457 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=maas-controller-94fd9d49d-cx8jr_opendatahub(8f1ea416-51ed-404b-a143-84230be0d622)\"" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" podUID="8f1ea416-51ed-404b-a143-84230be0d622" Sep 11 18:55:57.055128 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:55:57.055095 2563 scope.go:117] "RemoveContainer" containerID="b67de2afe1eba2d5c9b8c8898ffc81f2575d0a64b71546f6fa922ad73da4f69e" Sep 11 18:55:57.055352 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:55:57.055329 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=maas-controller-6c69d6b5d-fknnq_opendatahub(5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7)\"" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" podUID="5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7" Sep 11 18:56:06.131207 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:06.131165 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-qq4sv_f5392b9a-3f94-4e11-a551-6944660149a6/manager/0.log" Sep 11 18:56:06.245426 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:06.245373 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-c7vwr_c47a4a2a-320c-467a-8704-6110b9c097b5/manager/0.log" Sep 11 18:56:06.357143 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:06.357112 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6c69d6b5d-fknnq_5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7/manager/4.log" Sep 11 18:56:06.467941 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:06.467847 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-94fd9d49d-cx8jr_8f1ea416-51ed-404b-a143-84230be0d622/manager/4.log" Sep 11 18:56:06.578755 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:06.578733 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-mpxv2_eb78723e-f54e-438c-8328-759456ee91f1/server/0.log" Sep 11 18:56:06.697838 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:06.697806 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-chgzp_1cca8dee-f089-4c64-9838-e721507b2569/manager/0.log" Sep 11 18:56:06.813269 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:06.813239 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld_4f288658-5f78-4b23-a916-6121f1793fb3/copy-manifests/0.log" Sep 11 18:56:06.821916 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:06.821869 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld_4f288658-5f78-4b23-a916-6121f1793fb3/manager/0.log" Sep 11 18:56:07.054803 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:07.054774 2563 scope.go:117] "RemoveContainer" containerID="b435a6dc3bcc3b15db2bca18177f49afce6747b1f3b7e83f7c3a5b4601499a59" Sep 11 18:56:07.054975 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:56:07.054956 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=maas-controller-94fd9d49d-cx8jr_opendatahub(8f1ea416-51ed-404b-a143-84230be0d622)\"" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" podUID="8f1ea416-51ed-404b-a143-84230be0d622" Sep 11 18:56:07.420975 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:07.420942 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-6mnj6_423aa3c2-859e-4ef9-aeaf-e8e543ddb088/postgres/0.log" Sep 11 18:56:08.189821 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:08.189793 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-mvv7c_55149df4-4fa7-4e56-9913-a4ef3838643d/manager/0.log" Sep 11 18:56:08.301475 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:08.301447 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-625tz_254a47f5-19ea-4fd5-84d7-29427b749b9d/manager/0.log" Sep 11 18:56:08.754068 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:08.754038 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-kdgpx_79f2446b-9ff0-44cc-8aff-24ee7f116311/manager/0.log" Sep 11 18:56:09.105098 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:09.105068 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk_6e094fe2-8167-4db6-aa36-25f565b07177/istio-proxy/0.log" Sep 11 18:56:09.338007 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:09.337971 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cdff97488-84z2c_e78e9ec5-2629-4097-a4a7-ee2581075ad9/kube-auth-proxy/0.log" Sep 11 18:56:09.564600 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:09.564568 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-klg7x_0fa085de-dbaf-4ead-9545-3e80fdd52307/istio-proxy/0.log" Sep 11 18:56:12.055303 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:12.055269 2563 scope.go:117] "RemoveContainer" containerID="b67de2afe1eba2d5c9b8c8898ffc81f2575d0a64b71546f6fa922ad73da4f69e" Sep 11 18:56:12.055766 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:56:12.055542 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=maas-controller-6c69d6b5d-fknnq_opendatahub(5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7)\"" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" podUID="5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7" Sep 11 18:56:14.930670 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:14.930635 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fztb9/must-gather-6wtv9"] Sep 11 18:56:14.933871 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:14.933849 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fztb9/must-gather-6wtv9" Sep 11 18:56:14.936546 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:14.936491 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fztb9\"/\"openshift-service-ca.crt\"" Sep 11 18:56:14.936664 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:14.936546 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fztb9\"/\"kube-root-ca.crt\"" Sep 11 18:56:14.936731 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:14.936681 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fztb9\"/\"default-dockercfg-q2ncv\"" Sep 11 18:56:14.942993 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:14.942972 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fztb9/must-gather-6wtv9"] Sep 11 18:56:14.955140 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:14.955113 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:56:15.027051 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:15.027021 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:56:15.030391 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:15.030368 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9e665a00-656e-4369-9df4-36e764cf837d-must-gather-output\") pod \"must-gather-6wtv9\" (UID: \"9e665a00-656e-4369-9df4-36e764cf837d\") " pod="openshift-must-gather-fztb9/must-gather-6wtv9" Sep 11 18:56:15.030455 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:15.030412 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfk8k\" (UniqueName: \"kubernetes.io/projected/9e665a00-656e-4369-9df4-36e764cf837d-kube-api-access-cfk8k\") pod \"must-gather-6wtv9\" (UID: \"9e665a00-656e-4369-9df4-36e764cf837d\") " pod="openshift-must-gather-fztb9/must-gather-6wtv9" Sep 11 18:56:15.130964 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:15.130926 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9e665a00-656e-4369-9df4-36e764cf837d-must-gather-output\") pod \"must-gather-6wtv9\" (UID: \"9e665a00-656e-4369-9df4-36e764cf837d\") " pod="openshift-must-gather-fztb9/must-gather-6wtv9" Sep 11 18:56:15.131137 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:15.131008 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfk8k\" (UniqueName: \"kubernetes.io/projected/9e665a00-656e-4369-9df4-36e764cf837d-kube-api-access-cfk8k\") pod \"must-gather-6wtv9\" (UID: \"9e665a00-656e-4369-9df4-36e764cf837d\") " pod="openshift-must-gather-fztb9/must-gather-6wtv9" Sep 11 18:56:15.131255 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:15.131235 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9e665a00-656e-4369-9df4-36e764cf837d-must-gather-output\") pod \"must-gather-6wtv9\" (UID: \"9e665a00-656e-4369-9df4-36e764cf837d\") " pod="openshift-must-gather-fztb9/must-gather-6wtv9" Sep 11 18:56:15.144180 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:15.144145 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfk8k\" (UniqueName: \"kubernetes.io/projected/9e665a00-656e-4369-9df4-36e764cf837d-kube-api-access-cfk8k\") pod \"must-gather-6wtv9\" (UID: \"9e665a00-656e-4369-9df4-36e764cf837d\") " pod="openshift-must-gather-fztb9/must-gather-6wtv9" Sep 11 18:56:15.243078 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:15.242989 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fztb9/must-gather-6wtv9" Sep 11 18:56:15.511766 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:15.511741 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fztb9/must-gather-6wtv9"] Sep 11 18:56:15.514468 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:56:15.514437 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e665a00_656e_4369_9df4_36e764cf837d.slice/crio-e6e9767814092ec2a7309bdac6b3c9e0629de757207fe978c7340dd4ec672fc8 WatchSource:0}: Error finding container e6e9767814092ec2a7309bdac6b3c9e0629de757207fe978c7340dd4ec672fc8: Status 404 returned error can't find the container with id e6e9767814092ec2a7309bdac6b3c9e0629de757207fe978c7340dd4ec672fc8 Sep 11 18:56:16.173770 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:16.173739 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fztb9/must-gather-6wtv9" event={"ID":"9e665a00-656e-4369-9df4-36e764cf837d","Type":"ContainerStarted","Data":"e6e9767814092ec2a7309bdac6b3c9e0629de757207fe978c7340dd4ec672fc8"} Sep 11 18:56:17.181841 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:17.181802 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fztb9/must-gather-6wtv9" event={"ID":"9e665a00-656e-4369-9df4-36e764cf837d","Type":"ContainerStarted","Data":"a52c13c4a8ce681bcdf35099c2911b20592b73609a29609c6ac9e2325244d875"} Sep 11 18:56:17.182232 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:17.181849 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fztb9/must-gather-6wtv9" event={"ID":"9e665a00-656e-4369-9df4-36e764cf837d","Type":"ContainerStarted","Data":"77a61a2481a00abea9d94e8393c60a65e1dc9453fc78d5bb5c40b1caee29d2d4"} Sep 11 18:56:17.201358 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:17.201293 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fztb9/must-gather-6wtv9" podStartSLOduration=2.537731645 podStartE2EDuration="3.201272057s" totalImagesPullingTime="663.540412ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:56:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:56:17.199767133 +0000 UTC m=+698.662373145" watchObservedRunningTime="2026-09-11 18:56:17.201272057 +0000 UTC m=+698.663878075" Sep 11 18:56:17.268145 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:17.268112 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:56:17.334616 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:17.334580 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" limit=1024 Sep 11 18:56:17.931685 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:17.931656 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-sxqg6_b799f9cb-ce50-4731-9fbe-48f47807a053/global-pull-secret-syncer/0.log" Sep 11 18:56:17.980603 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:17.980564 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-gv5z5_22ef1e56-f6b2-4ebf-8be0-63669535237b/konnectivity-agent/0.log" Sep 11 18:56:18.054762 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:18.054733 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-192.ec2.internal_7241124c924c97f39a53ae6bf4dfdb02/haproxy/0.log" Sep 11 18:56:18.055003 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:18.054933 2563 scope.go:117] "RemoveContainer" containerID="b435a6dc3bcc3b15db2bca18177f49afce6747b1f3b7e83f7c3a5b4601499a59" Sep 11 18:56:18.055287 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:56:18.055263 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=maas-controller-94fd9d49d-cx8jr_opendatahub(8f1ea416-51ed-404b-a143-84230be0d622)\"" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" podUID="8f1ea416-51ed-404b-a143-84230be0d622" Sep 11 18:56:22.784061 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:22.783972 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-mvv7c_55149df4-4fa7-4e56-9913-a4ef3838643d/manager/0.log" Sep 11 18:56:22.818074 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:22.818032 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-625tz_254a47f5-19ea-4fd5-84d7-29427b749b9d/manager/0.log" Sep 11 18:56:22.943587 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:22.943441 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-kdgpx_79f2446b-9ff0-44cc-8aff-24ee7f116311/manager/0.log" Sep 11 18:56:24.758634 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:24.758605 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jm77n_078f13cc-8c84-4ebc-9452-d235511c38b1/node-exporter/0.log" Sep 11 18:56:24.785069 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:24.785044 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jm77n_078f13cc-8c84-4ebc-9452-d235511c38b1/kube-rbac-proxy/0.log" Sep 11 18:56:24.809289 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:24.809263 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jm77n_078f13cc-8c84-4ebc-9452-d235511c38b1/init-textfile/0.log" Sep 11 18:56:25.056476 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:25.056450 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_de371545-5c04-40e9-8047-146de0f87134/prometheus/0.log" Sep 11 18:56:25.088624 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:25.088592 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_de371545-5c04-40e9-8047-146de0f87134/config-reloader/0.log" Sep 11 18:56:25.118652 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:25.118569 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_de371545-5c04-40e9-8047-146de0f87134/thanos-sidecar/0.log" Sep 11 18:56:25.145894 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:25.145848 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_de371545-5c04-40e9-8047-146de0f87134/kube-rbac-proxy-web/0.log" Sep 11 18:56:25.170126 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:25.170096 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_de371545-5c04-40e9-8047-146de0f87134/kube-rbac-proxy/0.log" Sep 11 18:56:25.196364 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:25.196337 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_de371545-5c04-40e9-8047-146de0f87134/kube-rbac-proxy-thanos/0.log" Sep 11 18:56:25.220648 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:25.220618 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_de371545-5c04-40e9-8047-146de0f87134/init-config-reloader/0.log" Sep 11 18:56:27.055178 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:27.055149 2563 scope.go:117] "RemoveContainer" containerID="b67de2afe1eba2d5c9b8c8898ffc81f2575d0a64b71546f6fa922ad73da4f69e" Sep 11 18:56:27.056099 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:56:27.056064 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=maas-controller-6c69d6b5d-fknnq_opendatahub(5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7)\"" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" podUID="5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7" Sep 11 18:56:27.825303 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:27.825261 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fztb9/perf-node-gather-daemonset-8hz7v"] Sep 11 18:56:27.831717 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:27.831692 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fztb9/perf-node-gather-daemonset-8hz7v" Sep 11 18:56:27.837787 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:27.837352 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fztb9/perf-node-gather-daemonset-8hz7v"] Sep 11 18:56:27.878788 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:27.878733 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f62283f2-a132-4ec1-b33d-2c57217d436f-sys\") pod \"perf-node-gather-daemonset-8hz7v\" (UID: \"f62283f2-a132-4ec1-b33d-2c57217d436f\") " pod="openshift-must-gather-fztb9/perf-node-gather-daemonset-8hz7v" Sep 11 18:56:27.878975 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:27.878806 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzhrk\" (UniqueName: \"kubernetes.io/projected/f62283f2-a132-4ec1-b33d-2c57217d436f-kube-api-access-mzhrk\") pod \"perf-node-gather-daemonset-8hz7v\" (UID: \"f62283f2-a132-4ec1-b33d-2c57217d436f\") " pod="openshift-must-gather-fztb9/perf-node-gather-daemonset-8hz7v" Sep 11 18:56:27.878975 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:27.878878 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f62283f2-a132-4ec1-b33d-2c57217d436f-proc\") pod \"perf-node-gather-daemonset-8hz7v\" (UID: \"f62283f2-a132-4ec1-b33d-2c57217d436f\") " pod="openshift-must-gather-fztb9/perf-node-gather-daemonset-8hz7v" Sep 11 18:56:27.878975 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:27.878898 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f62283f2-a132-4ec1-b33d-2c57217d436f-lib-modules\") pod \"perf-node-gather-daemonset-8hz7v\" (UID: \"f62283f2-a132-4ec1-b33d-2c57217d436f\") " pod="openshift-must-gather-fztb9/perf-node-gather-daemonset-8hz7v" Sep 11 18:56:27.878975 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:27.878921 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f62283f2-a132-4ec1-b33d-2c57217d436f-podres\") pod \"perf-node-gather-daemonset-8hz7v\" (UID: \"f62283f2-a132-4ec1-b33d-2c57217d436f\") " pod="openshift-must-gather-fztb9/perf-node-gather-daemonset-8hz7v" Sep 11 18:56:27.980062 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:27.980015 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f62283f2-a132-4ec1-b33d-2c57217d436f-lib-modules\") pod \"perf-node-gather-daemonset-8hz7v\" (UID: \"f62283f2-a132-4ec1-b33d-2c57217d436f\") " pod="openshift-must-gather-fztb9/perf-node-gather-daemonset-8hz7v" Sep 11 18:56:27.980249 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:27.980072 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f62283f2-a132-4ec1-b33d-2c57217d436f-podres\") pod \"perf-node-gather-daemonset-8hz7v\" (UID: \"f62283f2-a132-4ec1-b33d-2c57217d436f\") " pod="openshift-must-gather-fztb9/perf-node-gather-daemonset-8hz7v" Sep 11 18:56:27.980249 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:27.980180 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f62283f2-a132-4ec1-b33d-2c57217d436f-sys\") pod \"perf-node-gather-daemonset-8hz7v\" (UID: \"f62283f2-a132-4ec1-b33d-2c57217d436f\") " pod="openshift-must-gather-fztb9/perf-node-gather-daemonset-8hz7v" Sep 11 18:56:27.980249 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:27.980211 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzhrk\" (UniqueName: \"kubernetes.io/projected/f62283f2-a132-4ec1-b33d-2c57217d436f-kube-api-access-mzhrk\") pod \"perf-node-gather-daemonset-8hz7v\" (UID: \"f62283f2-a132-4ec1-b33d-2c57217d436f\") " pod="openshift-must-gather-fztb9/perf-node-gather-daemonset-8hz7v" Sep 11 18:56:27.980420 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:27.980273 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f62283f2-a132-4ec1-b33d-2c57217d436f-proc\") pod \"perf-node-gather-daemonset-8hz7v\" (UID: \"f62283f2-a132-4ec1-b33d-2c57217d436f\") " pod="openshift-must-gather-fztb9/perf-node-gather-daemonset-8hz7v" Sep 11 18:56:27.980420 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:27.980379 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f62283f2-a132-4ec1-b33d-2c57217d436f-proc\") pod \"perf-node-gather-daemonset-8hz7v\" (UID: \"f62283f2-a132-4ec1-b33d-2c57217d436f\") " pod="openshift-must-gather-fztb9/perf-node-gather-daemonset-8hz7v" Sep 11 18:56:27.980559 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:27.980521 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f62283f2-a132-4ec1-b33d-2c57217d436f-lib-modules\") pod \"perf-node-gather-daemonset-8hz7v\" (UID: \"f62283f2-a132-4ec1-b33d-2c57217d436f\") " pod="openshift-must-gather-fztb9/perf-node-gather-daemonset-8hz7v" Sep 11 18:56:27.980622 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:27.980604 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f62283f2-a132-4ec1-b33d-2c57217d436f-podres\") pod \"perf-node-gather-daemonset-8hz7v\" (UID: \"f62283f2-a132-4ec1-b33d-2c57217d436f\") " pod="openshift-must-gather-fztb9/perf-node-gather-daemonset-8hz7v" Sep 11 18:56:27.980683 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:27.980654 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f62283f2-a132-4ec1-b33d-2c57217d436f-sys\") pod \"perf-node-gather-daemonset-8hz7v\" (UID: \"f62283f2-a132-4ec1-b33d-2c57217d436f\") " pod="openshift-must-gather-fztb9/perf-node-gather-daemonset-8hz7v" Sep 11 18:56:27.989292 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:27.989262 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzhrk\" (UniqueName: \"kubernetes.io/projected/f62283f2-a132-4ec1-b33d-2c57217d436f-kube-api-access-mzhrk\") pod \"perf-node-gather-daemonset-8hz7v\" (UID: \"f62283f2-a132-4ec1-b33d-2c57217d436f\") " pod="openshift-must-gather-fztb9/perf-node-gather-daemonset-8hz7v" Sep 11 18:56:28.146119 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:28.146038 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fztb9/perf-node-gather-daemonset-8hz7v" Sep 11 18:56:28.465990 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:28.465894 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fztb9/perf-node-gather-daemonset-8hz7v"] Sep 11 18:56:28.470627 ip-10-0-134-192 kubenswrapper[2563]: W0911 18:56:28.469915 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf62283f2_a132_4ec1_b33d_2c57217d436f.slice/crio-375c0ca6427089056554f21a58c3a6aee30a725981f77bfaa4d4e6786ed46351 WatchSource:0}: Error finding container 375c0ca6427089056554f21a58c3a6aee30a725981f77bfaa4d4e6786ed46351: Status 404 returned error can't find the container with id 375c0ca6427089056554f21a58c3a6aee30a725981f77bfaa4d4e6786ed46351 Sep 11 18:56:29.274562 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:29.274517 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fztb9/perf-node-gather-daemonset-8hz7v" event={"ID":"f62283f2-a132-4ec1-b33d-2c57217d436f","Type":"ContainerStarted","Data":"baf1dc52d9580226957aeb295c6351c9d15bca58d42a39b85db52004ac1205b5"} Sep 11 18:56:29.275020 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:29.274593 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fztb9/perf-node-gather-daemonset-8hz7v" event={"ID":"f62283f2-a132-4ec1-b33d-2c57217d436f","Type":"ContainerStarted","Data":"375c0ca6427089056554f21a58c3a6aee30a725981f77bfaa4d4e6786ed46351"} Sep 11 18:56:29.275020 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:29.274963 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fztb9/perf-node-gather-daemonset-8hz7v" Sep 11 18:56:29.294193 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:29.294148 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fztb9/perf-node-gather-daemonset-8hz7v" podStartSLOduration=2.294133955 podStartE2EDuration="2.294133955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:56:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:56:29.292796221 +0000 UTC m=+710.755402241" watchObservedRunningTime="2026-09-11 18:56:29.294133955 +0000 UTC m=+710.756739972" Sep 11 18:56:29.909446 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:29.909418 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dzb2p_0a1958ce-2c55-4c46-a2af-6b48ab70f9df/dns/0.log" Sep 11 18:56:29.930833 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:29.930801 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dzb2p_0a1958ce-2c55-4c46-a2af-6b48ab70f9df/kube-rbac-proxy/0.log" Sep 11 18:56:30.001624 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:30.001594 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7f246_60dd726a-4169-4a95-ade1-c067f1851aac/dns-node-resolver/0.log" Sep 11 18:56:30.626999 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:30.626973 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tnjsf_254dd733-80bd-4f2a-9fbc-a57622608b32/node-ca/0.log" Sep 11 18:56:31.621088 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:31.620995 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk_6e094fe2-8167-4db6-aa36-25f565b07177/istio-proxy/0.log" Sep 11 18:56:31.682225 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:31.682193 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cdff97488-84z2c_e78e9ec5-2629-4097-a4a7-ee2581075ad9/kube-auth-proxy/0.log" Sep 11 18:56:31.737791 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:31.737755 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-klg7x_0fa085de-dbaf-4ead-9545-3e80fdd52307/istio-proxy/0.log" Sep 11 18:56:32.054813 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:32.054786 2563 scope.go:117] "RemoveContainer" containerID="b435a6dc3bcc3b15db2bca18177f49afce6747b1f3b7e83f7c3a5b4601499a59" Sep 11 18:56:32.054977 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:56:32.054958 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=maas-controller-94fd9d49d-cx8jr_opendatahub(8f1ea416-51ed-404b-a143-84230be0d622)\"" pod="opendatahub/maas-controller-94fd9d49d-cx8jr" podUID="8f1ea416-51ed-404b-a143-84230be0d622" Sep 11 18:56:32.343995 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:32.343927 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7h9dc_7a322a44-c645-482e-811b-cae2ea119a2f/serve-healthcheck-canary/0.log" Sep 11 18:56:32.904051 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:32.904020 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-65929_1c275a87-26b0-4fd7-a4fc-8f12b496a04e/kube-rbac-proxy/0.log" Sep 11 18:56:32.927749 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:32.927709 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-65929_1c275a87-26b0-4fd7-a4fc-8f12b496a04e/exporter/0.log" Sep 11 18:56:32.951779 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:32.951752 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-65929_1c275a87-26b0-4fd7-a4fc-8f12b496a04e/extractor/0.log" Sep 11 18:56:35.221656 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:35.221628 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-qq4sv_f5392b9a-3f94-4e11-a551-6944660149a6/manager/0.log" Sep 11 18:56:35.246002 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:35.245958 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-c7vwr_c47a4a2a-320c-467a-8704-6110b9c097b5/manager/0.log" Sep 11 18:56:35.272887 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:35.272862 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6c69d6b5d-fknnq_5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7/manager/4.log" Sep 11 18:56:35.273064 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:35.272933 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6c69d6b5d-fknnq_5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7/manager/4.log" Sep 11 18:56:35.292079 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:35.292052 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fztb9/perf-node-gather-daemonset-8hz7v" Sep 11 18:56:35.295198 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:35.295157 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-94fd9d49d-cx8jr_8f1ea416-51ed-404b-a143-84230be0d622/manager/4.log" Sep 11 18:56:35.295490 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:35.295281 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-94fd9d49d-cx8jr_8f1ea416-51ed-404b-a143-84230be0d622/manager/4.log" Sep 11 18:56:35.318996 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:35.318972 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-mpxv2_eb78723e-f54e-438c-8328-759456ee91f1/server/0.log" Sep 11 18:56:35.367238 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:35.367194 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-chgzp_1cca8dee-f089-4c64-9838-e721507b2569/manager/0.log" Sep 11 18:56:35.391197 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:35.391163 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld_4f288658-5f78-4b23-a916-6121f1793fb3/manager/0.log" Sep 11 18:56:35.415341 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:35.415317 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dbc5c8f5c-hmdld_4f288658-5f78-4b23-a916-6121f1793fb3/copy-manifests/0.log" Sep 11 18:56:38.486104 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:38.485992 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk_6e094fe2-8167-4db6-aa36-25f565b07177/istio-proxy/0.log" Sep 11 18:56:38.565917 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:38.565891 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cdff97488-84z2c_e78e9ec5-2629-4097-a4a7-ee2581075ad9/kube-auth-proxy/0.log" Sep 11 18:56:38.616816 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:38.616784 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-klg7x_0fa085de-dbaf-4ead-9545-3e80fdd52307/istio-proxy/0.log" Sep 11 18:56:39.744106 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:39.744017 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-l72tk_6e094fe2-8167-4db6-aa36-25f565b07177/istio-proxy/0.log" Sep 11 18:56:39.791732 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:39.791698 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cdff97488-84z2c_e78e9ec5-2629-4097-a4a7-ee2581075ad9/kube-auth-proxy/0.log" Sep 11 18:56:39.830605 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:39.830563 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-klg7x_0fa085de-dbaf-4ead-9545-3e80fdd52307/istio-proxy/0.log" Sep 11 18:56:41.055486 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:41.055457 2563 scope.go:117] "RemoveContainer" containerID="b67de2afe1eba2d5c9b8c8898ffc81f2575d0a64b71546f6fa922ad73da4f69e" Sep 11 18:56:41.056451 ip-10-0-134-192 kubenswrapper[2563]: E0911 18:56:41.056407 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=maas-controller-6c69d6b5d-fknnq_opendatahub(5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7)\"" pod="opendatahub/maas-controller-6c69d6b5d-fknnq" podUID="5f69ed98-c1c8-449f-a7b1-bc7fce88d6e7" Sep 11 18:56:42.181738 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:42.181710 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-p5gd9_c39ed765-6eaa-420f-81e3-f47d5e972ff9/migrator/0.log" Sep 11 18:56:42.203896 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:42.203870 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-p5gd9_c39ed765-6eaa-420f-81e3-f47d5e972ff9/graceful-termination/0.log" Sep 11 18:56:43.641198 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:43.641168 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9xtl5_e9819d65-2122-497e-831c-917aa1fabd52/kube-multus-additional-cni-plugins/0.log" Sep 11 18:56:43.664954 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:43.664924 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9xtl5_e9819d65-2122-497e-831c-917aa1fabd52/egress-router-binary-copy/0.log" Sep 11 18:56:43.688518 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:43.688475 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9xtl5_e9819d65-2122-497e-831c-917aa1fabd52/cni-plugins/0.log" Sep 11 18:56:43.711423 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:43.711394 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9xtl5_e9819d65-2122-497e-831c-917aa1fabd52/bond-cni-plugin/0.log" Sep 11 18:56:43.731877 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:43.731848 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9xtl5_e9819d65-2122-497e-831c-917aa1fabd52/routeoverride-cni/0.log" Sep 11 18:56:43.752999 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:43.752972 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9xtl5_e9819d65-2122-497e-831c-917aa1fabd52/whereabouts-cni-bincopy/0.log" Sep 11 18:56:43.776810 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:43.776782 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9xtl5_e9819d65-2122-497e-831c-917aa1fabd52/whereabouts-cni/0.log" Sep 11 18:56:44.118075 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:44.118045 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-z8mxj_96d752b9-bf5e-4519-b2b8-6dbd832cab19/kube-multus/0.log" Sep 11 18:56:44.180255 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:44.180224 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lhj2z_b644d098-032f-42c9-9839-41ce9eb09844/network-metrics-daemon/0.log" Sep 11 18:56:44.200907 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:44.200880 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lhj2z_b644d098-032f-42c9-9839-41ce9eb09844/kube-rbac-proxy/0.log" Sep 11 18:56:45.137773 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:45.137716 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8tmtk_a688c734-cd8a-40eb-843f-d78e5ec411fd/ovn-controller/0.log" Sep 11 18:56:45.157906 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:45.157872 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8tmtk_a688c734-cd8a-40eb-843f-d78e5ec411fd/ovn-acl-logging/0.log" Sep 11 18:56:45.185048 ip-10-0-134-192 kubenswrapper[2563]: I0911 18:56:45.185019 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8tmtk_a688c734-cd8a-40eb-843f-d78e5ec411fd/kube-rbac-proxy-node/0.log"